CN104469619B - Controller applied to audio device and related operating method - Google Patents

Abstract

The present invention relates to a controller applied to an audio device and a related operation method; the controller can receive a first sound signal and a second sound signal respectively provided by two microphones, and includes an echo cancellation module and a beamforming module. The echo cancellation module can perform echo cancellation on the first sound signal and provide an intermediate signal accordingly, and the beamforming module can perform beamforming processing using the intermediate signal after echo cancellation and the second sound signal that has not been processed by echo cancellation.

Description

Controller applied to audio device and related operating method

technical field

The invention relates to a controller applied to an audio device and a related operation method, in particular to an audio device controller and a related operation method capable of effectively improving the sound collection effect with a low calculation amount.

Background technique

Audio devices capable of collecting and/or playing sounds have played an important role in modern information life; moreover, devices with voice control functions can also be regarded as audio devices. For example, audio devices can cover mobile phones, digital cameras/video cameras, navigation/positioning devices that can pronounce and accept voice control, wearable/handheld/portable computers/e-books/electronic dictionaries/computers, voice-activated TVs, stereos, multimedia players, toys and even interactive art works and more.

Please refer to FIG. 1 , which shows a known audio device 10 capable of playing sound and accepting voice control. The audio device 10 includes microphones 12 a and 12 b , speakers 14 a and 14 b , a controller 20 , an audio output module 23 and a playback module 24 . The microphones 12a and 12b are used to collect sound and convert the collected sound into signals Si_L and Si_R. The signals Si_L and Si_R are transmitted to the controller 20 .

The controller 20 includes a beam forming module 16 , an echo cancellation module 18 and a speech recognition module 22 . The audio output module 23 can provide signals Sp_L and Sp_R as audio source signals, and the playback module 24 can play according to the signals Sp_L and Sp_R, for example, drive the speakers 14a and 14b according to the signals Sp_L and Sp_R, so as to play the signals Sp_L and Sp_R as sounds .

In order to realize the function of voice control, the audio device 10 must focus on the position of the user to collect the voice control commands issued by the user, and prevent the broadcasting of the speakers 14a and 14b from affecting the sound collection, because the sound broadcast by the speakers 14a and 14b will form an echo. and is received by microphones 12a and 12b. In the controller 20 of the conventional audio device 10, the beamforming module 16 uses the signals Si_L and Si_R to perform beamforming processing, and accordingly provides the signal Sm1; the purpose of the beamforming is to enhance a certain focus area in the signal Sm1 sound in the center and suppress sound interference from other non-focused areas. The echo cancellation module 18 performs echo cancellation on the signal Sm1 according to the signal Sp_R, so as to provide the signal Sm2. Then, the voice recognition module 22 can use the signal Sm2 to perform voice recognition, and identify whether there is a voice control command and the content of the command from the signal Sm2, so that the controller 20 can control the audio device 10 accordingly.

As can be seen from FIG. 1 , the known audio device 10 performs echo cancellation after performing beamforming. Under this known architecture, although the controller 20 only needs a single echo cancellation module 18 to reduce the amount of computation, the beamforming will destroy the linear characteristics of the echo and generate a nonlinear signal, so that the echo cancellation module 18 cannot completely eliminate the echo, resulting in Affect the accuracy and recognition rate of speech recognition.

Contents of the invention

To overcome the disadvantages of the prior art, one of the objectives of the present invention is to provide a controller applicable to an audio device. The controller of the present invention can receive a first radio signal and a second radio signal respectively provided by two microphones, and includes an echo cancellation module and a beam forming module. The echo cancellation module performs echo cancellation on the first radio signal and provides an intermediate signal accordingly. The beamforming module is coupled to the echo cancellation module and the second radio signal, and performs beam forming processing on the intermediate signal and the second radio signal to provide an output signal; wherein, the second radio signal is not processed by echo cancellation. The controller can also include a voice recognition module, coupled to the beamforming module, to perform voice recognition on the output signal, and control the audio device according to the result of the voice recognition.

The audio device of the present invention may include one or more speakers, an audio output module and a playback module. The audio output module is used to provide a sound source signal for each speaker, the play module makes each speaker play a corresponding sound according to each audio signal, and the echo cancellation module can perform echo cancellation on the first radio signal according to the sound source signal.

One of the objects of the present invention is to provide an operation method applied to an audio device, comprising: receiving a first sound collection signal and a second sound collection signal from a first microphone and a second microphone respectively, and performing an operation on the first sound collection signal An echo cancellation process is performed to provide an intermediate signal, and a beamforming process is performed according to the intermediate signal and the second radio signal to provide an output signal; wherein the second radio signal is not processed by echo cancellation.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the preferred embodiments are specifically cited below, together with the accompanying drawings, and are described in detail as follows:

Description of drawings

FIG. 1 shows the controller architecture of a known audio device.

Figure 2 shows an audio device and its controller.

FIG. 3 shows an audio device and its controller according to an embodiment of the present invention.

FIG. 4 compares the echo cancellation effect and calculation amount in FIG. 1 to FIG. 3 by way of example.

Fig. 5 shows the flow of the operation method according to an embodiment of the present invention.

Symbol Description

10, 30, 50: Audio installations

12a-12b, 32a-32b, 52a-52b: Microphones

14a-14b, 34a-34b, 54a-54b: Speakers

16, 36, 56: beamforming module

18, 38a-38b, 58: Echo cancellation module

20, 40, 60: Controller

22, 42, 62: speech recognition module

23, 43, 63: audio output module

24, 44, 64: Play module

Si_L/Si_R, Sm1, Sm2, Sp_L/Sp_R, Sm_R/Sm_L, Si_a/Si_b, Sp_a/Sp_b, S1, S2: Signal

100: Process

102-108: Steps

detailed description

Please refer to FIG. 2 , which shows an audio device 30 . The audio device 30 can also play sound and receive voice control, and includes microphones 32 a and 32 b , speakers 34 a and 34 b , a controller 40 , an audio output module 43 and a playback module 44 . The microphones 32 a and 32 b are used to collect sound, and provide electronic signals Si_L and Si_R to transmit to the controller 40 .

The controller 40 includes two echo cancellation modules 38 a and 38 b , a beamforming module 36 and a speech recognition module 42 . The audio output module 43 can provide signals Sp_L and Sp_R as audio source signals, and the playing module 44 controls the speakers 34 a and 34 b according to the signals Sp_L and Sp_R to play the signals Sp_L and Sp_R as sounds.

In order to realize the voice control function, the audio device 30 must also focus on sound collection, and avoid the playback echoes of the speakers 34a and 34b from interfering with sound collection. In the controller 40 of the audio device 30 , the echo cancellation modules 38 a and 38 b respectively cancel the echoes from the signals Si_L and Si_R according to the signals Sp_L and Sp_R, and generate signals Sm_L and Sm_R. Then, the beamforming module 36 uses the signals Sm_L and Sm_R to perform beamforming processing, and accordingly generates the signal Sm2 as an output signal. In this way, the voice recognition module 42 can use the signal Sm2 to perform voice recognition, so that the controller 40 can control the audio device 30 accordingly.

Different from the known technology in FIG. 1 , the controller architecture in FIG. 2 first performs two-way equalized echo cancellation, and then performs beamforming, so as to prevent the echo characteristics from being damaged by the beamforming. However, the two-way equalization echo cancellation shown in FIG. 2 may require a lot of computation.

Please refer to FIG. 3 , which illustrates an audio device 50 according to an embodiment of the present invention. For example, the audio device 50 may be a device capable of playing sound and accepting voice control, such as a voice-activated TV or a voice-activated multimedia player. The audio device 50 may include one or more microphones (such as microphones 52a and 52b ), one or more speakers (such as speakers 54a and 54b ), an audio output module 63 , a playback module 64 and a controller 60 . The microphones 52 a and 52 b are used for sound collection, and respectively convert the collected sounds into electronic signals Si_a and Si_b (which can be regarded as first and second sound collection signals) and transmit them to the controller 60 .

The controller 60 may be a processor or a controller chip, and may also include peripheral supporting circuits and/or hardware of the controller chip, such as volatile and/or non-volatile memory and the like. The controller 60 may include a single echo cancellation module 58 , a beamforming module 56 and a speech recognition module 62 . In the audio device 50, the audio output module 63 can provide signals Sp_a and Sp_b (which can be regarded as audio source signals), and the playback module 64 drives the speakers 54a and 54b according to the signals Sp_a and Sp_b, so as to play the signals Sp_a and Sp_b as corresponding sounds . For example, the audio output module 63 may include an audio codec (audio codec) module for extracting signals of different channels from a stereo audio stream (not shown) as audio signals of different speakers, For example, the signals Sp_a and Sp_b of the speakers 54a and 54b.

The audio device 50 can focus on sound collection and suppress the echo caused by the sound from the speaker. For example, in order to realize the voice control function, the audio device 50 can focus on the position of the user to collect the voice control commands issued by the user, and prevent the sound from the speakers 54a and 54b from affecting the sound collection. In the controller 60, the echo cancellation module 58 is coupled to the microphone 52a, the beamforming module 56 and the audio output module 63, receives the signal Sp_a, uses the reference signal Sp_a to perform echo cancellation on the signal Si_a, and accordingly provides the signal S1 as a intermediary signal. The beamforming module 56 is coupled to the echo cancellation module 58 , the microphone 52 b and the voice recognition module 62 , and can use the signal S1 and the signal Si_b of the microphone 52 b to perform beamforming processing, thereby providing a signal S2 as an output signal. The voice recognition module 62 is coupled to the beamforming module 56 to perform voice recognition on the signal S2, so that the controller 60 can control the audio device 50 according to the voice recognition result.

It can be seen from FIG. 3 that the controller 60 of the present invention arranges the echo cancellation before the beamforming. In this way, the non-linear signal of the beamforming can be prevented from affecting the effect of the echo cancellation, and the beamforming can further prevent the speech recognition rate and accuracy from being interfered. For example, echo cancellation can be performed using the Normalized Least MeanSquare (NLMS, Normalized Least MeanSquare) algorithm. reflection, nonlinear resonance and/or beamforming, etc.), the more difficult it is to use the processed sound source signal to approximate the adaptive filter coefficients of the input echo through the NLMS algorithm. Therefore, if beamforming is placed before echo cancellation, it will make it more difficult for the echo cancellation module to learn the filter coefficients for echo cancellation, which will make it more difficult to cancel the echo. In contrast, the controller architecture of the present invention arranges the beamforming after the echo cancellation, so it can effectively prevent the beamforming from destroying the effect of the echo cancellation.

Furthermore, the controller 60 of the present invention can realize a single echo cancellation module 58, therefore, the computation amount of the controller 60 can be reduced, avoiding the extra computation required by multiple echo cancellations in FIG. 2 . Although the controller 60 only performs echo cancellation on the signal Si_a provided by the microphone 52a, and does not perform echo cancellation on the signal Si_b of the microphone 52b, according to the embodiment of the present invention, the echo in the signal Si_b will still be formed by the beamforming module 56 The processing is suppressed and eliminated. Therefore, overall, the echoes in the signals Si_a and Si_b will not interfere with the recognition rate of speech recognition.

One of the purposes of beamforming is to enhance the sound in the focused area and relatively suppress the sound in the non-focused area; for example, the focused area may be located on the geometric centerline of the microphones 54a and 54b. That is to say, the distance between the focal area and the microphones 54a and 54b is similar, so the sound emitted in the focal area is also similar in the signals Si_a and Si_b. If a sound has different performances in the signals Si_a and Si_b, or If it only appears in one of the signals Si_a and Si_b, it can be judged that it is not the sound from the focus area. In the embodiment of the present invention, although the signal Si_b of the microphone 52b is not echo-cancelled, the echo of the signal Si_b only appears in the signal Si_b input by the microphone 54b, but does not appear in the signal S1 transmitted by the echo cancellation module 58. Therefore, the beamforming module 56 will recognize the sound in the non-focus area; thus, the beamforming process of the beamforming module 56 will filter out the echo of the signal Si_b.

Please refer to FIG. 4 , which is an example to compare the echo cancellation effect and operation amount of the controllers in FIGS. 1 to 3 . In Fig. 4, the echo cancellation effect is quantified by Echo Return Loss Enhancement (ERLE, Echo Return Loss Enhancement); the higher the value, the better the echo cancellation effect. The amount of computation is represented by the clock pulse required for echo cancellation; the lower the value, the less the amount of computation required. As can be seen from FIG. 4 , the controller architecture of the present invention ( FIG. 3 ) can take into account both the echo cancellation effect and the low computation load. Not only is the echo cancellation effect excellent, but the computation load is also very low.

In the embodiment of FIG. 3, the voice recognition module 62 may also be a module with other functions, such as a recording module (for recording the signal S2 to a non-volatile memory), a transmission module (for transmitting the signal S2 to the network) and/or Or an audio processing module, such as an encoding module (encoding the signal S2 into a stream) or a spectrum conversion module (converting the signal S2 into a frequency domain), etc. Each module of the controller 60 may be realized by dedicated hardware, and/or by executing software and/or firmware programs by a hardware processor.

Please refer to FIG. 5 , which illustrates a process 100 according to an embodiment of the present invention, which can be applied to the audio device shown in FIG. 3 . The main steps of the process 100 can be described as follows.

Step 102: Receive a plurality of sound collection signals by multiple microphones, for example, the signals Si_a and Si_b are respectively obtained by the microphones 52a and 52b (FIG. 3).

Step 104: Among the multiple radio signals, perform echo cancellation processing on part of one or more radio signals, and do not undergo echo cancellation processing on the remaining one or more radio signals. For example, in the example of FIG. 3 , the signal Si_a is subjected to echo cancellation processing according to the signal Sp_a to form the signal S1 (intermediate signal), and the signal Si_b is not subjected to the echo cancellation processing.

Step 106 : Perform beamforming processing with the echo-cancelled signal (such as the signal S1 ) and the non-echo-cancelled signal (such as the signal Si_b ), so as to provide an output signal, such as the signal S2 in FIG. 3 .

Step 108: Apply the output signal provided in step 106. For example, voice recognition can be performed on the output signal S2, and the audio device 50 can be controlled according to the voice recognition result.

In summary, the present invention can be generalized as follows: the controller of the present invention can receive a plurality of radio signals provided by a microphone array (may include a plurality of microphones), and perform echo cancellation to part (one or more) of the radio signals wherein The remaining (one or more) radio signals do not need to be processed by echo cancellation; moreover, the radio signals after echo cancellation are integrated with the radio signals without echo cancellation for beamforming processing to achieve focused radio and echo eliminate. In other words, the present invention uses unbalanced echo cancellation for signals provided by different microphones, and then integrates them with beamforming to achieve focused sound collection and echo cancellation. Compared with the known technology, the present invention can prevent the echo cancellation from being affected by the beamforming, and does not need to perform echo cancellation on the microphones of all channels, so excellent echo cancellation effect and simplified calculation can be taken into account.

To sum up, although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be defined by the claims.

Claims (8)

1. A controller applied to an audio device, the controller receives a first radio signal and a second radio signal respectively provided by two microphones, and includes: An echo cancellation module, which performs an echo cancellation process on the first radio signal and provides an intermediate signal accordingly; and A beamforming module performs a beamforming process according to the intermediate signal and the second radio signal to provide an output signal, wherein the second radio signal is not subjected to the echo cancellation process. 2. The controller according to claim 1, wherein the audio device comprises an audio output module and a playback module, and the playback module plays according to an audio source signal output by the audio output module, wherein the echo cancellation The module performs the echo cancellation processing on the first sound receiving signal according to the sound source signal. 3. The controller of claim 1, further comprising: A speech recognition module performs a speech recognition on the output signal. 4. The controller according to claim 3, wherein the audio device is controlled according to the voice recognition result. 5. An operation method applied to an audio device, comprising: Receiving a first radio signal and a second radio signal from a first microphone and a second microphone respectively; performing an echo cancellation process on the first radio signal to provide an intermediate signal; and A beamforming process is performed according to the intermediate signal and the second radio signal to provide an output signal, wherein the second radio signal is not subjected to the echo cancellation process. 6. The operation method according to claim 5, wherein the audio device comprises an audio output module and a playback module, and the playback module plays a sound source signal output by the audio output module, wherein the first The step of performing the echo cancellation processing on the received sound signal and providing the intermediate signal accordingly is carried out according to the sound source signal. 7. The operating method according to claim 5, further comprising: performing a speech recognition on the output signal. 8. The operating method according to claim 7, further comprising: controlling the audio device according to the voice recognition result.