JP4709714B2 - Echo canceling apparatus, method thereof, program thereof, and recording medium thereof - Google Patents

Description

  The present invention relates to an echo canceling apparatus for hands-free communication such as a TV conference and an audio conference, a method thereof, a program thereof, and a recording medium thereof.

FIG. 1 shows an example of a functional configuration of a conventional echo canceling apparatus, and FIG. 2 shows a block diagram when a speaker is represented by a model in which linear characteristics and nonlinear characteristics are connected in parallel.
In the following description, the reproducing means is described as a speaker, and the sound collecting means is described as a microphone. In addition, a voice signal from a far-end speaker (not shown) via the communication network 2 is represented as a received signal x (t), and a voice uttered by the near-end speaker 7 is represented as a speaker voice s (t). The collected speaker voice s (t) is represented as a speaker received signal a (t), and the reproduced sound emitted from the speaker wraps around the microphone, and the collected voice signal is echo signal d (t). The voice signal picked up by the microphone is represented as a received signal y (t), and the transmitted signal or error signal to the far-end speaker is represented as e (t). However, t represents a discrete time.
The conventional echo canceling device 14 includes a variable filter 8, a subtracting unit 10, and a filter coefficient updating unit 12. The conventional echo canceller 14 cancels the echo signal d (t) mixed in the received sound signal y (t) in the loudspeaking call using the speaker 4 and the microphone 6. The input signal of the echo canceler 14 (the sound signal collected by the microphone 6), that is, the sound reception signal y (t) is composed of the echo signal d (t) and the speaker sound reception signal a (t). The conventional echo canceller 14 estimates the echo signal d (t) included in the received sound signal y (t) and subtracts it from the received sound signal y (t), thereby including it in the received sound signal y (t). The echo signal d (t) to be deleted is deleted.

First, a voice signal from a far-end speaker (not shown) is input to the speaker 4 as the received signal x (t) via the communication network 2. If the speaker characteristic is linear, the characteristic is g (t), and the impulse response from the speaker 4 to the microphone 6 is r ₁ (t). The echo signal d (t) can be expressed by the following equation (1).
d (t) = g (t) * r ₁ (t) * x (t) (1)
However, * represents a convolution operation. Next, the impulse response from the near-end speaker 7 to the microphone 6 is expressed as c ₁ (t), and as described above, the sound from the near-end speaker 7 is s (t), so the received signal y (T) can be expressed by the following formula (2).
y (t) = g (t) * r ₁ (t) * x (t) + c ₁ (t) * s (t) (2)
Here, what is required of the echo canceller 14 is to cancel the echo signal d (t) included in the received sound signal y (t). That is, the component of the first term g (t) * r ₁ (t) * x (t) on the right side of Expression (2) is eliminated.
The variable filter 8 convolves the estimated impulse response (hereinafter referred to as a pseudo impulse response h (t)) with the received signal x (t), and the pseudo echo signal h (t) * x ( t) is output. The subtraction unit 10 subtracts the pseudo echo signal h (t) * x (t) from the sound reception signal y (t), and the subtraction unit 10 uses the transmission signal e (t) from which the echo signal d (t) is deleted. Output. If the transmission signal e (t) is expressed by an equation, the following equation (3) is obtained.
e (t) = {g (t) * r ₁ (t) −h (t)} * x (t) + c ₁ (t) * s (t) (3)

The filter coefficient (pseudo impulse response h (t)) of the variable filter is updated by the filter coefficient updating unit 12 using the received signal x (t), the transmitted signal e (t), and the like. For this update, a learning identification (NLMS: Normalized-Lean-Squares) algorithm, a projection algorithm, a sequential least-square (RecursiveLeastSquare) algorithm, an LMS (LeastMeanSquare) algorithm, or the like is used. For example, when the learning identification algorithm is used, the update formula is expressed by the following formula (4).
H (t + 1) = H (t) + a.X (t) .e (t) / X (t) ^TX (t) (4)
Here, a is a preset step size, takes a value of 0 <a <2, ^AT represents a transposed matrix of vector A, and H (t) is a filter coefficient H (t) at time t. = (H (0), h (1),..., H (L-1)) ^T , L is the tap length of the filter of the variable filter 8, and X (t) is the received signal x at time t. It is a vector for L samples of (t), and is represented by X (t) = (x (t-0), x (t-1), ..., x (t-L + 1)) ^T.

As described above, the filter coefficient updating unit 12 updates the filter coefficient of the variable filter 8 from the reception signal x (t) and the transmission signal e (t) using the update expression represented by the above expression (4) or the like. . The variable filter 8 sequentially filters the received signal x (t) with the updated filter coefficient. By the above processing, the echo signal d (t) included in the sound reception signal y (t) is deleted. The transmission signal e (t) from which the echo signal d (t) has been deleted is output to the far end speaker (not shown) through the communication network 2. The details of the conventional echo canceller are described in Patent Document 1.
Patent No. 2602750

In the conventional echo canceller 14, the echo signal d (t) may not be sufficiently cancelled. The reason for this is thought to be as follows.
That is, in general, the speaker characteristic has a nonlinear characteristic such that the output reaches a peak with respect to an input signal (received signal x (t)) having a large amplitude. It can be considered separately. That is, as shown in FIG. 2, the speaker characteristics can be expressed by the parallel characteristics of the linear response portion with the impulse response of the speaker 4 as g (t) and the nonlinear portion as the function f (·). Considering the speaker characteristics, the echo signal d (t) can be expressed by the following equation (5).
d (t) = g (t) * r ₁ (t) * x (t) + r ₁ (t) * f (x (t)) (5)
The sound reception signal y (t) can be expressed by the following equation (6).
y (t) = d (t) + c ₁ (t) * s (t)
= G (t) * r ₁ (t) * x (t) + r ₁ (t) * f (x (t)) + c ₁ (t) * s (t) (6)
Accordingly, the transmission signal e (t) is expressed by the following equation (7).
e (t) = {g (t) * r ₁ (t) −h (t)} * x (t) + r ₁ (t) * f (x (t)) + c ₁ (t) * s (t) (7)
However, in the conventional echo canceller 14, the only echo that can be canceled is the echo that reaches the microphone 6 through the linear echo path, and the nonlinear echo cannot be canceled. That is, only g (t) * r ₁ (t) * x (t), which is the first term on the right side of the above formula (6), is deleted, and r ₁ (t) * f (x (x), which is the second term. The component t)) cannot be eliminated, and a non-linear echo signal is mixed in the transmission signal e (t). Therefore, there is a problem that the echo signal cannot be sufficiently erased when using a speaker having a strong nonlinearity.

According to the present invention, the received signal is converted into a reproduced sound by the reproducing means, and is emitted, and a signal (hereinafter referred to as an echo signal) in which the speaker received signal from the sound collecting means for the speaker and the reproduced sound are circulated. In the echo canceling apparatus that removes the echo signal from the received sound signal and outputs it as a transmitted signal, a first pseudo echo signal is generated from the received signal, and the plurality of sound collecting means A second pseudo echo signal is generated from the received sound signal, and the first pseudo echo signal and the second pseudo echo signal are subtracted from the received sound signal from the speaker sound collecting means, A transmission signal is output; at least the reception signal and the transmission signal are input; a filter coefficient of the first variable filter is updated; and at least a reception signal from the plurality of sound collection units; From the transmission signal, the second possible Update the filter coefficients of the filter, to detect the level of the received signal, if the detected level is lower than a predetermined threshold, without operating the second variable filter, the output of the second variable filter Is set to 0 .

  With the above configuration, even when a loudspeaker or the like with strong nonlinearity is used, the ability to remove echo signals can be enhanced, and the quality of the transmitted signal can be enhanced.

  The best mode for carrying out the present invention will be described below.

An example of the functional configuration of the present invention is shown in FIG. 3, and the main processing flow of the present invention is shown in FIG. The same functional components as those in FIG. 1 are denoted by the same reference numerals, and redundant description is omitted. The same applies to the following. The main microphone 61 is for collecting the voice of the speaker by the speaker sound collecting means, and the sound collecting means represents all microphones including the main microphone.
The echo canceller 40 includes a first variable filter 24, a second variable filter 26m (m = 2,..., M), an adder 36, a subtractor 38, a first filter coefficient update unit 34, a second filter And the filter coefficient updating unit 32. The number of microphones is a plurality of M (M ≧ 2), and the microphones are represented as 6 m (m = 1,..., M).
The input signal of the echo canceling device 40 of the present invention is an echo signal in which a reproduction signal from the speaker 4 wraps around and an audio signal from the near-end speaker 7, that is, a received sound signal y _m collected by the microphone 6m. (T). The output signal is a transmission signal e (t) to a far-end speaker (not shown). The echo canceller 40 cancels the echo signal d (t) to facilitate conversation. Each input signal of the echo canceller 40 is converted from an analog signal to a discrete time signal (digital signal) by an AD converter (not shown), and each output signal of the echo canceler 40 is discretely converted by a DA converter (not shown). A time signal (digital signal) is converted into an analog signal.

The received signal x (t) is reproduced as reproduced sound by the speaker 4. The details of the signal flow in the speaker 4 are shown in FIG. 2 as described above. As described above, the speaker characteristic has a non-linear characteristic such that the output reaches a peak with respect to the input of a signal having a large amplitude. Therefore, the speaker characteristic is considered as being divided into a linear characteristic and a non-linear characteristic. The impulse response g (t) of the linear characteristic part and the characteristic of the nonlinear part are expressed as a function f (·). Further, the impulse response from the speaker to each microphone 6m (m = 1,..., M) is represented as r _m (t), and the impulse response from the near-end speaker 7 to each microphone 6m is represented as c _m (t). Denoting, received sound signals y _m at each microphone 6 m _(t) can be expressed by the following equation (8).
y _m (t) = g (t) * r _m (t) * x (t) + r _m (t) * f (x (t)) + c _m (t) * s (t) (8)
In the first embodiment, the speaker received signal selection means (not shown) has a microphone 6m whose time average value of the ratio of the echo signal level talker received signal level from the plurality of microphones 6m is smaller than that of the other microphones 6m. Is selected as the main microphone 61. (Step S2).

The echo sound signal selection means (not shown) is a case where one or more microphones having a large time average value of ratios of echo signal level dialogue person sound signal levels are selected from the plurality of microphones 6m. However, in this case, the main microphone 61 is not selected (step S2).
As an example of the selection method of the speaker reception signal selection means, as shown in FIG. 3, one microphone is set as the main microphone 61, the direction of high sensitivity of the main microphone 61 is directed toward the near-end speaker 7, It is conceivable that all other microphones are sub-microphones 6m (m = 2,..., M) and the direction of high sensitivity of the sub-microphone 6m is directed toward the speaker 4. The direction of sensitivity of the main microphone 61 and the sub microphone 6m will be described in detail later. In this case, the near-end speaker 7 emits voice toward the main microphone 61.
The received sound signal y _m (t) (m = 1,..., M) collected by the main microphone 61 and the sub microphone 6m can be expressed in the same manner as the above equation (8). The received sound signal y _m (t) of the sub microphone 6m is input to the corresponding second variable filter 26m (m = 2,..., M). The second variable filter 26m convolves the received signal y _m (t) with the second filter coefficient h _m (t), that is, calculates the following equation (9) to calculate the second pseudo echo signal β _m. (T) (m = 2, ..., M) is generated (step S4).

β _m (t) = y _m (t) * h _m (t) (9)
The second pseudo echo signals β _m (t) are each input to the adder 36.

On the other hand, the received signal x (t) is input to the first variable filter 24. The first variable filter 24 convolves the received signal x (t) with the first filter coefficient h ₁ (t), that is, calculates the following equation (10) to obtain the first pseudo echo signal α (t ) Is generated (step S4).
α (t) = x (t) * h ₁ (t) (10)
The first pseudo echo signal α (t) and the second pseudo echo signal β _m (t) are input to the adder 36, and α (t) + Σ _{i = 2} ^M β _i (t) is calculated. The added signal α (t) + Σ _{i = 2} ^M β _i (t) is input to the subtracting unit 38.

On the other hand, the received sound signal y ₁ (t) collected by the main microphone 61 is input to the subtractor 38. The subtracting unit 38 subtracts the audio signal α (t) + Σ _{m = 2} ^M β _m (t) from the adding unit 36 from the received sound signal y ₁ (t), that is, the following equation (11) is calculated. The transmission signal e (t) is output (step S6).
e (t) = y ₁ (t) −α (t) −Σ _{i = 2} ^M β _i (t)
_{= Y 1 (t) -h 1} (t) * x (t) -Σ i = 2 M y i (t) * h i (t)
(11)
From the above equation (8)

y ₁ (t) = g (t) * r ₁ (t) * x (t) + r ₁ (t) * f (x (t)) + c ₁ (t) * s (t) (12)
y _i (t) = g (t) * r _i (t) * x (t) + r _i (t) * f (x (t)) + c _i (t) * s (t) (13)
Therefore, when Expressions (12) and (13) are substituted into Expression (11), the following Expression (14) is obtained.

e (t) = g (t) * r ₁ (t) * x (t) + r ₁ (t) * f (x (t)) + c ₁ (t) * s (t)
_{-H 1 (t) * x (} t) -Σ i = 2 M h i (t) * {g (t) * r i (t) * x (t)
_{+ R i (t) * f} (x (t)) + c i (t) * s (t)}
_{= {G (t) * r} 1 (t) -h 1 (t) -Σ i = 2 M h i (t) * g (t) * r i (t)} * x (t)
+ {R ₁ (t) −Σ _{i = 2} ^M h _i (t) * r _i (t)} * f (x (t))
+ {C ₁ (t) −Σ _{i = 2} ^M h _i (t) * c _i (t)} * s (t) (14)

Here, the first term of the expression (14) {g (t) * r ₁ (t) −h ₁ (t) −Σ _{i = 2} ^M h _i (t) * g (t) * r _i (t) } * X (t) is a linear acoustic echo component, and {g (t) * r ₁ (t) −h ₁ (t) −Σ _{i = 2} ^M h _i (t) * g (t) * r By setting h ₁ (t) and h _m (t) where _i (t)} = 0, linear acoustic echo can be eliminated.
The second term {r ₁ (t) −Σ _{i = 2} ^M h _i (t) * r _i (t)} * f (x (t)) in Equation (14) is a nonlinear acoustic echo component, By setting h _m (t) where r ₁ (t) −Σ _{i = 2} ^M h _i (t) * r _i (t) = 0, nonlinear acoustic echo can be eliminated.
The speaker received signal selecting means and the echo received signal selecting means are not necessarily provided. In this case, a received sound signal obtained by picking up the speaker voice from the main microphone 61 without a speaker received signal selection means is input to the subtractor 38, and a sub microphone 6m without an echo received signal selection means is provided. Is received by the second variable filter 26m.
The third term {c ₁ (t) −Σ _{i = 2} ^M h _i (t) * c _i (t)} * s (t) in the equation (14) is a speech component of the near-end speaker 7, and In addition to the speech component c ₁ (t) * s (t) of the near-end speaker 7 picked up by the microphone 61, Σ _{i = 2} ^M h _i (t) * c _i (t) * s which is a degradation component (T) exists.

Also, Σ _{i = 2} ^M h _i (t) * c _i (t) * s (t), which is a degradation component of the speech of the near-end speaker 7 included in the third term of the above formula (14), becomes large. If this happens, the quality of the transmitted voice will deteriorate. In order to prevent this, the above-described echo sound signal selection means is used. An example will be described below. For simplification, the case where there is only one main microphone 61 and one sub microphone 62 will be described.
In order to prevent the deterioration of the voice of the near-end speaker 7 based on {c ₁ (t) −Σ _{i = 2} ^M h _i (t) * c _i (t)} * s (t), h ₂ (t ) And c ₂ (t) must be reduced. For this purpose, the above-mentioned speaker received signal selecting means and echo received signal selecting means may be provided. For example, the arrangement of the main microphone 61 and the sub microphone 62 can be changed. As shown in FIG. 5, unidirectional microphones are used as the main microphone 61 and the sub microphone 62. The high sensitivity portion 61 a of the main microphone 61 is directed toward the near-end speaker 7, and the low sensitivity portion 61 b is directed toward the speaker 4. Further, the highly sensitive portion 62 a of the sub microphone 62 is directed to the speaker 4, and the low sensitive portion 62 b is directed to the near-end speaker 7. With this arrangement, in the main microphone 61, the amplitude of c ₂ (t) is smaller than the amplitude of c ₁ (t). Furthermore, since the amplitude of r ₁ (t) is smaller than the amplitude of r ₂ (t), the amplitude of the filter h ₂ (t) for canceling the echo is also reduced. Is because the second term in the above equation _(14), is set to _{r 1 (t) -h 2 (} t) * r 2 (t) = 0 and becomes _h 2 (t). With this arrangement, the degradation component h ₂ (t) * c ₂ (t) * s (t) of the voice of the near-end speaker 7 can be reduced.
In the first embodiment, as shown in FIG. 6, the high-sensitivity portion 61 a of the main microphone 61 is directed to the near-end speaker 7, and the high-sensitivity portion 6 ma of the microphone 6 m is directed to the speaker 4. The deterioration component of the voice of the person 7 can be reduced.

Next, a method for obtaining the coefficients h ₁ (t) and h _m (t) (m = 2,..., M) of the variable filter for suppressing the echo signal d (t) will be described. The filter coefficient h ₁ (t) is updated by the first filter coefficient update unit 34, and h _m (t) is updated by the second filter coefficient update unit 32 (step S8). The filter coefficients h ₁ (t), h _m (t) (m = 2,..., M) are sequentially updated so that the mean square of the echo components included in the transmission signal e (t) is reduced. Is obtained. However, the initial value of the filter coefficient is given in advance as an arbitrary value.
Typical algorithms for sequentially updating the filter coefficients include a learning identification (NLMS: Normalized-Lean-Squares) algorithm, a projection algorithm, a sequential least-squares (RecursiveLeastSquare) algorithm, or an LMS (LeastMeanSquare) algorithm. is there. Each algorithm will be briefly described below.

NLMS algorithm The NLMS algorithm is an algorithm for updating the filter coefficient using only the latest observed transmission signal e (t) of one sample, and has a feature that the amount of calculation is small. The update formula by the first filter coefficient update unit 34 is expressed by the following formula (15), and the update formula by the second filter coefficient update unit 32 is expressed by the following formula (16).
_{H 1 (t + 1) =} H 1 (t) + a 1 · X (t) · e (t) / {X (t) T X (t) + Σ i = 2 M Y i (t) T Y i (t) } (15)
H _m (t + 1) = H _m (t) + a _m · Y _m (t) · e (t) / {X (t) ^T X (t) + Σ _{i = 2} ^M Y _i (t) ^T Y _i (t )} (16)
Here, H ₁ (t), H _m (t) (m = 2,..., M) are filter coefficient vectors for the received signal x (t) at time t, and H _m (t) = ( h _m (0), h _m (1),..., h _m (L−1)) ^T (m = 1,..., M), where L is the number of taps. a ₁ and a _m are pre-set NLMS algorithm step sizes,
0 <a ₁ and a _m <2 are satisfied. Y (t) is a vector of L samples of the transmission signal y (t) at time t, and Y _m (t) = (y _m (t−0), y _m (t−1),. , Y _m (t−L + 1)) ^T.
Further, the denominator of the right side of the equation (4), comparing the denominator of the right side of the equation (15) (16), in the denominator of the right side of the equation (15) (16), extra Σ _{i =} ^{2 M} Y _i (t) ^T Y _i (t) is added. Σ _{i = 2} ^M Y _i (t) ^T Y _i (t) is the sum of the powers of the received sound signals y _m (t) collected by the microphones 6m. By adding the sum of the power of the received sound signal y _m (t) to the denominator, the divergence of the filter coefficient can be prevented.

Similar to the NLMS algorithm, the LMS algorithm is an algorithm for updating the filter coefficient using only the latest observed transmission signal e (t) of one sample, and has a feature that the amount of calculation is small. The update formula of the LMS algorithm can be expressed by the following formulas (17) and (18).
H ₁ (t + 1) = H ₁ (t) + b · X (t) · e (t) (17)
H _m (t + 1) = H _m (t) + b _m · Y _m (t) · e (t) (18)

Projection algorithm The projection algorithm is an algorithm for updating the filter coefficient using the transmission signal e (t) for the past u samples. The projection algorithm is an algorithm whose basic idea is to remove the correlation between adjacent input signal vectors. The projection algorithm has a feature that the amount of calculation is larger than the above-described NLMS algorithm, but the convergence speed of the transmission signal e (t) is high. The first filter coefficient h ₁ (t) and the second filter coefficient h _m (t) (m = 2,..., M) are updated by the following equation (19). In addition, the following formula | equation (19) is an update formula of a secondary filter coefficient.

However, the vector u (t) can be expressed by the following equation (20).

ただし、ベクトルＫ（ｔ）は以下の式（２２）で表すことができる。

RLS algorithm RLS algorithm is an algorithm for updating filter coefficients using all past transmission signals e (t), and the convergence speed of the transmission signal e (t) is faster than the projection algorithm described above. The amount of calculation is large. The RLS algorithm is to obtain a filter coefficient vector H ^ (t) that approximates all past input / output relationships with a least square error. The first filter coefficient h ₁ (t) and the second filter coefficient h _m (t) (m = 2,..., M) are updated by the following expression (21).

However, the vector K (t) can be expressed by the following equation (22).

ただし、λは忘却係数であり、０≦λ≦１を満たす係数である。
これらのアルゴリズムの詳細については、「音響エコーキャンセラのための適応信号処理の研究 牧野昭二 博士論文 １９９３ 東北大学」に記載されている。
また第１のフィルタ係数更新部３４および、第２のフィルタ係数更新部３２の更新アルゴリズムは各々違うアルゴリズムを用いても良い。 However, the vector P (t) is defined as an inverse matrix of the covariance matrix E [x (t) x (t) ^T ] of the input signal, and is an L × L square matrix using the tap number L. The vector P (t) satisfies the following expression (23).

However, λ is a forgetting factor, which is a factor satisfying 0 ≦ λ ≦ 1.
Details of these algorithms are described in "Research on Adaptive Signal Processing for Acoustic Echo Canceller, Dr. Shoji Makino Doctoral Dissertation 1993 Tohoku University".
Also, different algorithms may be used for the update algorithms of the first filter coefficient update unit 34 and the second filter coefficient update unit 32, respectively.

By applying these update algorithms, the first filter coefficient updating unit 34 and the second filter coefficient updating unit 32 apply the first filter coefficient h ₁ (t) until the transmission signal e (t) converges. The second filter coefficient h _m (t) (m = 2,..., M) is updated (step S8). The filter coefficients can be updated using these algorithms as well in the second to fourth embodiments described below.
With the functional configuration of the first embodiment, it is possible to cancel both the nonlinear acoustic echo generated when the speaker 4 has nonlinearity and the like, and the linear acoustic echo, thereby realizing high canceling performance. I can do it.

[Experimental result]
An experimental result for comparing the effect by applying the conventional echo canceling device 14 and the echo canceling device 40 of the second embodiment to an actual hands-free device will be described. FIG. 7 shows the arrangement of the echo canceller 40, the speaker 4, the near-end speaker 7 and the like described in the first embodiment used in this experiment. One main microphone and one sub microphone were used. The main microphone 61, the sub microphone 62, the speaker 4, and the near-end speaker 7 are arranged so that the straight line connecting the main microphone 61 and the sub microphone 62 and the straight line connecting the speaker 4 and the near-end speaker 7 are orthogonal to each other. Arrange. The distance between the main microphone 61 and the sub microphone 62 is 2 cm, the distance between the speaker 4 and the near-end speaker 7 is 50 cm, the distance between the speaker 4 and the main microphone 61 is 5 cm, and the diameter of the speaker 4 is 4 cm. The speaker characteristic is a sigmoid function that simulates the characteristic of clipping with an excessive input, and the spatial responses c ₁ (t), c ₂ (t), r ₁ (t), and r ₂ (t) are measured with the arrangement shown in FIG. Impulse response.

The high sensitivity portion 41 a of the main microphone 61 is directed toward the near-end speaker, and the high sensitivity portion 62 a of the sub microphone 62 is directed toward the speaker 4. The sampling frequency was 16 kHz, the reverberation time was 200 ms, the number of taps of the first variable filter 24 and the second variable filter 26 was 512 taps, and the adaptive algorithm used was the NLMS algorithm.
8 and 9 show the experimental results. FIG. 8 is a graph showing the amount of echo cancellation when white noise is input to the received signal x (t). The larger the value, the more echoes can be canceled. The horizontal axis represents time (s), and the vertical axis represents the amount of echo cancellation at that time. A thin line indicates the amount of echo erasure by the conventional echo erasing device 14, and a thick line indicates the amount of echo erasure by the echo erasing device 40 described in the first embodiment.
It can be understood that the maximum amount of echo cancellation when using the conventional echo cancellation apparatus 14 is about 20 dB. This is because the non-linear acoustic echo generated due to the non-linearity of the speaker 4 cannot be canceled by the echo canceling device 14 described in the related art.

On the other hand, it can be understood that the amount of echo cancellation when the echo cancellation apparatus 40 described in the first embodiment is used is about 40 dB. This is because the nonlinear acoustic echo in the echo canceling apparatus 40 described in the second embodiment can also be canceled.
FIG. 9 is a graph showing an impulse response to the near-end speaker 7. The horizontal axis represents the frequency (Hz), and the vertical axis represents the impulse response c ₁ (t) from the near-end speaker 7 to the main microphone 61. A thin line indicates before processing of the echo canceller 40, and a thick line indicates after processing of the echo canceller 40. It can be understood from the graph of FIG. 8 that there is almost no difference even when the impulse response c ₁ (t) before and after the processing is compared. From the above, it can be seen that in Example 1, the degradation component of the voice of the near-end speaker 7 is small.
From the above description, according to the first embodiment, both the linear acoustic echo and the nonlinear acoustic echo generated due to the nonlinearity of the speaker 4 can be eliminated, and a high echo cancellation method can be realized. Furthermore, it is possible to realize high-quality sound collection with little deterioration of the voice of the near-end speaker 7.

In the first embodiment, the main microphone 61 directs the high-sensitivity portion 61a toward the near-end speaker 7, directs the low-sensitivity portion toward the speaker 4, and the sub microphone 62 directs the high-sensitivity portion 62a toward the speaker 4, It has been explained that the deterioration component of the voice of the near-end speaker 7 can be reduced by directing the low-frequency portion toward the near-end speaker 7. The second embodiment is a functional configuration example in which the speaker reception signal selection unit is a main beamformer and the echo reception signal selection unit is a sub-beamformer. In the second embodiment, the main beam former and the sub beam former are used to reduce the degradation component of the voice of the near-end speaker 7.
FIG. 10 shows a functional configuration example of the second embodiment. The echo canceling apparatus 40 of the second embodiment is added with the sub beam former 50 and the main beam former 52 as compared with the echo canceling apparatus 40 described in the second embodiment. In this embodiment, the microphones 6m are not divided into main microphones (speaker sound collecting means) and sub microphones, and all the microphones 6m operate in parallel.

The sub beamformer 50 includes a sub fixed filter 50m (m = 1,..., M) and a sub adder 500, and the main beamformer 52 includes a main fixed filter 52m (m = 1,. ) And a main adder 520.
The main beamformer 52 increases sensitivity toward the near-end speaker 7 and decreases sensitivity to the speaker 4. Further, the sub-beamformer 50 increases sensitivity to the speaker 4 and decreases sensitivity to the near-end speaker 7. By using the main beamformer 52 and the sub-beamformer 50, it is possible to create a portion having high directivity and a portion having low directivity in an arbitrary direction, and it can be applied to various speaker and microphone arrangements.
In the description of the second embodiment, the description will be made in the frequency domain ω for simplification. The transfer function from the near-end speaker to the microphone 6 _m is C _m (ω), the transfer function from the speaker 4 to the microphone 6 _m is R _m (ω), and the m-channel main fixed filter 52 m in the main beamformer 52 is Each fixed filter coefficient is P _m (ω), and the fixed filter coefficient of the sub fixed filter 50 m of the sub beam former 50 is Q _m (ω).
Each fixed filter coefficient of the sub fixed filter 50m and the main fixed filter 52m is fixed from a predetermined value. Hereinafter, the design of the fixed filter coefficient will be described.

What is required of the main beamformer 52 is to pick up the voice s (ω) of the near-end speaker 7 and suppress the reproduced sound from the speaker 4. When these conditions are expressed by equations, the following equations (24) and (25) are obtained.
Σ _{i = 1} ^M C _i ′ (ω) · P _i (ω) = D (ω) (24)
Σ _{i = 1} ^M R _i ′ (ω) · P _i (ω) = 0 (25)
Here, D (ω) is a target impulse response. The target impulse response is, for example, an impulse response in which the amplitude value is a fixed value and the phase is a linear phase (fixed delay in the time domain). C _m ′ (ω) and R _m ′ (ω) set in advance the theoretical response of the direct wave calculated from the arrangement of the microphone 6m, the speaker 4, and the near-end speaker 7. The same applies to the following equations (26) and (27). A fixed filter coefficient P _m (ω) that satisfies the above equations (24) and (25) may be set. Next, what is required of the sub-beamformer 50 is to suppress the voice s (ω) of the near-end speaker 7 and collect the reproduced sound from the speaker 4. When these conditions are expressed by equations, the following equations (26) and (27) are obtained.
Σ _{i = 1} ^M C _i ′ (ω) · Q _i (ω) = 0 (26)
Σ _{i = 1} ^M R _i ′ (ω) · Q _i (ω) = K (ω) (27)
However, K (ω) is a target impulse response. The target impulse response is, for example, an impulse response in which the amplitude value is a fixed value and the phase is a linear phase (fixed delay in the time domain).
Other processes are the same as those in the first embodiment.

  As described above, if the main beamformer 52 and the sub-beamformer 50 are set, the sensitivity of the main beamformer 52 in the direction of the near-end speaker 7 is increased in the arrangement of the arbitrary microphone 6m and the speaker 4, and the speaker 4 The sub-beamformer 50 can reduce the sensitivity to the near-end speaker 7 by reducing the sensitivity to the near-end speaker 7 by preventing the deterioration of the near-end speaker voice.

The functional configuration example of this embodiment is obtained by adding the reception detector 60 and the switch 62m (m = 2,... M) to the echo canceller 40 described in the first embodiment. Each of the switches 62m is connected to each of the second variable filters 26m.
FIG. 11 shows a functional configuration example of the third embodiment. The audio signal j (t) from the communication network 2 is input to the reception detection unit 60 in addition to the first variable filter 24. The reception detection unit 60 observes the level of the audio signal j (t) and detects a certain section of the reception signal x (t). In the detection, for example, a given threshold value is compared with the level (power) of the audio signal j (t) in advance, and if the audio signal j (t) is larger, the received signal x (t) is included in that section. It is judged that
In a section in which the reception signal x (t) is included, the reception detection unit 60 turns on the switch 62m and connects the second variable filter 26m to the addition unit 36, respectively. In a section in which the reception signal x (t) is not included, the reception detection unit 60 turns off the connection of the switch 62m and disconnects the second variable filter 26m from the addition unit 36, respectively.

By these processes, the connection of the switch 62m is turned off in the section where the near-end speaker 7 speaks and the voice signal j (t) does not include the received signal x (t). The second pseudo echo signal q (t) from the variable filter 26 is not output to the adding unit 36, that is, the voice of the near-end speaker passing through the sub microphone 6m is blocked, and the transmission signal e (t) The near-end speaker's voice is no longer degraded.
Further, since the switch 62m is on in the section where the received signal x (t) is included in the voice signal j (t), the echo signal d (t) is deleted as described in the second embodiment. .
Further, in a section in which the received signal x (t) is included and a so-called double talk section in which the near-end speaker 7 is speaking, the switch 62m is turned on to cancel the echo signal d (t). In this double talk section, the voice of the near-end speaker 7 is deteriorated, but at the time of double talk, both the voice signal s (t) and the received signal x (t) can be heard from the near-end talker 7. Since the masking effect makes it difficult to hear sound deterioration, the perception of deterioration is reduced.

The functional configuration example of the fourth embodiment shown in FIG. 12 is obtained by adding the reception detection unit 60 described in the third embodiment to the echo canceller 40 described in the second embodiment. As in the third embodiment, the reception signal x (ω) is input to the reception detection unit 60, and in the section where the reception signal x (ω) is included in the voice signal j (ω) from the communication network 2, the switch 62 is turned on. The connection is turned on, and the echo signal d (ω) is deleted. Further, in a section where the received signal x (ω) is not included in the voice signal j (ω) from the communication network 2, the connection of the switch 62 is turned off and the voice of the near-end speaker 7 does not deteriorate.
In addition to the above embodiments, the echo canceller according to the present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the echo canceling device is not only executed in time series according to the order of description, but may be executed in parallel or individually as required by the processing capability of the device that executes the processing. .
Further, when the processing in the echo canceller of the present invention is realized by a computer, the processing contents of the functions that the echo canceller should have are described by a program. Then, by executing this program on a computer, the processing function in the echo canceller is realized on the computer.

The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like is used as an optical disc, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Recordable). ) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable Programmable-Read Only Memory), etc. can be used.
The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Further, the above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).
In this embodiment, the echo canceling apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

The block diagram which shows the function structural example of the system of a prior art. The figure which shows the flow of the reception signal x (t) in the speaker 4 of a prior art and this invention. 1 is a block diagram showing a functional configuration example of a system according to Embodiment 1 of the present invention. The flowchart which shows the flow of the main processes of Example 1 of this invention. The figure which shows arrangement | positioning of the main microphone 61 and the submicrophone 62 for suppressing the degradation component of the audio | voice of the near-end speaker 7 in Example 1 or 3 of this invention. FIG. 6 is a block diagram showing an example of a functional configuration when the main microphone 61 and the sub microphone 62 used in the present invention are arranged as shown in FIG. 5. FIG. 6 is an experimental layout diagram showing the difference in effect between the conventional echo canceling device 14 and the echo canceling device 40 of the present invention. The graph which shows the echo cancellation performance of the echo cancellation apparatus 14 of a prior art, and the echo cancellation apparatus 40 of this invention. The figure which shows the impulse response _cm (t) with respect to the near-end speaker 7 before and after the process by the echo cancellation apparatus 40 of this invention. The block diagram which shows the function structural example of the system of Example 2 of this invention. The block diagram which shows the function structural example of the system of Example 3 of this invention. The block diagram which shows the function structural example of the system of Example 4 of this invention.

Claims (12)

The reception means converts the received signal into a reproduced sound, utters, and a signal (hereinafter referred to as a speaker received signal) in which the near-end speaker sound is received by the speaker sound collecting means and the reproduced sound are In an echo canceller that removes the echo signal from a received signal consisting of a signal received by the speaker sound collection means (hereinafter referred to as an echo signal) and outputs the signal as a transmitted signal.
A first variable filter that receives the received signal and generates a first pseudo echo signal;
By one or more echo YoOsamu sound hand stage, received sound has been received signals are input, a second variable filter for generating a second pseudo echo signal,
A subtracting unit that subtracts the first pseudo echo signal and the second pseudo echo signal from the received sound signal at the sound collecting means for the speaker and outputs the transmission signal;
A first filter coefficient updating unit that receives at least the reception signal and the transmission signal and updates a filter coefficient of the first variable filter;
A second filter coefficient updating unit that receives at least a sound reception signal from the echo sound collection means and the transmission signal, and updates a filter coefficient of the second variable filter ;
When the level of the received signal is detected and the detected level is smaller than a predetermined threshold, the second variable filter is not operated and the output of the second variable filter is set to 0. And an echo canceller. The echo canceller according to claim 1,
Further, a received sound signal having a time average value of the ratio of the echo signal level dialogue person sound signal level smaller than that of the other received sound signals is selected from the plurality of sound collecting means and input to the subtracting unit. Speaker receiving signal selection means;
A sound reception signal having a time average value of the ratio of the echo signal level to the conversation person sound signal level larger than that of the other sound reception signals is selected from the plurality of sound collection means and input to the second variable filter. An echo canceling device comprising: an echo sound receiving signal selecting means for performing the operation. The echo canceller according to claim 2, wherein
The speaker received signal selection means has high sensitivity in the direction of the speaker, and from one sound collection means (hereinafter referred to as main sound collection means) as the speaker sound collection means among the plurality of sound collection means. Is a means of obtaining a sound pickup signal of
The echo canceling device, wherein the echo sound receiving signal selecting means is means for obtaining a collected sound signal from a sound collecting means other than the main sound collecting means and having high sensitivity in the reproducing means direction. The echo canceller according to claim 2, wherein
The speaker received signal selection means is a main beamformer that receives the received sound signals from all of the plurality of sound collecting means and suppresses the component of the echo signal in the received sound signals,
The echo sound receiving signal selecting means is a sub-beamformer that receives the sound receiving signals from all the plurality of sound collecting means and suppresses the components of the speaker sound receiving signals in the sound receiving signals. Echo canceling device. In the echo canceller according to any one of claims 1 to 4 ,
The update of the filter coefficients of the first filter coefficient update unit and the second filter coefficient update unit is performed by a learning identification (NLMS: Normalized Last-Mean-Squares) algorithm, a projection algorithm, or a sequential least squares (Recursive Last Square) algorithm. Alternatively, an echo canceller that updates sequentially using a LMS (Least Mean Square) algorithm. The reception means converts the received signal into a reproduced sound, utters, and a signal (hereinafter referred to as a speaker received signal) in which the near-end speaker sound is received by the speaker sound collecting means and the reproduced sound are In an echo canceling method for removing the echo signal from a received signal consisting of a signal received by a speaker sound pickup means (hereinafter referred to as an echo signal) and outputting the signal as a transmitted signal,
A first pseudo-echo signal generation process in which a first variable filter generates a first pseudo-echo signal from the received signal;
Second variable filter, by one or more echo YoOsamu sound hand stage, a second pseudo echo signal generating process of generating a second pseudo echo signal from the received sound has been received signals,
A subtracting step of subtracting the first pseudo echo signal and the second pseudo echo signal from a sound reception signal from the speaker sound collecting unit and outputting the transmission signal;
A first filter coefficient updating means for updating a filter coefficient of the first variable filter from at least the received signal and the transmitted signal;
A second filter coefficient updating step in which the second filter coefficient updating means updates the filter coefficient of the second variable filter from at least the sound reception signal from the echo sound collecting means and the transmission signal; ,
The reception detection means detects the level of the reception signal, and when the detected level is smaller than a predetermined threshold, the second variable filter is not operated and the output of the second variable filter is set to 0. An echo detection method characterized by comprising: The echo cancellation method according to claim 6 ,
Further, the speaker sound signal selection means selects a sound reception signal whose time average value of the ratio of the echo signal level to the conversation person sound signal level is smaller than the other sound reception signals from the plurality of sound pickup means. The process of selecting a speaker's received sound signal,
The echo sound receiving signal selecting means selects an echo receiving signal from among the plurality of sound collecting means for selecting a sound receiving signal whose time average value of the ratio of the echo signal level to the conversation person sound signal level is larger than that of the other sound receiving signals. An echo canceling method comprising: a sound signal selection process; The echo cancellation method according to claim 7 , wherein
In the speaker received signal selection process, the speaker received signal selection means has high sensitivity in the direction of the speaker, and one sound collection means as the speaker sound collection means among the plurality of sound collection means (Hereinafter referred to as the main sound collecting means)
The echo sound reception signal selection process is a process in which the echo sound signal selection means obtains a sound collection signal from a sound collection means other than the main sound collection means with high sensitivity in the reproduction means direction. Echo canceling method characterized. The echo cancellation method according to claim 7 , wherein
The speaker received signal selection process is a process of suppressing the component of the echo signal in the received signal from all the plurality of sound collecting means by the main beamformer,
The echo received signal selection process is a process of suppressing a component of the speaker received signal in the received signal from all the plurality of sound collecting means by a sub-beamformer. Method. In the echo cancellation method according to any one of claims 6 to 9 ,
The update of the filter coefficient in the first filter coefficient update process and the second filter coefficient update process is performed by a learning identification (NLMS: Normalized Last-Mean-Squares) algorithm, a projection algorithm, or a recursive least square (RecursiveLeastSquare) algorithm. Alternatively, an echo cancellation method, which is a process of sequentially updating using an LMS (Least Mean Square) algorithm. An echo cancellation program for causing a computer to execute each step of the echo cancellation method according to any one of claims 6 to 10 . The computer-readable recording medium which recorded the echo cancellation program of Claim 11 .

Images