KR101243897B1 - Blind Source separation method in reverberant environments based on estimation of time delay and attenuation of the signals - Google Patents

Abstract

The method for separating tacit sound sources in an echo environment based on a time delay and attenuation estimation of a signal according to the present invention comprises: receiving mixed signals from two or more microphones; Converting the mixed signals into mixed signals in a time-frequency domain by performing a short time fourier transform (STFT); For the STFT mixed signals, the frequency attenuation and time delay values are initialized, the initialized frequency attenuation and time delay values are converged, and the frequency-specific frequency attenuation and time delay values are converged. Generating a binary mask, separating signals by frequency using the binary mask for each frequency, and obtaining a correlation coefficient with respect to the signals separated by frequency, and matching the order of the signals separated by frequency; And reconstructing the ordered signals into inverse short time fourier transforms (ISTFTs) to sound source signals in a time domain.

Description

Blind Source separation method in reverberant environments based on estimation of time delay and attenuation of the signals}

The present invention relates to a tacit sound separation technique, and more particularly, in a reverberation environment based on a time delay and attenuation estimation of a signal for separating tactile sound signals by estimating different decay and time delay values for each frequency in consideration of a reverberation environment. It relates to a method of separating the tacit sound source.

Since humans can pay attention to and recognize a specific sound source signal in an environment in which a plurality of sound sources exist, a machine also needs to separate a specific sound source from a mixed signal for effective signal processing. As described above, the separation of individual sound source signals from a mixed signal in which several sound source signals are mixed is called blind source separation (BSS). Here, blind means that there is no information about the original sound source signal and no information about the mixed environment. The process of finally separating the sound source signal from the mixed signal is called demix or unmix.

Conventional tacit sound source separation methods can be divided into tacit sound source separation methods using one mixed signal and tacit sound source separation methods using a plurality of mixed signals according to the number of mixed signals to be used.

First, the implicit sound source separation method using one mixed signal separates the sound source signal through statistical inference by using the mixed signal input to one micro input, which is very poor in performance because it cannot use spatial information.

In addition, the implicit sound source separation method using a plurality of mixed signals separates sound source signals using spatial information as well as statistical guesses by using a plurality of mixed signals inputted with a micro signal, which is much better than a method using a single mixed signal. Shows performance.

In addition, the implicit sound source separation method using the plurality of mixed signals is classified again according to the relationship between the number of mixed signals used and the number of sound source signals to be separated. In other words, Independent Component Analysis (ICA) and Independent Vector Analysis (IVA) methods use a plurality of mixed signals to separate sound source signals through statistical signal processing based on the independence of sound sources. In other words, if the number of sound source signals is greater than the number of mixed signals, the performance may drop sharply. On the other hand, the beam forming method, the ESPRIT (Estimation of Signal Parameters via Rotational Invariance Technique) method, and the DUET (Degenerate Unmixing and Estimation Technique) method mainly use spatial information of the sound source and thus correlate with the number of mixed signals. The advantage is that the sound source signal can be separated without.

As described above, there are various implicit sound source separation methods, among which the DUET method is a representative sound source separation method performed in the time-frequency domain, similar to the processing of two ear signals in humans. ) And an intensity intensity (Interaural Intensity Different; IID) can be used to separate the sound source signal regardless of the number of sound sources.

However, the DUET method separates the tacit sound sources by using the same attenuation and time delay values estimated for all frequencies for each sound source, but in a real echo environment, the attenuation and time delay values are different for each frequency. There was a problem of low quality of the sound source.

<DUET method>

This DUET method will be described in more detail.

Referring to FIG. 1 illustrating a signal coming into two human ears and FIG. 2 illustrating a time delay and attenuation of a signal generated by a path difference, a human can determine the position of a sound source using only acoustic signals coming into both ears. Ears are attached to both sides of the head. That is, a signal with a low frequency causes a difference in intensity and time of a signal reaching each ear by a diffraction phenomenon in which sound propagates to the opposite side of the obstacle, and a signal having a high frequency is reflected by the head instead of a diffraction phenomenon. Likewise, since the signal intensity and time difference occur, the brain determines the direction of the sound source by using the relative strength and time difference of the two signals coming in the two ears, and by using this characteristic, the desired sound source direction among the sound source signals from various directions is used. The sound source signal coming from can be concentrated and listened to.

As such, the phenomenon in which a human recognizes a specific sound source desired in an environment in which a plurality of sound sources are mixed by using two ears is called a cocktail party effect. This human ear signal processing is very efficient because it consists only of the signals coming into the two ears in the environment where a large number of sound sources exist, as well as suitable for the actual acoustic environment.

The mixed signal model of the above DUET method will be described with reference to FIG. 3. In each of the two microphones, sound source signals are introduced through a straight path, and mixed signals have relative attenuation and time delay values due to the distance difference between the sound source and the microphone. Under these conditions, when there are N sound source signals, the mixed model of the DUET method may be represented by Equations 1 and 2 below.

는 제1마이크로 입력된 신호이고, 상기

는 제1마이크에 입력된 신호 중 N 개의 음원신호별 성분을 나타낸다. In Equation (1)

Is the first micro-input signal, and

Denotes components for each of the N sound source signals among the signals input to the first microphone.

는 제2마이크로 입력된 신호이고, 상기

및

는 각각 제1마이크에 대한 제2마이크 입력신호의 j번째 음원신호 성분의 상대적인 감쇄율 및 시간 지연 값이다.
In Equation (2)

Is a second micro-input signal, and

And

Are respectively the relative attenuation rate and time delay value of the j-th sound source signal component of the second microphone input signal relative to the first microphone.

When the mixed model in the time domain is transformed into a mixed model in the time-frequency domain by performing a short time fourier transform (STFT), Equation 3 is obtained.

,

는 시간(

)-주파수(

) 영역에서의 혼합 신호이고,

...

는 시간 지연 값이고,

...

는 감쇄율이고,

...

는 시간(

)-주파수(

) 영역에서의 음원 신호이다.
In Equation (3)

,

Is the time (

)-frequency(

) Is a mixed signal in the

...

Is the time delay value,

...

Is the decay rate,

...

Is the time (

)-frequency(

) Is the sound source signal in the region.

<W-Disjoint Orthogonal>

In the DUET method, W-disjoint orthogonal (WDO) that assumes that one sound source component is dominant at each time-frequency is applied to each sound source constituting the mixed signal.

와

는 시간(

)-주파수(

)에서의 서로 다른 음원 성분이다.
In Equation 4,

Wow

Is the time (

)-frequency(

Different sound source components at).

By the above WDO assumption, all time-frequency components of the mixed signal coming into the microphone are associated with only one sound source signal. The WDO hypothesis does not fully correspond to the actual acoustic signal, but is known to correspond very appropriately to the speech signal.

Accordingly, when the WDO assumption is applied to the mixed model in the time-frequency domain according to Equation 3, only one sound source signal in which one component of the time-frequency domain is dominant as shown in Equation 5 is used.

In Equation 5, j indicates one dominant sound source signal.

In addition, the relative attenuation and time delay values of the signal input to the microphone using the mixed signal based on the above-described WDO assumption are estimated according to Equation (6).

는 추정된 감쇄율이고,

는 추정된 시간 지연 값이다.
In Equation 6,

Is an estimated decay rate,

Is the estimated time delay value.

Attenuation and time delay values for all time-frequency components can be obtained using Equation 6 above, and the values are accumulated to obtain attenuation and time delay histograms for all time-frequency components as shown in FIG. 4. Can be generated. The peak values of the histogram represent attenuation and time delay values of the respective sound source signals constituting the mixed signals, and these values are used as initial values of the learning process for the attenuation and time delay values.

<Learning decay and time delay values>

If the j-th sound source signal is dominant at a specific time-frequency of the mixed signal as in Equation 5, the relationship between the two mixed signals is expressed as in Equation 7. Normalized to the attenuation value is expressed as Equation (8).

는 감쇄에 대한 정규화 값이고,

는 j번째 음원 신호에 대한 추정된 감쇄율이고,

는 j번째 음원 신호에 대한 추정된 시간 지연 값이다. In Equation 8,

Is the normalization value for the attenuation,

Is an estimated attenuation rate for the jth source signal,

Is an estimated time delay value for the j-th sound source signal.

Since it is impossible to know in advance which sound source signal is dominant at a particular time-frequency of the mixed signal, the cost function of Equation 9 is constructed using Equation 8.

는 비용함수이며, 이 비용함수

는 특정 시간-주파수에서 어떤 음원 신호가 지배적인지를 확인하고 그 음원 신호에 대해 추정된 감쇄율 및 시간 지연 값의 정확도를 나타낸다.
In Equation (9)

Is the cost function, and this cost function

Denotes which source signal is dominant at a particular time-frequency and indicates the accuracy of the estimated attenuation rate and time delay value for that source signal.

Since Equation 9 is a non-differential function, a cost function approximated by Equation 10 is possible.

는 근사화에서 연속 함수의 smoothness 정도를 결정하는 파라미터이다. 모든 음원에 대한 감쇄 및 시간 지연 값을 매 프레임 별로 추정하기 위해 수학식 10에 나타낸 비용함수 J의 최소값을 구하는 stochastic gradient descent 알고리즘을 적용하여 감쇄 및 시간 지연 값 각각으로 편미분하며, 이는 수학식 11 및 수학식 12에 나타낸 바와 같다. In Equation 10

Is a parameter that determines the degree of smoothness of the continuous function in the approximation. In order to estimate the attenuation and time delay values for every sound source every frame, the stochastic gradient descent algorithm, which obtains the minimum value of the cost function J shown in Equation 10, is applied to the attenuation and time delay values. It is as shown in (12).

The attenuation and time delay values are updated as shown in Equations 13 and 14 based on the partial derivatives obtained by Equations 11 and 12, which are repeated until the cost function J becomes minimum and convergence and time delay values converge. do.

및

는 각각 감쇄율 및 시간 지연 학습을 위한 학습률로서 상수로 주어진다.
In Equations 13 and 14

And

Is given as a constant as a learning rate for the decay rate and the time delay learning, respectively.

<Generate binary mask>

When the attenuation and time delay values are substituted into Equation 8, the j-th sound source signal is dominant at the time-frequency based on whether the result value of the values corresponding to the j-th sound source signal has the smallest value. If the j-th sound source signal is the dominant sound signal at the time-frequency, the time-frequency value is set to 1 in the mask for separating the j-th sound source signal according to the WDO assumption, and other time-frequency values are determined. Create a binary mask, which is set to zero, which is according to equation (15).

는 이진 마스크이며, 상기

는 j번째 음원 신호에 대한 감쇄 정규화 값

이 다른 음원 신호들에 대한 감쇄 정규화 값들

보다 가장 작은 조건을 의미한다.
In Equation 15,

Is a binary mask and said

Is the attenuation normalization value for the jth sound source signal

Attenuation Normalization Values for these Other Sound Source Signals

It means the smaller condition.

<Sound source separation>

The binary mask generated for each sound source signal is applied to the mixed signal to separate the sound source signal as follows.

는 j번째 음원 분리 신호이고,

는 제1마이크로부터의 혼합 신호이다.
In Equation 16

Is the jth sound source separation signal,

Is the mixed signal from the first microphone.

The signal separated through the binary mask is inverse short time fourier transform (ISTFT) to restore the sound source signal in the time domain.

In the above DUET method, it is assumed that the microphone is drawn in through a straight path between the sound source and the microphone, and attenuation and time delay due to the difference of the straight path occur in the process. Accordingly, the mixed filter between the sound source and the microphone assumed in the DUET method has one attenuation and time delay values due to a straight path as shown in FIG. 5.

However, in a real echo environment, as shown in FIG. 6, since the sound source signal is reflected by hitting various objects after starting, it is input to the microphone with different attenuation and time delay values through various paths as well as a straight path, as shown in FIG. 5. Instead of a simple shape, a filter having various attenuation and time delay values should be applied as shown in FIG.

Accordingly, the DUET method, which estimates only the relative attenuation and time delay values by a simple straight path, is not suitable for the actual echo environment. In fact, the experimental results of the DUET method also show a significant variation in performance depending on the environment, and especially in an environment with reflections, the performance drops sharply as the reflection becomes stronger. In other words, the DUET method shows a suitable sound separation performance in an ideal environment without reflections, but has a limitation that cannot be applied in an actual reflection environment.

It is an object of the present invention to provide a method for separating tacit sound sources in an echo environment based on a time delay and attenuation estimation of a signal that separates tacit sound signals by estimating different attenuation and time delay values for each frequency in consideration of the echo environment. do.

In addition, the present invention provides an initial value estimation based on cluster separation and an order shift adjustment based on the correlation coefficient of the spectral envelope in order to solve the data shortage and order shifting problem caused by estimating different attenuation and time delay values for each frequency. It is an object of the present invention to provide a method for separating tacit sound sources in an echo environment based on time delay and attenuation estimation of a signal to be implemented.

The tacit sound source separation method of the present invention for achieving the above object comprises the steps of: receiving mixed signals from two or more microphones; Converting the mixed signals into mixed signals in a time-frequency domain by performing a short time fourier transform (STFT); For the STFT mixed signals, the frequency attenuation and time delay values are initialized, the initialized frequency attenuation and time delay values are converged, and the frequency-specific frequency attenuation and time delay values are converged. Generating a binary mask, separating signals by frequency using the binary mask for each frequency, and obtaining a correlation coefficient with respect to the signals separated by frequency, and matching the order of the signals separated by frequency; And reconstructing the ordered signals into inverse short time fourier transforms (ISTFTs) to sound source signals in a time domain.

The present invention described above can improve the separation performance of the tacit sound source signal by separating tacit sound source signals by estimating different attenuation and time delay values for each frequency in consideration of the echo environment.

In addition, the present invention implements the order shift adjustment based on the initial value estimation based on cluster separation and the correlation coefficient of the spectral envelope to solve the problem of data shortage and order shift caused by estimating different attenuation and time delay values for each frequency. There is an effect that can be solved.

1 illustrates a signal coming into two ears of a human;
2 illustrates the attenuation and time delay of a signal caused by a path difference.
3 illustrates a mixed signal model in the DUET method.
4 shows attenuation and time delay histogram.
5 illustrates a filter model in the DUET method.
6 illustrates a mixed signal model in an echo environment.
7 illustrates a filter model in a real echo environment.
8 is a block diagram of a tacit sound source separation apparatus according to the present invention.
9 is a flowchart of a method for separating tacit sound sources according to the present invention.
10 illustrates a reordering problem.
11 illustrates a correlation coefficient calculation process.
12 is a diagram illustrating a process of setting a frequency order to perform ordering according to a correlation coefficient magnitude.
13 is a diagram illustrating an ordering process by comparing correlation coefficients.
14 is a diagram illustrating a comparison between a reference value and the entire ordered signal.

<Configuration of the silent sound source separation device>

The configuration of the tacit sound source separating apparatus in the echo environment based on the time delay and attenuation estimation of the signal according to the preferred embodiment of the present invention will be described with reference to FIG.

The tacit sound source separating apparatus includes first and second microphones 100 and 102, a short time fourier transformer (STFT) 104, a tacit sound source separating unit 106, and an inverse short time fourier transformer (ISTFT) 108.

The first and second microphones 100 and 102 output mixed signals corresponding to the input audio, respectively. Here, since the sound source signal is introduced into the first and second microphones 100 and 102 through various paths in the actual echo environment, when the echo filter hij between the sound source and the microphone is used, the output signal of the sound source signal and the microphone is represented by Equation 17 and It may be represented by Equation 18.

는 제1마이크(100)의 출력신호이며,

는 제1마이크(100)와 음원들 사이의 반향 필터이며,

는 음원 신호이다. In Equation 17,

Is an output signal of the first microphone 100,

Is an echo filter between the first microphone 100 and the sound sources,

Is the sound source signal.

는 제2마이크(102)의 출력신호이며,

는 제2마이크(102)와 음원들 사이의 반향 필터이며,

는 음원 신호이다. In Equation 18,

Is the output signal of the second microphone 102,

Is an echo filter between the second microphone 102 and the sound sources,

Is the sound source signal.

The STFT 104 receives the output signals of the first and second microphones 100 and 102 and outputs them as mixed signals in a time-frequency domain by performing a short time fourier transform (STFT).

The implicit sound source separating unit 106 receives the output signal of the STFT 104 and separates the sound source signals into output signals.

The inverse short time fourier transformer (ISTFT) 108 receives the separated sound source signals and restores the inverse short time fourier transform (ISTFT) to a sound source signal in the time domain.

<Second sound source separation procedure>

The processing procedure of the tacit sound source separation unit 106 of the tacit sound source separation device according to the preferred embodiment of the present invention will be described with reference to FIG.

The implicit sound source separating unit 106 performs attenuation and time delay values for respective frequencies with respect to the input mixed signal in step 200.

Then, the implicit sound source separating unit 106 learns the attenuation and time delay values for each frequency (step 202), and generates a binary mask for each frequency based on the learned attenuation and time delay values for each frequency (step 204). .

Thereafter, the blind source separating unit 106 separates signals by frequency using a binary mask for each frequency (step 206), obtains correlation coefficients for the signals separated by each frequency, and sets the order (step 208). The adjusted order is output to be optimized (step 210).

<Detailed explanation of the implicit sound source separation procedure>

Hereinafter, the tacit sound source separation procedure will be described in more detail.

<Mixed model>

In a real echo environment, the mixed signal entering the first and second microphones 100 and 102 through various paths may be represented by Equation 19 and Equation 20 using an echo filter hij between the sound source and the microphone.

The STFT of the mixed signal is converted into a signal in the time-frequency domain, and is expanded to a form having a relative attenuation and time delay value between the two mixed signals.

는 시간-주파수 영역에서의 혼합신호이고,

는 시간-주파수 영역에서의 반향필터이고, 상기

는 시간-주파수 영역에서의 음원신호이다. In Equation 19,

Is a mixed signal in the time-frequency domain,

Is an echo filter in the time-frequency domain, and

Is a sound source signal in the time-frequency domain.

...

는 N개의 음원 신호에 대한 감쇄율이고,

...

는 N개의 음원 신호에 대한 시간 지연 값이다. And said

...

Is the attenuation rate for N sound signals,

...

Is a time delay value for the N sound source signals.

When the WDO assumption is applied to Equation 19, the time-frequency component of the mixed signal is dominant, and only one sound source signal is dominant. In the time-frequency domain, the model of the mixed signal has a signal component other than the dominant sound source signal. It can be removed and expressed as Equation 20.

Therefore, the same sound source signal has different attenuation and time delay values for each frequency, and thus the existing DUET method for finding one attenuation and time delay value for the entire frequency cannot be applied to an actual echo environment. To solve this problem, the present invention estimates the attenuation and time delay values for all frequencies of the sound source signal.

,

)이 모든 주파수(w)에 대해 각기 다르게 추정됨을 나타낸다.
Equation 21 is an attenuation and time delay value (

,

) Is estimated differently for all frequencies w.

<Clear and Reset Time Delay Values>

As described above, the present invention must estimate different attenuation and time delay values at all frequencies for each of the sound source signals, and therefore, initial values of the attenuation and time delay values should be given differently for all frequencies. However, in calculating the initial value for each frequency, much less attenuation and time delay estimation values are used since only the corresponding frequency data is used as compared to the conventional DUET method which estimates the attenuation and time delay values for all time-frequency components to generate a histogram. To use them. Accordingly, the present invention employs a vector quantization method proposed by LBG (Linde, Buzo, Gray). The LBG method is a combination of a binary division method and a k-means clustering method. The LBG method uses a binary division to obtain a center and uses this as an initial value of the k-means clustering method.

The LBG method combines the binary division method and the k-means clustering method, and has both the advantages of fast operation of the binary division method and accuracy of the k-means clustering method. The k-means clustering method arbitrarily selects an initial center value, wherein the k-means clustering method is sensitive to the selected initial value. Therefore, LBG method finds the center by binary division and uses this as initial value of k-mean method to compensate for the shortcomings.

For the binary division of the LBG method, the attenuation and time delay values obtained by applying Equation 6 to one frequency are defined as a cluster and the center thereof is found according to Equation 22.

는 중심값을 나타내며,

는 클러스터에 포함되는 감쇄 및 시간 지연 값들의 수이다.In Equation 22,

Represents the center value,

Is the number of attenuation and time delay values included in the cluster.

는 감쇄 및 시간 지연 값의 한 쌍을 나타내고,

과

는 각각 추정된 감쇄율 및 시간 지연 값이다. In Equation 23

Represents a pair of attenuation and time delay values,

and

Are the estimated decay rate and time delay values, respectively.

When the centers of the attenuation and time delay values for each of the frequencies are obtained, two center values slightly shifted from the center values are obtained according to Equation (24). Here, the reason for obtaining two center values slightly shifted from the center is to divide into two clusters.

는 총 m개의 클러스터 중 분할 대상인 l번째 클러스터에 대한 중심값을 나타내며,

및

는 총 m+1개의 클러스터로 분할하기 위해 이동된 새로운 중심값이다. In Equation 24, ε is a small positive constant value for determining the moving width,

Represents the center value for the l th cluster to be split among a total of m clusters.

And

Is the new center value shifted to divide m + 1 clusters in total.

The two center values thus obtained are given as initial values of attenuation and time delay values, and the center values are updated by the k-means clustering method. Here, the initial value of the attenuation and time delay values corresponding to the number of sound source signals may be set by applying Equation 24 to a cluster having a large dispersion.

<Learning decay and time delay values>

The learning of the attenuation and time delay values is also done separately for each frequency and is estimated according to equation (25).

는 지배적인 하나의 음원 신호(j)에 대한 감쇄 정규화값이고,

는 감쇄값이고,

는 제1마이크(100)로의 혼합신호이고,

는 제2마이크(102)로의 혼합신호이고,

는 시간 지연 값이다.
In Equation 25,

Is the attenuation normalization value for one dominant sound source signal j,

Is an attenuation value,

Is the mixed signal to the first microphone 100,

Is the mixed signal to the second microphone 102,

Is the time delay value.

The attenuation and time delay values calculate the individual attenuation and time delay values for each frequency rather than the cumulative value for the entire frequency in accordance with Equations 26-28.

는 비용함수이고,

는 근사화에서 연속 함수의 평활(smoothness) 정도를 결정하는 파라미터이며,

...

는 제1 내지 제N 음원 신호에 대한 감쇄 정규화 값이다. In Equation 26

Is the cost function,

Is a parameter that determines the degree of smoothness of the continuous function in the approximation.

...

Is an attenuation normalization value for the first to Nth sound source signals.

를 감쇄값으로 편미분한 것이다. Equation 27 is a cost function

Is the partial derivative of the attenuation value.

를 시간 지연값으로 편미분한 것이다. Equation 28 is

Is the partial derivative of the time delay value.

Using the attenuation and time delay values obtained for each frequency, the attenuation and time delay values for each frequency are updated as in Equations 29 and 30, and the learning process is repeated until the values converge. Since the magnitude of energy for each frequency is different in the learning process, the energy value for each frequency of the signal is obtained as shown in Equation 31, and the learning rate is given according to the magnitude.

는 주파수별 에너지 값의 크기에 따른 감쇄율에 대한 학습률이고,

는 감쇄값이다. In Equation 29,

Is the learning rate for the attenuation rate according to the magnitude of the energy value for each frequency,

Is the attenuation value.

는 주파수별 에너지 값의 크기에 따른 시간 지연에 대한 학습률이고,

는 시간 지연 값이다. In Equation 30,

Is the learning rate for the time delay according to the magnitude of the energy value for each frequency.

Is the time delay value.

및

는 수학식 31에 의해 결정된다. remind

And

Is determined by equation (31).

,

,

와

는 각각 감쇄율 및 시간 지연에 대한 학습률로 변환하는 펙터(factor)이며,

는 제1 및 제2마이크(100,102)로부터의 혼합 신호의 전체 에너지 값이고,

는 학습률 펙터(factor) 설정을 위한 상수값,

는 주파수별 에너지 값의 최대값이다.
In Equation 31, β (ω) is a learning rate factor according to the magnitude of energy value of each frequency,

Wow

Are factors that translate into learning rates for decay rates and time delays, respectively.

Is the total energy value of the mixed signal from the first and second microphones 100, 102,

Is a constant value for setting the learning rate factor,

Is the maximum value of the energy value for each frequency.

<Order order>

The process of generating a binary mask using the attenuation and time delay values finally converged through learning, and separating the sound source from the mixed signal using the binary mask is the same as the conventional DUET method. However, since the present invention obtains the attenuation and time delay values independently for each frequency, as shown in FIG. 10, the order of changing the order of the sound sources for each frequency may occur in the separated sound source. When such a reversal phenomenon occurs, various sound source signals exist for each frequency in one reconstruction signal, and thus, sound source separation cannot be performed properly.

Accordingly, the present invention implements a reordering adjustment suitable for the DUET method.

The reordering adjustment according to the present invention is a method proposed by Murata et al. (N. Murata, S. Ikeda, and A. Ziehe, "An approach to blind source separation based on temporal structure of speech signals). " Neurocomputing , vol. 41, no. 1-4, pp. 1-24, Oct. 2001.) and then adjust the sequence to maximize the overall correlation coefficient.

The reordering adjustment obtains a correlation coefficient between signals separated through a binary mask for each frequency according to Equation 32, and determines a frequency order to be fit according to the magnitude of the correlation coefficient as shown in Equation 33.

는 상관 계수의 크기이고,

~

는 이진 마스크에 의해 분리된 신호들이다. In Equation 32,

Is the magnitude of the correlation coefficient,

~

Are signals separated by a binary mask.

Here, the small correlation coefficient may be the most obvious criterion because it means that the two sound sources can be easily distinguished. Accordingly, as shown in Equation 34, the signal value of the frequency having the smallest correlation coefficient is determined as the reference value.

는 기준값이고,

는 가장 상관 계수가 작은 주파수의 분리 음원 신호 값이다. In Equation 34,

Is the reference value,

Is the value of the separated sound source signal of the frequency with the smallest correlation coefficient.

The correlation coefficients are changed in order of low frequency according to Equation 35, and the correlation coefficients between the reference value and the sound source are calculated, and the order is adjusted according to the order in which the values are maximized. 13.

는 주파수

에서 맞춤된 순서열을 나타내며,

는 맞춤된 순서에 따른 분리 음원 신호이고,

는 이전 주파수까지 맞춤된 음원 신호로부터의 기준값이다.
In Equation 35

Frequency

Represents a custom sequence in

Is a separate sound source signal in a customized order,

Is the reference value from the sound source signal fitted up to the previous frequency.

Then, the reference value is updated by adding a separate sound source signal fitted to the reference value as shown in Equation 36.

는 맞춤된 음원 신호로부터의 기준값이고,

는 맞춤된 순서에 따른 분리 음원 신호이다. In Equation 36,

Is the reference value from the customized sound source signal,

Is a separate sound source signal in a customized order.

However, the method of Murata et al. May have incorrect ordering and missing order reversal because the reference value is updated and ordered at the same time and the process is completed at once. Since these errors adversely affect the overall performance, Sawada et al. (H. Sawada, R. Mukai, S. Araki, and S. Makino, "Robust and precise method for solving the permutation problem of frequency-domain blind source separation, "in Proc. Int. Symp. Independent Component Analysis Blind Signal Separation (ICA), Nara, Japan, Apr. 2003, pp. 505-510.). Apply the process of optimization.

The correlation coefficient of the entire signal is obtained by newly adjusting the sequence information based on the sum of the envelope signals based on the sequence information of the previous repetition step as shown in Equation 37, and then summing the reference value and the correlation coefficient for each frequency and sound source. And the ordering process by the order information of the previous iteration step is repeated until this value is the maximum. This iterative optimization process ends when the value of the overall correlation coefficient no longer increases compared to that of the previous iteration step.

는 신호 전체의 상관 계수이다.
In Equation 37

Is the correlation coefficient of the whole signal.

In the method of Sawada et al., After the optimization process of maximizing the correlation coefficient of the entire signal, the detail process is performed to maximize the correlation coefficient based on the center of the neighboring frequency and the harmonic frequency. However, unlike the implicit sound source separation method based on independent component analysis, the detail matching process uses neighboring and harmonic frequencies because the sound source separation method of the present invention uses the result of maximizing the WDO condition according to the arrival direction of the sound source for each frequency. The associations between them become much worse, making them more difficult to converge into one place. This is more prominent in a large echo environment, and the performance degradation of Sawada's method can be confirmed from the experimental results.

Therefore, first, iterative optimization process that maximizes the overall correlation coefficient used in Sawada's method is performed by initializing the result by Murata's method. In this way, the separated signal is finally obtained by the following equation (38).

는 분리된 최종 신호이며,

는 전체 상관 계수 최대화 과정을 통해 맞춤된 순서에 따른 분리 음원 신호이고, M은 전체 상관계수 최대화 과정의 마지막 반복 횟수를 나타낸다. In Equation 38,

Is the final separated signal,

Is the separated sound source signal according to the customized order through the entire correlation coefficient maximization process, and M represents the number of last repetitions of the overall correlation coefficient maximization process.

100: first microphone
102: second microphone
104: STFT
106: implicit sound source separation unit
108: ISTFT

Claims (6)

In the tacit sound source separation method performed by the tacit sound source separation device,
Receiving mixed signals from two or more microphones;
Converting the mixed signals into mixed signals in a time-frequency domain by performing a short time fourier transform (STFT);
For STFT mixed signals,
Initialize the attenuation and time delay values for each frequency,
Learn to converge the initialized frequency attenuation and time delay values, generate a frequency-specific binary mask based on the learned frequency attenuation and time delay values,
Separating signals by frequency using the frequency-specific binary mask, and obtaining a correlation coefficient with respect to the signals separated by the frequencies to adjust the order of the signals separated by the frequencies;
And reconstructing the ordered signals by inverse short time fourier transform (ISTFT) to sound source signals in a time domain.
Initialization of the attenuation and time delay value,
For each frequency we define the attenuation and time delay values as a cluster,
The center value of the cluster is detected according to Equation 39,
The method of claim 2, wherein the center value is determined as an initial value of the attenuation and time delay values.
Equation 39

In Equation 39,

Represents the center value,

Is the number of decay and time delay values included in the cluster,

Is a pair of decay and time delay values, estimated decay rate

And time delay values

Consists of delete The method of claim 1,
The center value is moved by Equation 40, and the moved center value is set as a new initial value of the divided cluster of the attenuation and time delay values.
Equation 40

In Equation 40, ε is a small positive constant value for determining a moving width,

Represents the center value for the l th cluster to be split among a total of m clusters.

And

Is the new center shifted to split m + 1 clusters in total. The method of claim 1,
Learning to converge the frequency attenuation and time delay values,
A method of separating tactile sound sources according to Equations 41 to 47 until the attenuation and time delay values converge to minimize the cost function.
Equation 41

In Equation 41,

Is the normalization value for the attenuation,

Is an estimated attenuation rate for the jth source signal,

Is an estimated time delay value for the j-th sound source signal,

And

Respectively represent the first and second microphone output signals in the time-frequency domain.
Equation 42

In Equation 42

Is the cost function,

Is a parameter that determines the degree of smoothness of the continuous function in the approximation.

...

Is an attenuation normalization value for the first to Nth sound source signals.
Equation 43

Equation 43 is a cost function

Is the partial derivative of the attenuation value.
Equation 44

Equation 44 is a cost function

Is a partial derivative of time delay.

Equation 45
In Equation 45,

Is the learning rate for the attenuation rate according to the magnitude of the energy value for each frequency,

Is attenuation value.
Equation 46

In Equation 46,

Is the learning rate for the time delay according to the magnitude of the energy value for each frequency.

Is the time delay value.
Equation 47

,

,

In Equation 47, β (ω) is a learning rate factor according to the magnitude of the energy value for each frequency,

Wow

Are factors that translate into learning rates for decay rates and time delays, respectively.

Is the total energy value of the mixed signal from the microphones,

Is a constant value for setting the learning factor factor,

Is the maximum value of the energy value for each frequency. The method of claim 1,
Finding the correlation coefficients for the signals separated by frequency and ordering the signals separated by frequency,
A correlation coefficient for the separated signals is calculated according to Equation 48,
Determining a frequency order to fit according to Equation 49 according to the calculated magnitude of the correlation coefficient,
The method of claim 2, characterized in that the order of the signals separated by the reference value based on the signal value of the lowest frequency coefficient according to (50) and (51).
Equation 48

In Equation 48,

Is the magnitude of the correlation coefficient,

~

Are signals separated by a binary mask.
Equation 49

Equation 50

In Equation 50,

Is the reference value,

Is the value of the separated sound source signal with the lowest correlation coefficient.
Equation 51

In Equation 51

Frequency

Represents a custom sequence in

Is a separate sound source signal in a customized order,

Is the reference value from the source signal fitted to the previous frequency. The method of claim 5,
After adjusting the order of the separated signals,
Re-adjust order information based on the sum of the envelope signals based on the order information of the separated signals;
And summing a reference value and a correlation coefficient for each frequency and sound source, and repeating the ordering process until the summed value is maximized.

Images