CN105403860B - A kind of how sparse sound localization method related based on domination - Google Patents

Abstract

The present invention relates to a multi-sparse sound source localization method based on dominance correlation, comprising: converting the sound source signal received through a microphone array into a digital sound signal; extracting the frequency spectrum of the digital sound signal of each microphone; Calculate the spatial correlation matrix on each frequency point from the spectrum of the digital sound signals of all microphones on the point; extract the main eigenvector of the spatial correlation matrix; determine the time delay set of all microphone pairs on each frequency point; use an iterative method to calculate The azimuth angle of the incident direction of the dominant sound source at each frequency point; the azimuth angle of the incident direction of the dominant sound source at all frequency points is statistically analyzed to determine the final incident direction of the sound source and the number of sound sources. The method considers acoustic robustness and is suitable for real-time localization of multiple sparse sound sources.

Description

A multi-sparse sound source localization method based on dominance correlation

technical field

The invention relates to the field of sound source localization, in particular to a multi-sparse sound source localization method based on dominance correlation.

Background technique

Sound source localization includes single sound source localization and multiple sound source localization. The sound source localization technology can indicate the spatial orientation of the sound source target and provide important spatial information for subsequent information collection and processing.

Multi-sound source localization technology occupies an important position in the application of microphone arrays. It can be used in remote conferences to indicate the direction of beam focus for microphone arrays and provide pointing information for conference cameras.

In the field of sound source localization, the sound source localization algorithm based on multi-source signal classification and spatial beam scanning is the most widely used algorithm, and it is famous for its robustness under noise and reverberation conditions. However, these algorithms have a common feature: their cost functions are non-convex and non-concave, resulting in multiple extrema corresponding to the optimal estimation of the sound source position. Therefore, such methods generally use grid traversal to find the global optimal solution of the sound source location, resulting in a sharp increase in the amount of calculations, and it is difficult to meet the needs of real-time positioning.

The multi-sound source localization method based on sparsity in the time-frequency domain overcomes the above-mentioned difficulties. This method assumes that only one sound source occupies a dominant position at each frequency point, and the interference of other sound sources can be ignored. This assumption simplifies a multi-sound source localization problem to a single sound source localization problem at a single frequency point. Although the amount of calculation is greatly reduced, the lack of acoustic robustness makes it difficult for such methods to perform well in complex acoustic environments. excellent performance.

Contents of the invention

The purpose of the present invention is to overcome the defects of large amount of calculation and lack of acoustic robustness existing in the existing multi-sound source localization method, and utilize the dominant correlation of sparse sound sources in the time-frequency domain to provide a robust and efficient Multi-source localization method.

In order to achieve the above object, the present invention proposes a multi-sparse sound source localization method based on dominance correlation, including:

Step 101), converting the sound source signal received by the microphone array into a digital sound signal, and the microphone array includes K microphones;

Step 102), extract the frequency spectrum of the digital sound signal of each microphone;

Step 103), utilize the frequency spectrum of the digital sound signal of all microphones to calculate the spatial correlation matrix on each frequency point on the same frequency point of adjacent time at t moment;

Step 104), decompose the spatial correlation matrix on each frequency point at time t to obtain the main eigenvector at each frequency point at time t, and each component of the main eigenvector corresponds to the acquisition signal of a microphone;

Step 105), using the principal eigenvector on each frequency point at t time to obtain the time delay set of M pairs of microphones at each frequency point at t time, said M equals K(K-1) 2;

Step 106), according to the time delay set of M pairs of microphones on each frequency point at t time, adopt the iterative method to calculate the azimuth angle of the sound source incident direction that is in a dominant position at each frequency point at t time;

Step 107), statistically analyzing the azimuth angles of the dominant sound source incident directions of all frequency points at time t;

Step 108), outputting the finally determined incident direction of the sound source and the number of sound sources at time t.

In the above-mentioned technical solution, in step 105), the time delay collection of M pairs of microphones on each frequency point at the time t is obtained by using the main eigenvector at each frequency point at the time t time includes:

At the f-th frequency point at time t, the main eigenvector is expressed as: [u _1,t,f ,u _2,t,f ,…u _K,t,f ], which is composed of the p-th and q-th microphones The time phase difference of the mth (m=1,2,...,M) pair of microphones for:

Among them, ∠(.) represents the operation of obtaining the complex phase, Indicates the angular frequency;

At the f-th frequency point at time t, according to the distance r _m constraint of the m-th pair of microphones, the time-phase aliasing set L _{m,t,f is} obtained:

Among them, c is the speed of sound;

At the fth frequency point at time t, the time delay set of the mth pair of microphones is:

in, ,express time period in frequency.

In the above technical solution, the step 106) further includes:

Step 106-1), select the initial sound source incident direction

Step 106-2), selecting a time delay value from each time delay set;

make Select a time delay value τ _m,t,f from each time delay set B _m,t ,f, satisfying:

Among them, g _m = (g _mx , g _my , g _mz ) represents the directional unit vector of the m-th pair of microphones;

Step 106-3), finding new weight coefficients w _m,t,f ;

in:

δ _m,t,f = arccos(p ^T g _m )-arccos(cτ _m,t,f /r _m ), m=1,2,...,M

Step 106-4), calculate the new sound source incident direction

in:

Step 106-5), judging Convergence; if the judgment result is affirmative, proceed to step 106-6); otherwise, proceed to step 106-2) to continue execution;

Step 106-6), calculate the dominant sound source incident direction azimuth angle.

In the above technical solution, in step 106-1), the selection of the initial sound source incidence direction include:

On the grid point space with an azimuth angle of 360°×elevation angle of 90°, uniformly select several incident directions of sound sources as candidate values, and mark them as a vector set {ψ ₁ ,ψ ₂ ,…ψ _H , H>8 and an integer}; For each candidate sound source incidence direction ψ _h , h=1, 2,...H, calculate the sum of its distances to all time delay sets, expressed as:

Among them, % represents the floating-point remainder operation;

At the fth frequency point at time t, the initial sound source incident direction Satisfy:

In the above technical solution, in step 107), the statistical analysis includes: histogram analysis and cluster analysis.

The advantages of the present invention are:

1. Through the dominant similarity between adjacent time and frequency points, fully mine the information on adjacent time and frequency points to achieve reliable spatial position estimation;

2. A paired time-delay extraction method based on the spatial correlation matrix of time-frequency blocks is proposed. This method uses the weight coefficients of signal enhancement and time delay to measure reliability, thereby realizing a robust multi-sparse sound source localization method.

Description of drawings

Fig. 1 is a flow chart of a kind of multi-sparse sound source localization method based on dominance of the present invention;

Fig. 2 is the flow chart of the method for calculating the dominant sound source incident direction azimuth on each frequency point of the present invention;

Fig. 3 is the flow chart of the present invention carrying out histogram analysis to the dominant sound source incident direction azimuth angle on all frequency points;

Fig. 4 is a schematic diagram of the azimuth histogram analysis of the present invention.

detailed description

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

With reference to Fig. 1, method of the present invention comprises the following steps:

Step 101), converting the sound source signal received by the microphone array into a digital sound signal;

The microphone array includes K microphones;

Step 102), the digital sound signal is preprocessed, and the frequency spectrum of the digital sound signal of each microphone is extracted by Fast Fourier Transform (FFT);

The preprocessing of the digital sound signal includes: first padding the digital sound signal of each frame to N points, N=2 ⁱ , i is an integer, and i≥8; then, for the digital sound signal of each frame Perform windowing or pre-emphasis processing, and the windowing function uses Hamming window (hamming) or Hanning window (hanning);

Fast Fourier transform is performed on the digital sound signal at time t, and the discrete spectrum of the digital sound signal at time t is obtained as

Among them, y _{k, t, n} represent the nth sampling point of the signal collected by the kth microphone at time t, Y _{k, t, f} (k=1,2...K, f=0,1,...N-1) Indicates the Fourier transform coefficient of the fth frequency point of the signal collected by the kth microphone at time t.

Step 103), utilize the frequency spectrum of the digital sound signal of all microphones on the same frequency point in adjacent time to calculate the spatial correlation matrix of each frequency point at t moment;

Let x _t,f be a complex vector generated at the fth frequency point at time t: x _t,f ={Y _1,t,f ,Y _2,t,f ...Y _K,t,f }, where The autocorrelation matrix is: Where: () ^H represents the conjugate transpose;

The complex autocorrelation matrix R _{t, f} is expressed as the mean value of the autocorrelation matrix on the adjacent time frequency points centered on the f frequency point:

Among them, A represents the number of frames adjacent to time t;

The spatial correlation matrix of x _t,f is R _t,f .

Step 104), decompose the spatial correlation matrix on each frequency point at time t to obtain the main eigenvector at each frequency point at time t; each component of the vector corresponds to the acquisition signal of a microphone;

Step 105), use the principal eigenvector on each frequency point at t time to obtain the time delay set of M pairs of microphones at each frequency point at t time, and the M is equal to K(K-1) 2; the specific process is:

At the f-th frequency point at time t, the main eigenvector is expressed as: [u _1,t,f ,u _2,t,f ,…u _K,t,f ], which is composed of the p-th and q-th microphones The time phase difference of the mth (m=1,2,...,M) pair of microphones for:

Among them, ∠(.) represents the operation of obtaining the complex phase, Indicates the angular frequency;

At the f-th frequency point at time t, according to the distance r _m constraint of the m-th pair of microphones, the time-phase aliasing set L _{m,t,f is} obtained:

Among them, c is the speed of sound;

At the fth frequency point at time t, the time delay set of the mth pair of microphones is:

in, ,express time period in frequency.

Step 106), according to the time delay set of M pairs of microphones on each frequency point at t time, adopt the iterative method to calculate the azimuth angle of the sound source incident direction that is in a dominant position at each frequency point at t time;

Referring to Figure 2, the specific implementation steps are as follows:

Step 106-1), select the initial sound source incident direction

On the grid point space with an azimuth angle of 360°×elevation angle of 90°, uniformly select several incident directions of sound sources as candidate values, and mark them as a vector set {ψ ₁ ,ψ ₂ ,…ψ _H , H>8 and an integer}; For each candidate sound source incidence direction ψ _h , h=1, 2,...H, calculate the sum of its distances to all time delay sets, expressed as:

Wherein, g _m =(g _mx , g _my , g _mz ) represents the directional unit vector of the m-th microphone pair, and % represents the floating-point remainder operation.

At the fth frequency point at time t, the initial sound source incident direction Satisfy:

Step 106-2), selecting a time delay value from each time delay set;

make Select a time delay value τ _m,t,f from each time delay set B _m,t ,f, satisfying:

Step 106-3), finding new weight coefficients w _m,t,f ;

in:

δ _m,t,f = arccos(p ^T g _m )-arccos(cτ _m,t,f /r _m ), m=1,2,...,M

Step 106-4), calculate the new sound source incident direction

in:

Step 106-5), judging Convergence; if the judgment result is affirmative, proceed to step 106-6); otherwise, proceed to step 106-2) to continue execution;

In this example, judging The method of convergence is:

judge Whether it is smaller than the threshold ε, where ε=0.01.

Step 106-6), calculate the dominant sound source incident direction The azimuth angle of ; the calculation method is:

Step 107) Statistically analyze the azimuth angles of the dominant sound source incident directions of all frequency points at time t;

The statistical analysis includes: histogram analysis and cluster analysis.

In this embodiment, the histogram analysis is performed on the azimuth angles of the dominant sound source incident directions of all frequency points at time t, referring to Fig. 3, the specific steps are as follows:

Step 107-1), constructing and smoothing the azimuth histogram;

Divide the azimuth angle of 360 degrees into 360 equal parts, construct a histogram on these equal parts, and use a window function to smooth the histogram.

Step 107-2), calculating the histogram threshold value;

Histogram threshold value = average histogram value + (maximum histogram value - average histogram value) × empirical coefficient;

The value range of the empirical coefficient is between 0 and 1; in this embodiment, the value of the empirical coefficient is 0.3.

Step 107-3), determine the final azimuth and the number of azimuths;

Referring to FIG. 4 , the peak in the histogram whose peak value is greater than the histogram threshold is selected as the final azimuth, and the number of peaks is the number of azimuths.

Step 107-4), determining the final sound source incident direction and the number of sound sources.

According to the selected azimuth, the incident direction of the sound source corresponding to the azimuth is obtained, and the number of azimuths is the number of incident directions of the sound source.

Step 108), outputting the finally determined incident direction of the sound source and the number of sound sources at time t.

In other embodiments, cluster analysis may be performed on the azimuth angles of the dominant sound source incidence directions at all frequency points at time t. The specific processing method belongs to common knowledge and will not be repeated here.

Claims (5)

1. A multi-sparse sound source localization method based on dominance correlation, comprising: Step 101), converting the sound source signal received by the microphone array into a digital sound signal, and the microphone array includes K microphones; Step 102), extract the frequency spectrum of the digital sound signal of each microphone; Step 103), utilize the frequency spectrum of the digital sound signal of all microphones to calculate the spatial correlation matrix on each frequency point on the same frequency point of adjacent time at t moment; Step 104), decompose the spatial correlation matrix on each frequency point at time t to obtain the main eigenvector at each frequency point at time t, and each component of the main eigenvector corresponds to the acquisition signal of a microphone; Step 105), using the principal eigenvector on each frequency point at t time to obtain the time delay set of M pairs of microphones at each frequency point at t time, said M equals K(K-1)/2; Step 106), according to the time delay set of M pairs of microphones at each frequency point at t time, adopt an iterative method to calculate the azimuth angle of the sound source incident direction that is dominant at each frequency point at t time; Step 107), statistically analyzing the azimuth angles of the dominant sound source incident directions of all frequency points at time t; Step 108), outputting the finally determined incident direction of the sound source and the number of sound sources at time t. 2. The multi-sparse sound source localization method based on dominance correlation according to claim 1, characterized in that, in step 105), the use of the principal eigenvector at each frequency point at time t to obtain each The time delay set of M pairs of microphones at the frequency point includes: At the f-th frequency point at time t, the main eigenvector is expressed as: [u _1,t,f ,u _2,t,f ,…u _K,t,f ], which is composed of the p-th and q-th microphones The time phase difference of the mth (m=1,2,...,M) pair of microphones for: Among them, ∠(.) represents the operation of obtaining the complex phase, Indicates the angular frequency; At the f-th frequency point at time t, according to the distance r _m constraint of the m-th pair of microphones, the time-phase aliasing set L _{m,t,f is} obtained: Among them, c is the speed of sound; At the fth frequency point at time t, the time delay set of the mth pair of microphones is: <mrow><msub><mi>B</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>=</mo><mo>{</mo><msub><mover><mi>&amp;tau;</mi><mo>^</mo></mover><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>=</mo><msub><mover><mi>&amp;tau;</mi><mo>&amp;OverBar;</mo></mover><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>+</mo><msub><mi>l</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><msub><mi>T</mi><mi>f</mi></msub><mo>|</mo><msub><mi>l</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>&amp;Element;</mo><msub><mi>L</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>}</mo><mo>,</mo><mi>m</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo>mo><mo>...</mo><mo>,</mo><mi>M</mi></mrow> in, express time period in frequency. 3. The multi-sparse sound source localization method based on dominant correlation according to claim 2, characterized in that, step 106) further comprises: Step 106-1), select the initial sound source incident direction make Step 106-2), selecting a time delay value from each time delay set; make Select a time delay value τ _m,t,f from each time delay set B _m,t ,f, satisfying: <mrow><msub><mi>&amp;tau;</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>=</mo><mi>arg</mi><munder><mi>min</mi><mrow><mi>&amp;tau;</mi><mo>&amp;Element;</mo><msub><mi>B</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub></mrow></munder><mrow><mo>|</mo><mrow><msubsup><mi>pg</mi><mi>m</mi><mi>T</mi></msubsup><msub><mi>r</mi><mi>m</mi></msub><mo>-</mo><mi>c</mi><mi>&amp;tau;</mi></mrow><mo>|</mo></mrow><mo>,</mo><mi>m</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>...</mo><mo>,</mo><mi>M</mi><mo>;</mo></mrow> 1 Among them, g _m = (g _mx , g _my , g _mz ) represents the directional unit vector of the m-th pair of microphones; Step 106-3), finding new weight coefficients w _m,t,f ; <mrow><msub><mi>w</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>=</mo><mi>exp</mi><mrow><mo>(</mo><msubsup><mi>&amp;delta;</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow><mn>2</mn></msubsup><mo>/</mo><msup><mi>&amp;sigma;</mi><mn>2</mn></msup><mo>)</mo></mrow><mo>/</mo><munderover><mo>&amp;Sigma;</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><mi>exp</mi><mrow><mo>(</mo><msubsup><mi>&amp;delta;</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow><mn>2</mn></msubsup><mo>/</mo><msup><mi>&amp;sigma;</mi><mn>2</mn></msup><mo>)</mo></mrow><mo>,</mo><mi>m</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>...</mo><mo>,</mo><mi>M</mi></mrow> in: δ _m,t,f = arccos(p ^T g _m )-arccos(cτ _m,t,f /r _m ), m=1,2,...,M <mrow><msup><mi>&amp;sigma;</mi><mn>2</mn></msup><mo>=</mo><munderover><mo>&amp;Sigma;</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><msubsup><mi>&amp;delta;</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow><mn>2</mn></msubsup><mo>/</mo><mi>M</mi></mrow> Step 106-4), calculate the new sound source incident direction <mrow><mfenced open = "(" close = ")"><mtable><mtr><mtd><msub><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mn>1</mn><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub></mtd></mtr><mtr><mtd><msub><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mn>2</mn><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub></mtd></mtr></mtable></mfenced><mo>=</mo><msup><mrow><mo>&amp;lsqb;</mo><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><msub><mi>w</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><msubsup><mi>g</mi><mi>m</mi><mo>&amp;prime;</mo></msubsup><msubsup><mi>g</mi><mi>m</mi><mrow><mo>&amp;prime;</mo><mi>T</mi></mrow></msubsup><mo>&amp;rsqb;</mo></mrow><mrow><mo>-</mo><mn>1</mn></mrow></msup><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><msub><mi>cw</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><msub><mi>&amp;tau;</mi><mrow><mi>m</mi><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><msubsup><mi>g</mi><mi>m</mi><mrow><mo>&amp;prime;</mo><mi>T</mi></mrow></msubsup><mo>/</mo><msub><mi>r</mi><mi>m</mi></msub></mrow> <mrow><msub><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mn>3</mn><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow></msub><mo>=</mo><msqrt><mrow><mn>1</mn><mo>-</mo><msubsup><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mn>1</mn><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow><mn>2</mn></msubsup><mo>-</mo><msubsup><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mn>2</mn><mo>,</mo><mi>t</mi><mo>,</mo><mi>f</mi></mrow><mn>2</mn></msubsup></mrow></msqrt></mrow> Where: g′ _m = (g _mx , g _my ); Step 106-5), judging convergence; if the judgment result is affirmative, go to step 106-6); otherwise, let Go to step 106-2) to continue execution; Step 106-6), calculate the dominant sound source incident direction azimuth angle. 4. The multi-sparse sound source localization method based on dominant correlation according to claim 3, characterized in that, in step 106-1), the selection of the initial sound source incident direction include: On the grid point space with an azimuth angle of 360°×elevation angle of 90°, uniformly select several incident directions of sound sources as candidate values, and mark them as a vector set {ψ ₁ ,ψ ₂ ,…ψ _H , H>8 and an integer}; For each candidate sound source incidence direction ψ _h , h=1, 2,...H, calculate the sum of its distances to all time delay sets, expressed as: <mrow><msub><mi>b</mi><mi>f</mi></msub><mrow><mo>(</mo><msub><mi>&amp;psi;</mi><mi>h</mi></msub><mo>)</mo></mrow><mo>=</mo><munderover><mo>&amp;Sigma;</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><mo>&amp;lsqb;</mo><msubsup><mi>&amp;psi;</mi><mi>H</mi><mi>T</mi></msubsup><msub><mi>g</mi><mi>m</mi></msub><msub><mi>r</mi><mi>m</mi></msub><mo>-</mo><mi>c</mi><mi>&amp;tau;</mi><mo>)</mo><mi>%</mi><mrow><mo>(</mo><msub><mi>cT</mi><mi>f</mi></msub><mo>)</mo></mrow><msup><mo>&amp;rsqb;</mo><mn>2</mn></msup></mrow> Among them, % represents the floating-point remainder operation; At the fth frequency point at time t, the initial sound source incident direction Satisfy: <mrow><msub><mover><mi>&amp;gamma;</mi><mo>^</mo></mover><mrow><mi>t</mi><mo>,</mo><mi>f</mi><mo>,</mo><mn>0</mn></mrow></msub><mo>=</mo><munder><mrow><mi>arg</mi><mi>min</mi></mrow><mrow><mn>1</mn><mo>&amp;le;</mo><mi>h</mi><mo>&amp;le;</mo><mi>H</mi></mrow></munder><msub><mi>b</mi><mi>f</mi></msub><mrow><mo>(</mo><msub><mi>&amp;psi;</mi><mi>h</mi></msub><mo>)</mo></mrow><mo>.</mo></mrow> 5. The multi-sparse sound source localization method based on dominance correlation according to claim 1, characterized in that, in step 107), the statistical analysis comprises: histogram analysis and cluster analysis.