CN111863018A - Directional sound pickup method and related device under dual microphones - Google Patents

Abstract

The application provides a directional pickup method under a double-microphone and a related device, wherein the method comprises the following steps: according to the sampling voice signals of the double microphones, atoms used for representing the direction to be picked up in a preset dictionary matrix are determined as target atoms, and a frequency domain filter is constructed based on the target atoms in the dictionary matrix so as to pick up the voice signals in the direction to be picked up in the sampling voice signals. Because the frequency domain filter reflects the frequency distribution of the voice in the direction to be picked up in the sampled voice signal, the voice signal obtained by filtering the sampled voice signal according to the frequency domain filter is the voice signal in the direction to be picked up, that is, the voice signal obtained by filtering does not contain the voice signals in other directions, and further, the anti-interference capability of directional pickup realized by the scheme provided by the application is improved.

Description

Directional sound pickup method and related device under dual microphones

technical field

The present application relates to the field of speech recognition, and in particular, to a directional sound pickup method with dual microphones and a related device.

Background technique

With the development of voice interaction technology, the traditional voice recognition system that only supports the main driver's sound source can no longer meet the demand. At present, it is necessary to support the function of speech recognition of the sound source of the main and auxiliary drivers at the same time, that is, a speech recognition system that can pick up the sound source of the main driver and the sound source of the auxiliary driver is required. In vehicle-mounted scenarios, speech recognition is usually affected by environmental noises such as road noise, wind noise, tire noise, and interference from music and human voices, resulting in a serious decline in the effect of speech recognition. Therefore, for the directional pickup in the vehicle-mounted scene with dual microphones, when many people in the car speak at the same time, it is necessary to isolate the sound source of the main and auxiliary drivers, and at the same time suppress the interference of the rear passengers, so that the voice recognition system can pick up the sound source of the main and auxiliary drivers according to the sound source. , to accurately identify the instructions contained in the main and co-pilot sound sources.

For the directional sound pickup in the vehicle-mounted scene with two microphones, the traditional sound pickup method is the beamforming method, such as minimum variance undistorted response beamforming, linear constrained minimum variance beamforming, and generalized sidelobe suppression.

However, when the traditional sound pickup method is applied to the car scene with dual microphones in the car, there is a problem of poor anti-interference ability, that is, there is a lot of noise such as interfering speech and music remaining in the picked-up sound source of the main and auxiliary drivers.

SUMMARY OF THE INVENTION

The present application provides a directional sound pickup method with dual microphones and a related device, which aim to solve the problem of poor anti-interference ability of the traditional beamforming method to pick up the sound source of the main and auxiliary drivers.

In order to achieve the above purpose, the application provides the following technical solutions:

The present application provides a directional sound pickup method with dual microphones, including:

Calculate the time delay of each atom of the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom indicates that the preset speech component represented by the atom under the sampled speech signal reaches the dual microphones time difference;

In the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom of the preset sound pickup beam range, as the target atom;

Calculate the ratio of the sum of the frequency amplitude values of the target atoms under each frequency of the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio in the characterizing the speech signal, so A frequency domain filter describing the frequency distribution of speech in the direction to be picked up

According to the frequency domain signal and the frequency domain filter, the voice signal in the to-be-picked direction in the sampled voice signal is determined.

Optionally, the determining, according to the frequency domain signal and the frequency domain filter, the voice signal in the to-be-picked direction in the sampled voice signal, comprising:

Filter the frequency-domain signal by using the frequency-domain filter to obtain a filtered frequency-domain signal;

The time-frequency inverse transformation is performed on the filtered frequency domain signals respectively to obtain the voice signal in the to-be-picked direction in the sampled voice signal.

Optionally, the frequency domain signal includes: a first frequency domain signal and a second frequency domain signal;

The time delay of each atom of the preset dictionary matrix is calculated according to the frequency domain signal of the sampled speech signal of the dual microphones, including:

所述第一频域信号和所述第二频域信号，计算每个原子的时延函数；其中，f表示频率，d表示原子，W_fd表示所述字典矩阵；d表示时延；X_lf与X_rf分别表示第一频域信号和第二频域信号；F_dτ表示计算得到的原子的时延函数；in accordance with

Calculate the delay function of each atom for the first frequency domain signal and the second frequency domain signal; wherein, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix; d represents the delay; X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal respectively; F _dτ represents the calculated atomic delay function;

For the delay function of each atom, the delay when the delay function takes a maximum value is taken as the delay of the atom, and the delay of each atom of the dictionary matrix is obtained.

Optionally, the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix and the sum of the frequency amplitude values of all atoms at the frequency is calculated separately, and each frequency and the corresponding ratio are formed for use. A frequency domain filter that characterizes the frequency distribution of the voice in the direction to be picked up in the sampled voice signal, including:

If the absolute value of the difference between the delay of the atom of the dictionary matrix and the delay of the voice in the direction to be picked up is smaller than the preset threshold, the binary value of the atom takes the value of 1, otherwise the binary value takes the value is 0;

Calculate the weighted sum of the frequency amplitude values of all atoms and the corresponding binary values under each frequency of the dictionary matrix, and the ratio of the sum of the frequency amplitude values of all atoms at the frequency;

A frequency domain filter used to characterize the frequency distribution of the speech in the to-be-picked direction in the sampled speech signal is composed of each frequency and the corresponding ratio.

Optionally, the generation process of the preset dictionary matrix includes:

The preset training data corresponding to the two microphones are respectively subjected to time-frequency transformation and absolute values are obtained to obtain two non-negative amplitude spectrum matrices;

Decompose the magnitude spectrum matrix into a target dictionary matrix and a coefficient matrix through a non-negative matrix decomposition algorithm;

According to the preset objective function, the target dictionary matrix and the coefficient matrix are iteratively updated until the target dictionary matrix obtained when the objective function converges is used as the preset dictionary matrix.

The application also provides a directional sound pickup device with dual microphones, including:

The first calculation module is used to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom represents the time delay represented by the atom under the sampled speech signal. The time difference between the preset speech components reaching the dual microphones;

The first determination module is used for taking the atoms in the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up, which belongs to the preset range of the sound pickup beam, as the target atom;

The second calculation module is used to calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency of the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio to represent the In the voice signal, the frequency domain filter of the frequency distribution of the voice in the direction to be picked up;

The second determination module is configured to determine, according to the frequency domain signal and the frequency domain filter, the speech signal in the to-be-picked direction in the sampled speech signal.

Optionally, the second determining module is configured to determine the voice signal in the to-be-picked direction in the sampled voice signal according to the frequency domain signal and the frequency domain filter, including:

The second determining module is specifically configured to use the frequency domain filter to filter the frequency domain signal to obtain a filtered frequency domain signal; respectively perform inverse time-frequency transform on the filtered frequency domain signal, The voice signal in the to-be-picked direction in the sampled voice signal is obtained.

Optionally, the frequency domain signal includes: a first frequency domain signal and a second frequency domain signal;

The first calculation module is used to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones, including:

所述第一频域信号和所述第二频域信号，计算每个原子的时延函数；其中，f表示频率，d表示原子，W_fd表示所述字典矩阵；d表示时延；X_lf与X_rf分别表示第一频域信号和第二频域信号；F_dτ表示计算得到的原子的时延函数；The first calculation module is specifically used for

Calculate the delay function of each atom for the first frequency domain signal and the second frequency domain signal; wherein, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix; d represents the delay; X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal respectively; F _dτ represents the calculated atomic delay function;

For the delay function of each atom, the delay when the delay function takes a maximum value is taken as the delay of the atom, and the delay of each atom of the dictionary matrix is obtained.

Optionally, the second calculation module is used to calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and calculate the ratio of each frequency A frequency domain filter used to characterize the frequency distribution of the voice in the direction to be picked up in the sampled voice signal with the corresponding ratio, including:

The second calculation module is specifically configured to, if the absolute value of the difference between the time delay of the atom of the dictionary matrix and the time delay of the voice in the direction to be picked up is less than the preset threshold, the binary value of the atom The value is 1, otherwise the binary value is 0; calculate the weighted sum of the frequency amplitude value of all atoms under each frequency of the dictionary matrix and the corresponding binary value, and, the frequency amplitude value of all atoms at this frequency The ratio of the sum and each frequency and the corresponding ratio are used to form a frequency domain filter for characterizing the frequency distribution of the voice in the direction to be picked up in the sampled voice signal.

The present application further provides a storage medium, where the storage medium includes a stored program, wherein the program executes any one of the above-mentioned directional sound pickup methods with dual microphones.

The directional sound pickup method and related device under the dual microphones described in the present application calculate the time delay of each atom of the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; wherein, the calculated time delay of each atom is The time delay represents the time difference for the preset speech component represented by the atom to reach the dual microphones under the sampled speech signal. Therefore, in the present application, among the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom that belongs to the preset range of the sound pickup beam as the target atom, and the target atom represents the sound to be picked up. Atoms for directional speech.

Furthermore, by calculating the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, the frequency domain filter composed of each frequency and the corresponding ratio reflects the : The frequency distribution of the voice in the direction to be picked up within the preset sound pickup beam range. Therefore, the voice signal in the direction to be picked up obtained by filtering the frequency filter in the present application does not include signals in other directions. Therefore, the anti-interference ability of the directional sound pickup solution under the dual microphone provided by the present application to achieve directional sound pickup is improved.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic diagram of a training process of a dictionary matrix for completing training disclosed in an embodiment of the application;

FIG. 2 is a flowchart of a method for directional sound pickup with dual microphones disclosed in an embodiment of the present application;

Figure 3 (a) is a comparison diagram in the time domain between the sampled speech signal and the directionally picked-up speech signal when sound sources are simultaneously generated on the main and auxiliary vehicles disclosed in the embodiment of the application;

Figure 3 (b) is a comparison diagram in the frequency domain between the sampled speech signal and the directionally picked-up speech signal when sound sources are simultaneously generated on the main and auxiliary vehicles disclosed in the embodiment of the present application;

4 is a schematic structural diagram of a directional pickup device with dual microphones disclosed in an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a device disclosed in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

FIG. 1 provides a training process of a dictionary matrix according to an embodiment of the present application, which may include the following steps:

S101. Perform time-frequency transformation on the preset training data corresponding to the dual microphones respectively, and obtain absolute values to obtain two non-negative amplitude spectrum matrices.

In this example, the preset training data may be 5-min dual-channel noisy speech data. Of course, in practice, the duration of the preset training data may be determined according to the actual situation, and the specific content of the duration is not limited in this embodiment.

In this embodiment, an amplitude spectrum matrix obtained can be represented as X _ft , where f represents frequency and t represents time.

S102 , decompose the magnitude spectrum matrix into a target dictionary matrix and a coefficient matrix through a non-negative matrix decomposition algorithm.

In this embodiment, the magnitude spectrum matrix is decomposed into a dictionary matrix and a coefficient matrix by non-negative matrix factorization (NMF). For the convenience of description, the dictionary matrix obtained by decomposing this step is called The target dictionary matrix. In this embodiment, the dictionary matrix may be represented by W _fd , where d is the number of columns of the dictionary matrix, and the total number of columns also represents the number of atoms included in the dictionary matrix. The coefficient matrix can be represented by H _dt .

In practice, the number of atoms of the dictionary matrix may be 2 ⁿ , of course, it may also be other numbers, and the specific value of the number is not limited in this embodiment. Among them, the increase of the number of atoms in the dictionary matrix will improve the ability of voice pickup and interference suppression in the direction to be picked up, but it will also increase the computational complexity, which needs to be selected according to the actual situation in practical applications.

The specific implementation manner of this step is in the prior art, and details are not repeated here.

S103. Iteratively update the target dictionary matrix and the coefficient matrix according to the preset target function, until the target dictionary matrix obtained when the target function converges is used as the dictionary matrix for completing the training.

In this embodiment, iterative optimization is performed on the decomposed target dictionary matrix and coefficient matrix. Specifically, according to a preset target function, the target dictionary matrix and the coefficient matrix are iteratively updated until the target function converges. The target dictionary matrix obtained when the target function converges is used as the dictionary matrix for completing the training. In this embodiment, the dictionary matrix after training can be used as a preset dictionary matrix.

In this embodiment, the objective function may be Euclidean distance or KL divergence. Of course, in practice, other functions may also be used as the objective function, and this embodiment does not limit the specific form of the objective function.

FIG. 2 is a directional sound pickup method under dual microphones provided by an embodiment of the present application, comprising the following steps:

S201. Acquire voice signals collected by the dual microphones respectively.

In this embodiment, the voice signals respectively collected by the dual microphones may include the sound sources of the driver and the passenger, as well as sound sources in other directions in the vehicle.

In this step, the voice signals respectively collected by the dual microphones are acquired.

S202: Sample the speech signals collected by the dual microphones respectively, to obtain sampled speech signals corresponding to the dual microphones respectively.

In this embodiment, the sampling frequency may be 16000 Hz, and a frame length of 512 sampling points may be selected as the current sampled speech signal to be processed, and the following steps are used for processing. In practice, the frame shift is 10ms (160 sampling points), the Hanning window is selected, and the sampling points with a frame length of 512 obtained by each frame shift are used as the sampled speech signal, and the following steps are processed respectively. In this embodiment, for the convenience of description, the following procedure is introduced for a frame length of 512 sampling points obtained by one sampling as a sampled speech signal.

It should be noted that the sampling frequency of 16000 Hz, the frame length of 512 sampling points, and the frame shift of 10 ms are only a specific implementation. In practice, these parameters can also take other values, which are not specific in this embodiment. The value is limited.

S203. Perform time-frequency transformation on the sampled voice signals corresponding to the two microphones respectively, to obtain a first frequency domain signal and a second frequency domain signal.

In this embodiment, the time-frequency transform may be Fourier transform. Of course, in practice, in addition to Fourier transform, the time-frequency transform may also adopt other transform methods. This embodiment does not specifically describe the time-frequency transform. The transformation method is limited.

In this embodiment, for convenience of description, time-frequency transform is performed on the sampled speech signals corresponding to the two microphones respectively, and the obtained signals are referred to as a first frequency domain signal and a second frequency domain signal.

S204 , according to the first frequency domain signal and the second frequency domain signal, calculate the time delay of each atom of the dictionary matrix that has completed the training.

In this embodiment, the first frequency domain signal and the second frequency domain signal include voice and audio domain signals from different directions, for example, both include voice and audio domain signals from the main driving direction and voice and audio domain signals from the auxiliary driving direction.

The value of the dictionary matrix represents the preset frequency amplitude values of each atom at each preset frequency, wherein the calculated time delay of each atom represents the time difference between the speech components represented by the atom reaching the dual microphones under the sampled speech signal. Therefore, in this step, according to the first frequency domain signal and the second frequency domain signal, calculate the time delay of each atom of the dictionary matrix that has completed the training, that is, the first frequency domain signal and the second frequency domain signal from various directions. The delay of the speech signal is converted into the delay of each atom.

Optionally, the process of calculating the time delay of each atom may include the following steps A1 to A2:

A1. Calculate the time delay function of each atom according to the first frequency domain signal and the second frequency domain signal.

In this step, the delay function of each atom can be calculated according to formula (1).

In the formula, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix that has completed the training; d represents the time delay, X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal respectively, and F _dτ represents the calculated Atomic delay function.

A2. According to the delay function of each atom, the delay when the delay function takes a maximum value is taken as the delay of the atom, and the delay of each atom in the dictionary matrix is obtained.

The specific implementation manner of this step is in the prior art, and details are not repeated here.

The above-mentioned first frequency domain signal and second frequency domain signal may be collectively referred to as frequency domain signals.

The purpose of the above S201-S204 is to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones.

S205. In the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom in the preset range of the sound pickup beam, as the target atom.

In this step, the target atom is an atom whose time delay and the time delay of the voice in the direction to be picked up belong to the preset sound pickup beam range. The direction of the sound to be picked up is determined according to the directional voice that actually needs to be extracted, for example, the voice of the direction to be picked up is the main driving sound source.

Optionally, in this embodiment, the direction to be picked up may be preset or determined in this embodiment. Wherein, this embodiment determines the direction of sound to be picked up, and can perform sound source localization according to the sound source orientation method to obtain a sound source localization result, and determine the sound to be picked up direction according to the specified sound source localization result. Among them, sound source localization methods include but are not limited to methods based on time difference of arrival (eg GCCPHAT), methods based on high resolution spectral estimation (MUSIC), and methods based on steerable beamforming.

S206: Calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form a frequency domain filter with each frequency and the corresponding ratio.

Optionally, in practice, the specific implementation of this step may include the following steps B1 to B2:

B1. If the absolute value of the difference between the delay of the atom in the dictionary matrix and the delay of the voice in the direction to be picked up is smaller than the preset threshold, then the binary value of the atom takes the value of 1, otherwise the binary value takes the value of 0 to obtain A binary matrix consisting of the binary values of each atom and.

Specifically, this step can be implemented by formula (2).

表示预设阈值，δ表示预设的拾音波束范围，τ₀表示待拾音方向的语音的时延，

表示步骤B1中计算出的原子的时延，M_d表示原子的二值取值。In the formula,

represents the preset threshold, δ represents the preset range of the pickup beam, τ ₀ represents the time delay of the voice in the direction to be picked up,

represents the time delay of the atom calculated in step B1, and M _d represents the binary value of the atom.

In this embodiment, the range of the sound pickup beam of the system can be adjusted by adjusting the size of δ. This embodiment does not limit the specific value of δ.

B2. Calculate the weighted sum of the frequency amplitude values of all atoms at each frequency of the dictionary matrix and the corresponding binary values, and the ratio of the sum of the frequency amplitude values of all atoms at this frequency, and combine each frequency with the corresponding ratio. frequency domain filter.

In this step, the value of the amplitude at each frequency in the frequency domain filter can be calculated by formula (3).

In the formula, M _d represents the binary value of the atom, W _fd represents the dictionary matrix, d represents the atom, and If represents the amplitude value at the frequency _f .

In this application, the calculated time delay of each atom represents the time difference between the preset speech components represented by the atom in the sampled speech signal reaching the dual microphones. Therefore, in the present application, among the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom that belongs to the preset range of the sound pickup beam as the target atom, and the target atom represents the sound to be picked up. Atoms of Directional Speech

Therefore, this step calculates the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix to the sum of the frequency amplitude values of all atoms, and the frequency domain filter composed of each frequency and the corresponding ratio reflects: The amplitude distribution of the speech in the direction to be picked up within the preset range of the sound pickup beam at each frequency, that is, the frequency distribution of the speech in the direction to be picked up. Therefore, using the filter obtained in this step to filter the first frequency domain signal and the second frequency domain signal respectively, the frequency domain of the voice in the direction to be picked up respectively contained in the first frequency domain signal and the second frequency domain signal can be obtained. Signal.

S207 , using a frequency domain filter to filter the first frequency domain signal and the second frequency domain signal, respectively, to obtain the filtered first frequency domain signal and the second frequency domain signal.

In this step, the way of filtering the first frequency domain signal and the second frequency domain signal respectively can be implemented by formula (4).

In the formula, I _f represents the frequency domain filter, X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal, Y _lf and Y _rf represent the filtered first frequency domain signal and the filtered second frequency domain signal. domain signal.

S208: Perform inverse time-frequency transform on the filtered first frequency domain signal and the filtered second frequency domain signal, respectively, to obtain a voice signal in the direction to be picked up in the sampled voice signal.

In this step, the time-frequency inverse transform can be an inverse Fourier transform. Of course, in practice, the time-frequency inverse transform can also be other inverse transform forms. This embodiment does not limit the specific form of the time-frequency inverse transform, as long as It is sufficient to correspond to the method of time-frequency transformation.

The purpose of the above S207-S208 is to determine the voice signal in the direction to be picked up in the sampled voice signal according to the frequency domain signal and the frequency domain filter.

It has been verified by experiments that the directional sound pickup solution under the dual microphones provided in this embodiment can suppress the interference of the non-to-be-picked sound direction by 15-18 dB in the directional sound pickup scenario of the main and auxiliary drivers with dual microphones in the vehicle. Therefore, it has strong performance. Interference suppression ability, that is, strong anti-interference ability.

In the scene where the main and co-pilot colleagues are talking, using the solution of this embodiment, the comparison diagram in the time domain of the input sampled voice signal and the output voice signal after pickup is shown in Figure 3(a), where the above In order to sample the voice signal, the following is the voice signal after pickup. The comparison in the frequency domain is shown in Fig. 3(b), where the upper part is the amplitude spectrum of the sampled speech signal, and the lower part is the amplitude spectrum of the speech signal after pickup. From Figure 3(a), it can be seen that the voice signal obtained by picking up the sound is significantly less than the sampled voice signal. It can be seen from Figure 3(b) that the amplitude spectrum obtained by picking up the voice is obviously less than that of the sampled speech.

This embodiment has the following beneficial effects:

Beneficial effect one:

In this embodiment, the difference between the corresponding time delay and the time delay of the speech in the direction to be picked up belongs to the target atoms in the preset sound pickup beam range, and the frequency amplitude values of the target atoms in the dictionary matrix at various frequencies , to obtain a frequency domain filter used to reflect the frequency distribution of the voice in the direction to be picked up within the preset sound pickup beam range. Since the frequency filter reflects: the frequency distribution of the voice in the direction to be picked up within the preset range of the sound pickup beam, this embodiment can improve the ability to suppress the voice signal in the spatial direction not to be picked up, and further, this embodiment The solutions provided by the embodiments have enhanced anti-interference capability and enhanced robustness.

Beneficial effect two:

Using the directional sound pickup solution provided by the dual microphones in this embodiment, it is only necessary to obtain the speech signals collected by the dual microphones and the dictionary matrix trained in advance to obtain the speech in the spatial direction to be picked up. test knowledge. In addition, the dictionary matrix is trained in advance, so that the speech in the direction to be picked up is picked up based on the trained dictionary matrix, which reduces the computational complexity of the reasoning process. Therefore, the real-time performance of this embodiment is good, and therefore, it has a high practical application value.

FIG. 4 provides a directional sound pickup device with dual microphones according to an embodiment of the present application, which may include: a first calculation module 401, a first determination module 402, a second calculation module 403, and a second determination module 404, wherein,

The first calculation module 401 is configured to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom represents the representation of the atom under the sampled speech signal The time difference between the preset speech components reaching the dual microphones. .

The first determination module 402 is configured to use, among the atoms of the dictionary matrix, the difference between the time delay and the time delay of the speech in the direction to be picked up, which belongs to the preset range of the sound pickup beam, as the target atom.

The second calculation module 403 is used to respectively calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio to represent In the voice signal, the frequency domain filter of the frequency distribution of the voice in the direction to be picked up.

The second determining module 404 is configured to determine, according to the frequency-domain signal and the frequency-domain filter, the speech signal in the direction to be picked up in the sampled speech signal.

Optionally, the second determining module 404 is configured to determine, according to the frequency domain signal and the frequency domain filter, the voice signal in the direction to be picked up in the sampled voice signal, including:

The second determination module 404 is specifically configured to use a frequency domain filter to filter the frequency domain signal to obtain a filtered frequency domain signal; perform time-frequency inverse transformation on the filtered frequency domain signal respectively to obtain the to-be-picked voice signal in the sampled voice signal. voice signal in the sound direction.

Optionally, the frequency domain signal includes: a first frequency domain signal and a second frequency domain signal; the first calculation module 402 is configured to calculate each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones. delays, including:

第一频域信号和第二频域信号，计算每个原子的时延函数；其中，f表示频率，d表示原子，W_fd表示所述字典矩阵；d表示时延；X_lf与X_rf分别表示第一频域信号和第二频域信号；F_dτ表示计算得到的原子的时延函数；分别针对每个原子的时延函数，将时延函数的取极大值情况下的时延，作为原子的时延，得到字典矩阵的各个原子的时延。The first calculation module 401 is specifically used for

The first frequency domain signal and the second frequency domain signal are used to calculate the delay function of each atom; wherein, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix; d represents the delay; X _lf and X _rf respectively Represents the first frequency domain signal and the second frequency domain signal; F _dτ represents the calculated atomic delay function; for the delay function of each atom, the delay when the delay function takes a maximum value, As the delay of the atom, the delay of each atom of the dictionary matrix is obtained.

Optionally, the second calculation module 403 is used to calculate the ratio of the sum of the frequency amplitude values of the target atoms under each frequency in the dictionary matrix and the sum of the frequency amplitude values of all atoms at the frequency, and compare each frequency with the corresponding value. The ratio constitutes a frequency domain filter used to characterize the frequency distribution of the voice in the direction to be picked up in the sampled voice signal, including:

The second calculation module 403 is specifically used for if the absolute value of the difference between the time delay of the atom of the dictionary matrix and the time delay of the voice in the direction to be picked up is less than the preset threshold, then the binary value of the atom is 1, Otherwise, the binary value is 0; calculate the weighted sum of the frequency amplitude values of all atoms under each frequency of the dictionary matrix and the corresponding binary values, and the ratio of the sum of the frequency amplitude values of all atoms at this frequency, and calculate each The frequency and the corresponding ratio constitute a frequency domain filter used to characterize the frequency distribution of the speech in the to-be-picked direction in the sampled speech signal.

The directional pickup device with dual microphones includes a processor and a memory. The above-mentioned first calculation module 401 , first determination module 402 , second calculation module 403 and second determination module 404 are all stored in the memory as program units, and are stored in the memory by the processor. The above-mentioned program elements stored in the memory are executed to realize the corresponding functions.

The processor includes a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to one or more, by adjusting the kernel parameters to provide a fast and accurate directional pickup method under dual microphones.

An embodiment of the present invention provides a storage medium on which a program is stored, and when the program is executed by a processor, the directional sound pickup method under the dual microphones is implemented.

An embodiment of the present invention provides a processor for running a program, wherein the directional sound pickup method under the dual microphones is executed when the program is running.

An embodiment of the present invention provides a device. As shown in FIG. 5 , the device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; processing The device is used to call the program instructions in the memory to execute the above-mentioned method for identifying the person in the same field. The devices in this article can be servers, PCs, PADs, mobile phones, and so on.

The application also provides a computer program product that, when executed on a data processing device, is adapted to execute a program initialized with the following method steps:

Calculate the time delay of each atom of the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom indicates that the preset speech component represented by the atom under the sampled speech signal reaches the dual microphones time difference;

In the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom of the preset sound pickup beam range, as the target atom;

Calculate the ratio of the sum of the frequency amplitude values of the target atoms under each frequency of the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio to characterize the speech signal, A frequency domain filter for the frequency distribution of the voice in the direction to be picked up;

According to the frequency domain signal and the frequency domain filter, the voice signal in the to-be-picked direction in the sampled voice signal is determined.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

In a typical configuration, a device includes one or more processors (CPUs), memory, and a bus. Devices may also include input/output interfaces, network interfaces, and the like.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read only memory (ROM) or flash memory (flash RAM), the memory including at least one memory chip. Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed or inherent to such a process, method, article of manufacture or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture or apparatus that includes the element.

It will be appreciated by those skilled in the art that the embodiments of the present application may be provided as a method, a system or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above are merely examples of the present application, and are not intended to limit the present application. Various modifications and variations of this application are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the scope of the claims of this application.

If the functions described in the methods of the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they may be stored in a readable storage medium of a computing device. Based on this understanding, the part of the embodiments of the present application that contribute to the prior art or the part of the technical solution may be embodied in the form of a software product, and the software product is stored in a storage medium and includes several instructions to make a A computing device (which may be a personal computer, a server, a mobile computing device or a network device, etc.) executes all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The features described in the various embodiments of this specification can be replaced or combined with each other, and each embodiment focuses on the differences from other embodiments, and the same or similar parts of the various embodiments can be referred to each other.

The above description of the disclosed embodiments enables any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, this application is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a directional pickup method under a dual microphone, is characterized in that, comprising: Calculate the time delay of each atom of the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom indicates that the preset speech component represented by the atom under the sampled speech signal reaches the dual microphones time difference; In the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up belongs to the atom of the preset sound pickup beam range, as the target atom; Calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency of the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio to characterize the sampled speech signal. , the frequency domain filter of the frequency distribution of the voice in the direction to be picked up; According to the frequency domain signal and the frequency domain filter, the voice signal in the to-be-picked direction in the sampled voice signal is determined. 2. The method according to claim 1, wherein the determining the voice signal in the direction to be picked up in the sampled voice signal according to the frequency domain signal and the frequency domain filter comprises: Filter the frequency-domain signal by using the frequency-domain filter to obtain a filtered frequency-domain signal; The time-frequency inverse transformation is performed on the filtered frequency domain signals respectively to obtain the voice signal in the to-be-picked direction in the sampled voice signal. 3. The method according to claim 1, wherein the frequency domain signal comprises: a first frequency domain signal and a second frequency domain signal; The time delay of each atom of the preset dictionary matrix is calculated according to the frequency domain signal of the sampled speech signal of the dual microphones, including: in accordance with

Calculate the delay function of each atom for the first frequency domain signal and the second frequency domain signal; wherein, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix; d represents the delay; X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal respectively; F _dτ represents the calculated atomic delay function; For the delay function of each atom, the delay when the delay function takes a maximum value is taken as the delay of the atom, and the delay of each atom of the dictionary matrix is obtained. 4. The method according to claim 1, wherein the ratio of the sum of the frequency amplitude values of the target atoms under each frequency in the dictionary matrix and the sum of the frequency amplitude values of all atoms at the frequency is calculated respectively. , and each frequency and the corresponding ratio are formed into a frequency domain filter used to characterize the frequency distribution of the voice in the direction of the sampled voice signal in the sampled voice signal, including: If the absolute value of the difference between the delay of the atom of the dictionary matrix and the delay of the voice in the direction to be picked up is smaller than the preset threshold, the binary value of the atom takes the value of 1, otherwise the binary value takes the value is 0; Calculate the weighted sum of the frequency amplitude values of all atoms and the corresponding binary values under each frequency of the dictionary matrix, and the ratio of the sum of the frequency amplitude values of all atoms at the frequency; A frequency domain filter used to characterize the frequency distribution of the speech in the to-be-picked direction in the sampled speech signal is composed of each frequency and the corresponding ratio. 5. The method according to claim 1, wherein the generation process of the preset dictionary matrix comprises: The preset training data corresponding to the two microphones are respectively subjected to time-frequency transformation and absolute values are obtained to obtain two non-negative amplitude spectrum matrices; Decompose the magnitude spectrum matrix into a target dictionary matrix and a coefficient matrix through a non-negative matrix decomposition algorithm; According to the preset objective function, the target dictionary matrix and the coefficient matrix are iteratively updated until the target dictionary matrix obtained when the objective function converges is used as the preset dictionary matrix. 6. A directional sound pickup device under a dual microphone is characterized in that, comprising: The first calculation module is used to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones; the time delay of each atom represents the time delay represented by the atom under the sampled speech signal. The time difference between the preset speech components reaching the dual microphones; The first determination module is used for taking the atoms in the atoms of the dictionary matrix, the difference between the time delay and the time delay of the voice in the direction to be picked up, which belongs to the preset range of the sound pickup beam, as the target atom; The second calculation module is used to calculate the ratio of the sum of the frequency amplitude values of the target atoms at each frequency of the dictionary matrix to the sum of the frequency amplitude values of all atoms at the frequency, and form each frequency and the corresponding ratio to represent the In the sampled voice signal, the frequency domain filter of the frequency distribution of the voice in the direction to be picked up; The second determination module is configured to determine, according to the frequency domain signal and the frequency domain filter, the speech signal in the to-be-picked direction in the sampled speech signal. 7 . The device according to claim 6 , wherein the second determining module is configured to determine the to-be-picked sound in the sampled speech signal according to the frequency-domain signal and the frequency-domain filter. 8 . Directional voice signals, including: The second determining module is specifically configured to use the frequency domain filter to filter the frequency domain signal to obtain a filtered frequency domain signal; respectively perform inverse time-frequency transform on the filtered frequency domain signal, The voice signal in the to-be-picked direction in the sampled voice signal is obtained. 8. The apparatus according to claim 6, wherein the frequency domain signal comprises: a first frequency domain signal and a second frequency domain signal; The first calculation module is used to calculate the time delay of each atom in the preset dictionary matrix according to the frequency domain signal of the sampled speech signal of the dual microphones, including: The first calculation module is specifically used for

Calculate the delay function of each atom for the first frequency domain signal and the second frequency domain signal; wherein, f represents the frequency, d represents the atom, and W _fd represents the dictionary matrix; d represents the delay; X _lf and X _rf represent the first frequency domain signal and the second frequency domain signal respectively; F _dτ represents the calculated atomic delay function; For the delay function of each atom, the delay when the delay function takes a maximum value is taken as the delay of the atom, and the delay of each atom of the dictionary matrix is obtained. 9 . The device according to claim 6 , wherein the second calculation module is used to calculate the sum of the frequency amplitude values of the target atoms at each frequency in the dictionary matrix and the sum of the frequency amplitudes of all atoms at the frequency. 10 . The ratio of the sum of frequency amplitude values, each frequency and the corresponding ratio are formed into a frequency domain filter used to characterize the frequency distribution of the voice in the direction to be picked up in the sampled voice signal, including: The second calculation module is specifically configured to, if the absolute value of the difference between the time delay of the atom of the dictionary matrix and the time delay of the voice in the direction to be picked up is less than the preset threshold, the binary value of the atom The value is 1, otherwise the binary value is 0; calculate the weighted sum of the frequency amplitude value of all atoms under each frequency of the dictionary matrix and the corresponding binary value, and, the frequency amplitude value of all atoms at this frequency The ratio of the sum and each frequency and the corresponding ratio are used to form a frequency domain filter for characterizing the frequency distribution of the voice in the direction to be picked up in the sampled voice signal. 10 . A storage medium, wherein the storage medium comprises a stored program, wherein the program executes the directional sound pickup method with dual microphones according to any one of claims 1 to 5 . 11 .

Images