CN116359843A - Sound source positioning method, device, intelligent equipment and medium - Google Patents

Abstract

The application discloses a sound source positioning method, a sound source positioning device, intelligent equipment and a sound source positioning medium, and belongs to the field of voice processing. The method comprises the following steps: acquiring audio time domain data acquired by at least two microphones in a microphone array for a sound source respectively and frequency response error compensation values of the microphones in the at least two microphones; calibrating the amplitude of each frame based on the corresponding frequency response error compensation value for each audio time domain data to obtain calibrated audio time domain data so that the amplitude of the corresponding frame in all the calibrated audio time domain data is consistent; and positioning the sound source based on all the calibrated audio time domain data. According to the method and the device, each microphone required for sound source positioning generates consistent amplitude signals for the same sound source, so that positioning accuracy in a sound source positioning method is improved.

Description

Sound source localization method, device, intelligent device and medium

technical field

The present application relates to the field of speech processing, and in particular to a sound source localization method, device, intelligent device and medium.

Background technique

In related technologies, smart devices such as smart speakers and smart TVs are equipped with far-field voice pickup. Among them, the front-end processing of the far-field voice pickup function is the sound source localization function, and the sound source localization function depends on the delay information between matching frames among multiple microphones in the microphone array set by the smart device.

However, due to the individual differences and installation errors between the microphones in the microphone array, each microphone will produce different frequency response data for the same sound source, which affects the calculation accuracy of the matching frame, which in turn leads to the calculation accuracy of the sound source localization function. reduce.

application content

The main purpose of the present application is to provide a sound source localization method, device, intelligent equipment and medium, aiming at solving the existing technical problem that the accuracy of sound source localization needs to be improved.

In order to achieve the above purpose, the present application provides a sound source localization method, which is used in a smart device, the smart device includes a microphone array, and the method includes:

Acquiring audio time-domain data respectively collected by at least two microphones in the microphone array for the sound source, and a frequency response error compensation value of each of the at least two microphones;

For each of the audio time domain data, based on the corresponding frequency response error compensation value, the amplitude of each frame is calibrated to obtain calibrated audio time domain data, so that all the calibrated audio time domain data The amplitudes of the corresponding frames in the data are consistent;

The sound source is localized based on all the calibrated audio time domain data.

In a possible embodiment of the present application, acquiring the frequency response error compensation value of each of the at least two microphones includes:

Acquiring the actual audio data collected by each of the at least two microphones for the same test sound source data;

determining the test audio energy value of each actual audio data;

Based on the test audio energy value, determine the reference audio energy value corresponding to the same test sound source data;

Based on the test audio energy value and the reference audio energy value, the frequency response error compensation value of the corresponding microphone is determined to obtain the frequency response error compensation value of each microphone.

In a possible embodiment of the present application, based on the test audio energy value and the reference audio energy value, the frequency response error compensation value of the corresponding microphone is determined to obtain the frequency response error compensation value of each microphone, including:

determining the ratio of the test audio energy value to the reference audio energy value;

Obtaining a compensation coefficient of a corresponding microphone based on the ratio;

Based on the compensation coefficient, a frequency response error compensation value of the corresponding microphone in the time domain is obtained.

In a possible embodiment of the present application, based on the test audio energy value, the reference audio energy value corresponding to the same test sound source data is determined, including:

determine the average energy value of all test audio energy values;

Use the average energy value as the reference audio energy value.

In a possible embodiment of the present application, the same test sound source data is preset frequency sweep audio data, and the preset frequency sweep audio data includes at least the audio of the target frequency band;

Determine the test audio energy value for each actual audio data, including:

A test audio energy value of audio distributed in a target frequency band in each actual audio data is determined.

In a possible embodiment of the present application, the sound source is located based on all the calibrated audio time domain data, including:

Obtain audio feature values in the calibrated audio time domain data;

Based on the audio feature value and the preset keyword feature value, any two of all the calibrated audio time domain data are compared, and the voice keyword frame group matching the audio feature value is selected;

Determining time delay information between two audio frames in the voice keyword frame group;

Based on the delay information, the sound source is located.

In a possible embodiment of the present application, based on the audio feature value and the preset keyword feature value, any two of all the calibrated audio time domain data are compared, and the voices with matching audio feature values are screened out. Keyword frame group, including:

Based on the audio characteristic value, compare any two calibrated audio time domain data, and filter out at least one audio frame group whose audio characteristic value matches;

A voice keyword frame group matching a preset keyword feature value is screened out from at least one audio frame group.

In the second aspect, the present application also provides a sound source localization device configured on a smart device, the smart device includes a microphone array, and the device includes:

An information acquisition module, configured to acquire audio time-domain data collected by at least two microphones in the microphone array for the sound source, and a frequency response error compensation value of each of the at least two microphones;

The audio calibration module is used to calibrate the amplitude of each frame based on the corresponding frequency response error compensation value for each audio time domain data, and obtain the calibrated audio time domain data, so that all calibrated audio time domain data The amplitudes of the corresponding frames in the data are consistent;

The sound source localization module is used for localizing the sound source based on all the calibrated audio time domain data.

In a third aspect, the present application also provides a smart device, including:

A microphone array, the microphone array includes a plurality of microphones, and the microphones are used to collect audio analog data;

an analog-to-digital converter for converting audio analog data into audio time-domain data;

The controller is connected with the analog-to-digital converter for receiving audio time-domain data. The controller includes a processor, a memory and a sound source localization program stored in the memory. When the sound source localization program is run by the processor, the sound source localization program is realized. The steps of the positioning method.

In a fourth aspect, the present application also provides a computer-readable storage medium, on which a sound source localization program is stored, and when the sound source localization program is run by a processor, the steps of the sound source localization method are realized.

Therefore, an embodiment of the present application provides a sound source localization method, including: acquiring audio time-domain data collected by at least two microphones in the microphone array for the sound source, and frequency response error compensation for each of the at least two microphones value; for each audio time domain data, based on the corresponding frequency response error compensation value, the amplitude of each frame is calibrated to obtain the calibrated audio time domain data, so that all the corresponding frames in the calibrated audio time domain data The amplitudes of the audio signals are consistent; based on all the calibrated audio time domain data, the sound source is localized.

Therefore, when the present application processes all the audio time-domain data collected by the microphone array, the frequency response error compensation value of each microphone will be used to compensate the amplitude of each frame in the audio time-domain data, thereby eliminating the The frequency response error caused by factors such as individual differences and assembly tolerances enables each microphone to generate a consistent amplitude signal for the same sound source, which facilitates subsequent data processing between different microphones, thereby improving the positioning accuracy in the sound source localization method. This makes the recognition accuracy of the far-field voice function of the smart device configured with the microphone array better.

Description of drawings

FIG. 1 is a schematic structural diagram of an intelligent device of the present application;

Fig. 2 is a schematic flow chart of the first embodiment of the sound source localization method of the present application;

Fig. 3 is a schematic flow chart of the second embodiment of the sound source localization method of the present application;

Fig. 4 is a schematic flow chart of the third embodiment of the sound source localization method of the present application;

Fig. 5 is a block diagram of the sound source localization device of the present application.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In related technologies, smart speakers, televisions and other smart devices equipped with a far-field voice pickup function are more and more useful in daily life. The front-end processing of the far-field voice pickup function is the sound source localization function, and the sound source localization function depends on the time delay information between matching frames among multiple microphones in the microphone array set by the smart device. Specifically, the sound source localization is to use multiple microphones to measure the sound signal at different positions in the environment. Since the time when the sound signal arrives at each microphone has different degrees of delay, the measured sound signal is processed by an algorithm, thus Obtain the arrival direction (including azimuth and elevation angle) and distance of the sound source point relative to the microphone.

However, in actual production, there must be individual differences between each microphone, as well as manufacturing errors, etc., resulting in a frequency response error between each microphone. Generally speaking, the allowable frequency response error of the microphone is ±3dB, that is For the same sound source, the frequency response difference between multiple microphones can reach up to 6dB. In addition, during the installation process of the microphone array, there is still an assembly error, for example, the allowable assembly error may be 0.03 mm. These individual differences and assembly errors will cause each microphone in the microphone array to produce different frequency responses to the same sound source, that is, the sound pressure value is different, which affects the calculation accuracy of the matching frame, thereby reducing the calculation accuracy of the sound source localization function. It also affects the calculation accuracy of speech recognition.

For this reason, the present application provides a solution, by confirming that each microphone in the microphone array collects the same sound source, the frequency response error compensation value required for the same frequency response is obtained, thereby using the frequency response error compensation value of each microphone to adjust the audio frequency The amplitude of each frame in the time domain data is compensated to eliminate the frequency response error of each microphone due to individual differences and assembly tolerances, so that each microphone in the microphone array generates a consistent amplitude signal for the same sound source, which is convenient for subsequent different The data processing between the microphones further improves the positioning accuracy in the sound source localization method, making the recognition accuracy of the far-field voice function of the smart device configured with the microphone array better.

The technical terms involved in this embodiment are described below:

The frequency range refers to the range between the lowest effective playback frequency and the highest effective playback frequency that the speaker can reproduce.

Frequency response refers to the phenomenon that when an audio signal output at a constant voltage is played through a speaker, the volume generated by the speaker increases or attenuates with the frequency, and the phase changes with the frequency. The associated change relationship (variation) is called the frequency response, and the unit is decibel (dB).

The frequency response error refers to the difference in the average volume produced within the same octave bandwidth, which is used to measure the accuracy of the sound reproduction of the speaker.

The following embodiments of this application will describe the smart devices used in the technical implementation of this application:

Referring to FIG. 2, FIG. 2 is a schematic structural diagram of a smart device in a hardware operating environment involved in the solution of the embodiment of the present application.

As shown in FIG. 2 , the smart device may include: a microphone array 107 , an analog-to-digital converter 1006 and a controller.

The microphone array 1007 includes a plurality of microphones, and the microphone array provided in this embodiment may be a miniature microphone array installed in, for example, a smart speaker or installed in a mobile phone, a tablet computer, and the like for sound pickup. A microphone is used to capture audio analog data.

The analog-to-digital converter 1006 is respectively connected with the microphone array 1007 and the controller, and is used for converting the audio analog data of the analog signal into the audio time domain data of the digital information format.

The controller includes: a processor 1001 , such as a central processing unit (Central Processing Unit, CPU), a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the structure shown in FIG. 2 does not constitute a limitation on the smart device, and may include more or less components than shown in the figure, or combine some components, or arrange different components.

As shown in FIG. 2 , the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a sound source localization program.

In the smart device shown in Figure 2, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the smart device of the present application can be set in In the smart device, the smart device invokes the sound source localization program stored in the memory 1005 through the processor 1001, and executes the sound source localization method provided in the embodiment of the present application.

Based on but not limited to the above-mentioned hardware structure of the smart device, the present application provides a first embodiment of a sound source localization method. Referring to FIG. 2 , FIG. 2 shows a schematic flowchart of a first embodiment of a sound source localization method of the present application.

It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

In this embodiment, the sound source localization method includes:

Step S100, acquiring audio time-domain data respectively collected by at least two microphones in the microphone array for the sound source, and a frequency response error compensation value of each of the at least two microphones.

Specifically, the execution subject of the method in this embodiment is a smart device. The smart device is equipped with a microphone array and has a far-field voice function. The smart device may be a smart speaker, a smart TV, or a mobile phone. It can be understood that the smart device can also be configured in the smart home system to receive the voice control command and send the voice control command to the server so that the corresponding smart home device executes the corresponding action. The following takes the smart device as a smart speaker as an example for specific description.

The sound source may be a user who gives a voice control command, and the user inputs a corresponding voice control command by speaking preset keywords. For example, in a study room scene, the microphone array in the smart speaker receives the sound information of "Xiao x, Xiao x" from the user.

It can be understood that for the sound source, each microphone in the microphone array collects an analog signal respectively, and the analog signal is converted into a digital signal through an ADC (Analog-to-Digital Converter, analog-to-digital converter), and then the smart speaker The SOC (System on Chip, system-on-chip) receives a digital signal, that is, the audio time-domain data in this embodiment is a digital signal of sound collected by each microphone.

The frequency response error compensation value is the value saved by the SOC of each smart speaker, and is the compensation value required for each microphone to obtain the same amplitude for the same audio. The frequency response error compensation value can be calculated by the manufacturer in the test environment by analyzing the recording files of the same test excitation signal collected by the microphone. The frequency response error compensation value can also be obtained when the smart speaker is equipped with a calibration mode, and the user can enter the calibration mode for measurement.

It is worth mentioning that, since each microphone must have a frequency response error, for the same sound source, the digital signal restored from the analog signals collected by multiple microphones on the same smart device will have an error in the audio energy value or loudness or sound pressure value. A certain deviation, the frequency response error compensation value in this embodiment is to make the digital time domain signal restored by each microphone tend to be consistent in the audio energy value or loudness or the amplitude of the sound pressure value, not necessarily to make each microphone restore The audio energy value or loudness or sound pressure value of the output digital signal is consistent with the audio energy value of the original sound signal.

Since the sound source localization function requires at least two microphones, in this embodiment, audio time domain data collected by at least two microphones and frequency response error compensation values of each of the at least two microphones need to be acquired. Of course, in order to improve the accuracy of sound source localization, audio time domain data collected by all microphones in the microphone array and frequency response error compensation values of all microphones may be used. The following uses the audio time domain data collected by all the microphones in the microphone array and the frequency response error compensation values of all the microphones as an example for illustration.

Step S200, for each audio time domain data, based on the corresponding frequency response error compensation value, the amplitude of each frame is calibrated, and the calibrated audio time domain data is obtained, so that all the calibrated audio time domain data correspond to The amplitude of the frame is consistent.

For any frame in the time-domain waveform signal obtained by converting the analog signal formed by each microphone, the coordinate value on the vertical axis of the time-domain waveform graph is the amplitude of the sound. Due to the influence of the frequency response error, the original amplitude corresponding to the same moment in the time-domain waveform data obtained by different microphones is different. In this embodiment, the original amplitude of each frame is compensated by the frequency response error compensation value corresponding to each microphone, so as to obtain new time-domain waveform data, that is, calibrated audio time-domain data. All calibrated audio data is consistent for the same audio amplitude of the sound.

For example, the amplitude of the audio frame sent by the sound source at a certain moment is A, and the amplitude corresponding to the audio frame in the first waveform data corresponding to the first microphone is data ₁ , and the frequency response error corresponding to the first microphone is The compensation value is ff ₁ , the amplitude corresponding to the audio frame in the second waveform data corresponding to the second microphone is data ₂ , and the frequency response error compensation value corresponding to the second microphone is ff ₂ , the third microphone corresponds to the third The amplitude corresponding to the audio frame in the waveform data is data ₃ , and the frequency response error compensation value corresponding to the third microphone is ff ₃ . Therefore, the amplitude corresponding to the audio frame in the first calibrated waveform data is data ₁ + ff ₁ , and the amplitude corresponding to the audio frame in the calibrated second waveform data is data ₂ + ff ₂ , after calibration The amplitude corresponding to the audio frame in the third waveform data is data ₃ +ff ₃ . After calibration, data ₁ +ff ₁ =data ₂ +ff ₂ =data ₃ +ff ₃ . As for the specific value relationship between data ₁ +ff ₁ =data ₂ +ff ₂ =data ₃ +ff ₃ and A, this embodiment does not limit it.

Step S300, based on all the calibrated audio time domain data, locate the sound source.

After the calibrated audio time domain data is obtained, the sound source localization algorithm can be processed through the consistent amplitude signal, thereby obtaining the arrival direction (including azimuth, elevation angle) and distance of the sound source point relative to the microphone, etc. , so as to realize sound source localization, and then realize the subsequent far-field voice pickup function.

In this embodiment, when processing multiple audio time-domain data collected by the microphone array, the frequency response error compensation value of each microphone is used to compensate the amplitude of each frame in the audio time-domain data, thereby eliminating the Due to the frequency response error caused by factors such as individual differences and assembly tolerances, each microphone in the microphone array generates a consistent amplitude signal for the same sound source, which facilitates subsequent data processing between different microphones, thereby improving the sound source localization method. The positioning accuracy makes the recognition accuracy of the far-field voice function of smart devices equipped with microphone arrays better.

Based on the foregoing embodiments, a second embodiment of the sound source localization method of the present application is provided. Referring to FIG. 3 , FIG. 3 is a schematic flowchart of a second embodiment of a sound source localization method of the present application.

It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

In this embodiment, the step of obtaining the frequency response error compensation value of each of the at least two microphones specifically includes:

Step S21 , acquiring actual audio data collected by each microphone for the same test sound source data.

Specifically, a microphone calibration mode can be configured for the smart speaker, and the user controls the smart speaker to enter the microphone calibration mode by inputting a corresponding preset command. After entering the microphone calibration mode, the user can play the test audio data through playback devices such as artificial mouth. It can be understood that, in order to adapt to the frequency response characteristic, the test audio data may be a piece of preset frequency sweep audio data output at the same volume, but whose frequency is continuously changed from 100 Hz to 20 khz. At the same time, the smart speaker controls the microphone array to start recording, and each microphone collects a recording file, thereby obtaining multiple recording files. The recording file is actual audio data, that is, the actual audio data collected by each microphone is obtained.

Step S22, determining the test audio energy value of each actual audio data.

The smart speaker can calculate the test audio energy value of each actual audio data according to the actual audio data: e ₁ , e ₂ ,... e _n ; n is the number of microphones in the microphone array, and e _n represents the test audio energy of the nth microphone value.

It is worth mentioning that in the field of speakers, 200HZ is the crossover point of the subwoofer, and 800HZ is the crossover point of the mid-bass. And the frequency distribution of people's general speech is 200-1000hz, so in one embodiment, step S22 specifically includes:

A test audio energy value of audio distributed in a target frequency band in each actual audio data is determined.

Among them, the target frequency band is 200hz-800hz. That is, in this embodiment, the smart speaker can calculate the test audio energy values of each actual audio data at 200hz-800hz according to the actual audio data: e ₁ , e ₂ , ... e _n . At this time, the test audio energy value may be an average audio energy value of audio distributed in the target frequency band in the actual audio data.

Step S23, based on the test audio energy value, determine the reference audio energy value corresponding to the same test sound source data.

After obtaining all test sample data, that is, all test audio energy values, a reference audio energy value can be determined according to part of the test audio energy values or all test audio energy values. It can be understood that it is more accurate to use the reference audio energy value determined according to all test audio energy values as a reference. For example, the reference audio energy value may be an average value, a mode number, or an average number of all test audio energy values.

As a specific implementation manner, step S23 specifically includes:

Step S231, determining the average energy value of all test audio energy values;

Step S232, taking the average energy value as the reference audio energy value.

并将平均能量值作为基准音频能量值e₀。Specifically, the average energy value of all test audio energy values is calculated

And take the average energy value as the reference audio energy value e ₀ .

Step S24: Determine the frequency response error compensation value of the corresponding microphone based on the test audio energy value and the reference audio energy value, so as to obtain the frequency response error compensation value of each microphone.

Through the numerical relationship between the test audio energy value and the reference audio energy value, the frequency response error compensation value of the microphone corresponding to the test audio energy value can be determined, thereby obtaining the frequency response error compensation value of each microphone.

As an embodiment, step S24 specifically includes:

Step S241. Determine the ratio of the test audio energy value to the reference audio energy value.

Step S242, based on the ratio, obtain the compensation coefficient of the corresponding microphone.

Step S243, based on the compensation coefficient, obtain the frequency response error compensation value of the corresponding microphone in the time domain.

然后根据比值r计算得到补偿系数f。Specifically, for the test audio energy values: e ₁ , e ₂ , . . . , e _n . Calculate the ratio r of each test audio energy value relative to the reference audio energy value e ₀ . like

Then calculate the compensation coefficient f according to the ratio r.

因此，可以计算得到各个麦克风对应的补偿系数：f₁、f₂、f₃、…、f_n。f_n为第n个麦克风的补偿系数。As an option, the compensation coefficient f can be calculated according to the ratio r and formula one. Formula 1 is specifically:

Therefore, compensation coefficients corresponding to each microphone can be calculated: f ₁ , f ₂ , f ₃ , . . . , f _n . f _n is the compensation coefficient of the nth microphone.

Then, based on the compensation coefficients corresponding to each microphone: f ₁ , f ₂ , f ₃ , . . . , f _n , the frequency response error compensation value of each microphone can be calculated and obtained.

As an embodiment, based on the compensation coefficients corresponding to each microphone: f ₁ , f ₂ , f _{3 , .} Formula two is:

ff _n = 2 ³¹ f _n ;

Wherein, ff _n is the frequency response error compensation value of the nth microphone.

In this embodiment, a method is provided to determine the numerical relationship between the audio energy values of the actual audio data obtained by each microphone for the same test sound source data, and then determine according to the numerical relationship so that each microphone can obtain the same amplitude when collecting the same audio The method of compensation value of the signal, the compensation value determined in this way can more accurately calibrate the amplitude of each frame, so as to calibrate the amplitude of the corresponding frames to be consistent.

Based on the foregoing embodiments, a third embodiment of the sound source localization method of the present application is provided. Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a third embodiment of a sound source localization method of the present application.

It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

In this embodiment, step S300 specifically includes:

Step S310, obtaining audio feature values in the calibrated audio time domain data.

Specifically, audio feature values include, but are not limited to, energy features such as root mean square energy of each frame of audio time domain data, partial waveforms, or global data, time domain features, frequency domain features, music theory features, and perceptual features.

Step S320 , based on the audio feature value and the preset keyword feature value, compare any two of all the calibrated audio time-domain data, and filter out the voice keyword frame group whose audio feature value matches.

After processing the calibrated audio time-domain data to obtain the audio characteristic value, any two calibrated audio time-domain data in all the calibrated audio time-domain data can be compared based on the audio characteristic value, thereby Filter out the voice keyword frame groups that match the audio feature value. The matching of the audio feature values means that the audio feature values of the two audio frames are consistent, and the voice keyword frame group is the feature that both audio frames include the voice keyword.

As an embodiment, step S320 specifically includes:

Step S321 , based on the audio characteristic value, compare any two of all the calibrated audio time-domain data, and filter out at least one audio frame group whose audio characteristic value matches.

Step S322 , filtering out the voice keyword frame group matching the preset keyword feature value from at least one audio frame group.

It can be understood that, since the distances between the microphones in the microphone array and the user are inconsistent, the time at which the user's voice signal is received is also inconsistent. When locating the sound source, it is necessary to filter out the audio frames of the same sound signal from the different calibrated audio time domain data, that is, to compare any two calibrated audio time domain data, and to filter out the corresponding audio feature values. At least one audio framegroup that matches. It can be understood that since the calibration process is performed through the frequency response error compensation value, the amplitude values of the audio frames for the same acoustic signal in different calibrated audio time domain data are consistent, that is, in different calibrated audio time domain data The waveforms for the same segment of audio signal are consistent, therefore, at least one audio frame group with matching audio feature values can be more accurately screened out.

Then, according to the preset keyword feature value, the voice keyword frame group matching the preset keyword feature value is selected from at least one audio frame group. The feature value of the preset keyword may be extracted according to the preset keyword. For example, if the preset keyword is "small x, small x", the smart speaker can extract the audio feature value of "small x, small x" and save it as a preset keyword feature value, and then at least one The eigenvalues of the audio frames in the audio frame group are compared with the preset keyword eigenvalues one by one, and the speech keyword frame group is screened out.

Step S330. Determine the time delay information between two audio frames in the voice keyword frame group.

Step S340, based on the delay information, locate the sound source.

After the voice keyword frame group is determined, the time delay information can be calculated according to the respective time stamps of the two audio frames, and then the sound source can be located based on the time delay information and the corresponding microphone position information.

For example, if the voice keyword frame group includes the audio frame at time t _i of the first microphone and the audio frame at time t _j of the second microphone, the delay information can be calculated according to |t _i -t _j |.

Based on the same inventive concept, please refer to Figure 5, the present application also provides a sound source localization device, including:

An information acquisition module, configured to acquire audio time-domain data collected by at least two microphones in the microphone array for the sound source, and a frequency response error compensation value of each of the at least two microphones;

The audio calibration module is used to calibrate the amplitude of each frame based on the corresponding frequency response error compensation value for each audio time domain data, and obtain the calibrated audio time domain data, so that all calibrated audio time domain data The amplitudes of the corresponding frames in the data are consistent;

The sound source localization module is used for localizing the sound source based on all the calibrated audio time domain data.

It should be noted that, for various implementations of the sound source localization device in this embodiment and the technical effects achieved by it, reference may be made to various implementations of the sound source localization method in the foregoing embodiments, which will not be repeated here.

In addition, the embodiment of the present application also proposes a computer storage medium, on which a sound source localization program is stored, and when the sound source localization program is executed by a processor, the steps of the above sound source localization method are realized. Therefore, details will not be repeated here. In addition, the description of the beneficial effect of adopting the same method will not be repeated here. For the technical details not disclosed in the embodiments of the computer-readable storage medium involved in the present application, please refer to the description of the method embodiments of the present application. Certainly for example, program instructions can be deployed to be executed on one computing device, or on multiple computing devices located at one site, or alternatively, on multiple computing devices distributed across multiple sites and interconnected by a communication network to execute.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware. The above programs can be stored in a computer-readable storage medium. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the above-mentioned storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

In addition, it should be noted that the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units , which can be located in one place, or can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. It can be understood and implemented by those skilled in the art without creative effort.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general-purpose hardware, and of course it can also be realized by special hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, Special components, etc. to achieve. In general, all functions completed by computer programs can be easily realized by corresponding hardware, and the specific hardware structure used to realize the same function can also be varied, such as analog circuits, digital circuits or special-purpose circuit etc. However, for this application, software program implementation is a better implementation mode in most cases. Based on this understanding, the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a computer floppy disk , U disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, RandomAccessMemory), magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, A server, or a network device, etc.) executes the methods of various embodiments of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims (10)

1. A method for localizing a sound source, characterized in that it is used in a smart device, and the smart device includes a microphone array, and the method comprises: Acquiring audio time-domain data respectively collected by at least two microphones in the microphone array for the sound source, and a frequency response error compensation value of each of the at least two microphones; For each of the audio time domain data, based on the corresponding frequency response error compensation value, the amplitude of each frame is calibrated to obtain calibrated audio time domain data, so that all the calibrated audio time domain data The amplitudes of the corresponding frames in the data are consistent; The sound source is localized based on all the calibrated audio time domain data. 2. The sound source localization method according to claim 1, wherein the obtaining the frequency response error compensation value of each of the at least two microphones comprises: Acquiring actual audio data collected by each of the at least two microphones for the same test sound source data; determining a test audio energy value for each of said actual audio data; Based on the test audio energy value, determine the reference audio energy value corresponding to the same test sound source data; Based on the test audio energy value and the reference audio energy value, determine the frequency response error compensation value of the corresponding microphone, so as to obtain the frequency response error compensation value of each of the microphones. 3. The sound source localization method according to claim 2, wherein the frequency response error compensation value of the corresponding microphone is determined based on the test audio energy value and the reference audio energy value, To obtain the frequency response error compensation value of each of the microphones, including: determining a ratio of the test audio energy value relative to the reference audio energy value; Obtaining a corresponding compensation coefficient of the microphone based on the ratio; Based on the compensation coefficient, a corresponding frequency response error compensation value of the microphone in the time domain is obtained. 4. The sound source localization method according to claim 2, wherein, based on the test audio energy value, determining the reference audio energy value corresponding to the same test sound source data comprises: determining an average energy value of all said test audio energy values; The average energy value is used as the reference audio energy value. 5. The sound source localization method according to any one of claims 2 to 4, wherein the same test sound source data is preset frequency sweep audio data, and the preset frequency sweep audio data includes at least the target frequency band audio; The determination of the test audio energy value of each of the actual audio data includes: Determining test audio energy values of audio distributed in the target frequency band in each of the actual audio data. 6. The sound source localization method according to claim 1, wherein the localization of the sound source based on all the calibrated audio time domain data comprises: Obtain audio feature values in the calibrated audio time domain data; Based on the audio characteristic value and the preset keyword characteristic value, any two of all the calibrated audio time domain data are compared, and the speech keyword frame group matching the audio characteristic value is selected. ; Determining delay information between two audio frames in the voice keyword frame group; Based on the time delay information, the sound source is located. 7. The sound source localization method according to claim 6, wherein, based on the audio feature value and the preset keyword feature value, any two in all the calibrated audio time domain data All compare, filter out the speech keyword frame group that described audio feature value matches, comprise: Based on the audio feature value, comparing any two of all the calibrated audio time-domain data, and screening out at least one audio frame group whose audio feature value matches; The voice keyword frame group matching the preset keyword feature value is screened out from at least one of the audio frame groups. 8. A sound source localization device, characterized in that it is configured in a smart device, the smart device includes a microphone array, and the device includes: An information acquisition module, configured to acquire audio time-domain data respectively collected by at least two microphones in the microphone array from the sound source, and a frequency response error compensation value of each of the at least two microphones; An audio calibration module, for each of the audio time domain data, based on the corresponding frequency response error compensation value, to calibrate the amplitude of each frame, to obtain the calibrated audio time domain data, so that all the The amplitudes of the corresponding frames in the calibrated audio time domain data are consistent; A sound source localization module, configured to localize the sound source based on all the calibrated audio time domain data. 9. A smart device, characterized in that it comprises: Microphone array, described microphone array comprises a plurality of microphones, and described microphone is used for collecting audio analog data; an analog-to-digital converter for converting said audio analog data into audio time-domain data; a controller, the controller is connected with the analog-to-digital converter and is used to receive the audio time-domain data, the controller includes a processor, a memory and a sound source localization program stored in the memory, the When the sound source localization program is run by the processor, the steps of the sound source localization method according to any one of claims 1-7 are realized. 10. A computer-readable storage medium, characterized in that a sound source localization program is stored on the computer-readable storage medium, and when the sound source localization program is run by the processor, any of claims 1-7 is implemented. A step of the sound source localization method.

Images