CN117935818B - Audio encoding and decoding device, method and system with automatic gain control function - Google Patents

Abstract

The present application provides an audio codec device, method and system with an automatic gain control function. The present application innovatively integrates environmental noise monitoring and AGC functions inside the audio codec, so that the AGC module can dynamically adjust the audio gain parameters according to the real-time environmental noise data, while ensuring the consistency and clarity of the audio output under various environmental conditions, thereby greatly improving the processing quality of the audio signal and the user's auditory experience. Secondly, integrating the AGC function into the audio codec significantly reduces external hardware dependence and the complexity of system integration, reduces the need for additional equipment, simplifies the system deployment process, and reduces the technical barriers to using AGC technology. Finally, integrating the AGC function into the audio codec can make full use of the high sampling rate capability of the audio codec to process audio signals with lower algorithm delays, thereby achieving faster and more accurate responses.

Description

Audio codec device, method and system with automatic gain control function

Technical Field

The present application relates to the field of audio coding and decoding, and in particular to an audio coding and decoding device, method and system with an automatic gain control function.

Background Art

The audio codec system is used to effectively transmit and store audio data between different connections and devices. It includes an audio encoder and an audio decoder. The encoder is used to convert the original analog audio signal into a digital format, while the decoder is used to reconvert the digital signal into an analog signal format for the playback of the audio signal. Audio codec technology can achieve fast and efficient network transmission of compressed audio files. Specifically, in mobile phone and conference system scenarios, audio codecs can ensure the clarity and reliability of voice communication.

Automatic Gain Control (AGC) is one of the commonly used technologies in the field of audio and signal processing. AGC technology automatically adjusts the strength of the output signal according to the preset target level based on the strength of the input signal monitored in real time to maintain the stability and consistency of the output signal, thereby ensuring that the output signal remains at a constant level. AGC technology can ensure that the receiver can receive clear and consistent sound regardless of how the sender's voice volume changes. Audio clarity and consistency are particularly important in voice communications, broadcasting, and conferencing systems. AGC can effectively reduce volume fluctuations caused by factors such as microphone distance, input volume changes, and background noise.

However, the difficulty in the existing AGC system is that the traditional AGC system lacks environmental monitoring function, which makes it difficult to distinguish different auditory backgrounds, and therefore it is difficult to adjust the target volume according to the user's environment. When the noise level of the surrounding environment changes, the AGC system can only maintain the volume at a fixed level, resulting in the volume being too low in a noisy environment or too high in a quiet environment, thus affecting the user experience of the audio receiver. Secondly, the parameter setting of the traditional AGC system has a high technical barrier, which requires professional knowledge to correctly configure and adjust its parameters. Such adjustments include but are not limited to setting the target volume level, adjusting the response speed, and handling sudden volume changes, which affects the popularity and efficiency of the AGC system. At the same time, the use of the AGC system in the process of audio encoding and decoding is highly complex and difficult to deploy.

Summary of the invention

In view of the shortcomings of the prior art mentioned above, the purpose of the present application is to provide an audio codec device, method and system with automatic gain control function, which is used to solve the problems of high difficulty in deploying the automatic gain control system in the audio coding and decoding process, high technical barriers in parameter setting, and the lack of environmental monitoring function of the automatic gain control system, thereby reducing the clarity and reliability of the transmitted audio.

To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides an audio codec device with an automatic gain control function, including: an analog-to-digital conversion unit, electrically connected to an environmental noise monitoring unit and an audio interface, the analog-to-digital conversion unit being used to receive a near-end audio analog signal and perform an analog-to-digital conversion operation, a data matching operation, and a data caching operation to generate a corresponding near-end audio digital signal, and output the near-end audio digital signal to the environmental noise monitoring unit and the data interface; an environmental noise monitoring unit, electrically connected to an automatic gain control unit, the environmental noise monitoring unit being used to receive the near-end audio digital signal A noise monitoring operation is performed to generate environmental noise data, and the environmental noise data is output to the automatic gain control unit; the automatic gain control unit is electrically connected to the control interface, the audio interface and the digital-to-analog conversion unit, and the automatic gain control unit is used to receive the environmental noise data from the environmental noise monitoring unit, receive the remote audio digital signal from the audio interface, and receive control information from the control interface, perform an automatic gain control operation on the remote audio digital signal based on the environmental noise data and the control information to generate an automatic gain audio signal, and output the automatic gain audio signal to the digital-to-analog conversion unit.

In some embodiments of the first aspect of the present application, the environmental noise monitoring unit includes: a preprocessing module, configured to perform a frequency domain preprocessing operation on the received near-end audio digital signal to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to a voice activity detection module; a voice activity detection module, configured to receive the near-end audio frequency domain signal, calculate the short-time energy and zero crossing rate of the near-end audio frequency domain signal, perform voice activity detection based on the short-time energy and zero crossing rate to generate a voice activity detection result, and send the voice activity detection result to a noise power spectrum density estimation module; a noise power spectrum density estimation module, configured to receive the voice activity detection result, perform noise power spectrum density analysis based on the voice activity detection result to generate a noise power spectrum density, and send the noise power spectrum density to a noise feature extraction module; a noise feature extraction module, configured to receive the noise power spectrum density, perform a noise feature extraction operation based on the noise power spectrum density to generate a comprehensive noise feature and send it to a noisiness measurement module; and a noisiness measurement module, configured to receive the comprehensive noise feature, calculate environmental noise data based on the comprehensive noise feature, and send it to the automatic gain control unit.

In some embodiments of the first aspect of the present application, the preprocessing module includes the following modules to perform frequency domain preprocessing operations: a DC component removal processing module, which is used to receive a near-end audio digital signal and remove the DC component in the signal, and send the near-end audio digital signal with the DC component removed to a window function processing module; a window function processing module, which is used to filter the received near-end audio digital signal with the DC component removed using a Hanning window function, and send the signal processed by the Hanning window function to a short-time Fourier transform module; a short-time Fourier transform module, which is used to receive the signal processed by the window function, convert it from the time domain to the frequency domain to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to the voice activity detection module.

In some embodiments of the first aspect of the present application, the noise feature extraction operation includes: calculating the total noise energy, spectrum flatness and spectrum abstract based on the noise power spectral density; wherein the comprehensive noise feature includes the total noise energy, the spectrum flatness and the spectrum abstract.

In some embodiments of the first aspect of the present application, the automatic gain control unit includes: a target volume determination module, which includes three input terminals and one output terminal, the three input terminals are respectively connected to the control interface, the audio interface and the environmental noise monitoring unit, and the output terminal is connected to the gain calculation module; the target volume determination module calculates the target volume value of the remote audio digital signal based on the control information and the environmental noisiness data; a gain calculation module, which includes an input terminal and an output terminal; the input terminal is connected to the target volume determination module, and the output terminal is connected to the gain smoothing adjustment module; the gain calculation module calculates the remote audio digital signal according to a preset gain coefficient and the target volume value of the remote audio digital signal The gain value is calculated and sent to a gain smoothing adjustment module; the gain smoothing adjustment module includes an input terminal and an output terminal; the input terminal is connected to the gain calculation module, and the output terminal is connected to the gain application module; the gain smoothing adjustment module performs a smoothing operation on the gain value of the remote audio digital signal based on a preset smoothing factor, and sends the smoothed gain value to the gain application module; the gain application module includes an input terminal and an output terminal, the input terminal is connected to the gain smoothing adjustment module, and the output terminal is connected to the digital-to-analog conversion unit; the gain application module obtains an automatic gain audio signal by calculating the smoothed gain value, and sends the automatic gain audio signal to the digital-to-analog conversion unit.

In some embodiments of the first aspect of the present application, the digital-to-analog conversion unit includes: a data cache module, which is electrically connected to the automatic gain control unit and the data matching module, and is used to perform delay matching and data cache operations on the automatic gain audio signal, and send it to the data matching module; a data matching module, which is electrically connected to the mixing module, and is used to perform format conversion and matching operations on the automatic gain audio signal to generate a digital-to-analog intermediate signal and send it to the mixing module; a mixing module, which is electrically connected to the digital-to-analog conversion submodule, and is used to mix and superimpose the digital-to-analog intermediate signal with the digital signals of other remote audio streams or the locally stored digital audio streams to generate a mixed signal and send the mixed signal to the digital-to-analog conversion submodule; a digital-to-analog conversion submodule, which is used to perform a digital-to-analog conversion operation on the mixed signal, and generate a remote audio analog signal and output it.

To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides an audio codec method with automatic gain control function, which is applied to an audio codec, and the method includes: receiving a near-end audio analog signal and performing analog-to-digital conversion operation, data matching operation and data caching operation to generate a corresponding near-end audio digital signal; performing a noise monitoring operation on the near-end audio digital signal to generate environmental noisy data; receiving control information and a far-end audio digital signal, and performing an automatic gain control operation on the far-end audio digital signal based on the environmental noisy data and the control information to generate an automatic gain audio signal; performing a data caching operation, a format conversion matching operation, a mixing and superposition operation and a digital-to-analog conversion operation on the automatic gain audio signal to generate a far-end audio analog signal that has undergone automatic gain control and output it.

In some embodiments of the second aspect of the present application, the noise monitoring operation includes: receiving a near-end audio digital signal and removing a DC component from the signal; filtering the near-end audio digital signal with the DC component removed using a Hanning window function; converting the near-end audio digital signal processed by the Hanning window function from the time domain to the frequency domain to generate a near-end audio frequency domain signal; calculating the short-time energy and zero-crossing rate of the near-end audio frequency domain signal, and performing voice activity detection based on the short-time energy and zero-crossing rate to generate a voice activity detection result; performing noise power spectral density analysis based on the voice activity detection result to generate a noise power spectral density; calculating the total noise energy, spectral flatness and spectral abstract based on the noise power spectral density to generate a comprehensive noise feature including the total noise energy, spectral flatness and spectral abstract, and calculating the environmental noisiness data through the comprehensive noise feature.

In some embodiments of the second aspect of the present application, the automatic gain control operation includes: calculating the target volume value of the far-end audio digital signal based on the control information and the ambient noise data; calculating the gain value of the far-end audio digital signal according to a preset gain coefficient and the target volume value of the far-end audio digital signal; smoothing the gain value of the far-end audio digital signal based on a preset smoothing factor; and obtaining an automatic gain audio signal by calculating the smoothed gain value.

To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides an audio codec system with automatic gain control function, including: a main processor and the above-mentioned audio codec device with automatic gain control function.

As described above, the audio codec device, method and system with automatic gain control function in the field of audio codec of the present application have the following beneficial effects: the present application innovatively integrates environmental noise monitoring and AGC functions inside the audio codec, so that the AGC module can dynamically adjust the audio gain parameters according to the real-time environmental noise data, while ensuring the consistency and clarity of the audio output under various environmental conditions, thereby greatly improving the processing quality of the audio signal and the user's auditory experience. Secondly, integrating the AGC function into the audio codec significantly reduces the external hardware dependence and the complexity of system integration, reduces the need for additional equipment, simplifies the system deployment process, and reduces the technical barriers to using AGC technology. Finally, integrating the AGC function into the audio codec can make full use of the high sampling rate capability of the audio codec to process the audio signal with a lower algorithm delay, thereby achieving a faster and more accurate response.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic structural diagram of external connections in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 2 shows a schematic structural diagram of the internal connections in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 3 shows a schematic diagram of the structure of an analog-to-digital conversion unit in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 4 shows a schematic diagram of the structure of an environmental noise monitoring unit in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 5 shows a schematic diagram of the structure of an automatic gain control unit in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 6 shows a schematic diagram of the structure of a digital-to-analog conversion unit in an embodiment of an audio codec device with an automatic gain control function of the present application.

FIG. 7 shows a flow chart of an embodiment of an audio encoding and decoding method with an automatic gain control function of the present application.

FIG. 8 shows a schematic structural diagram of an audio codec system with an automatic gain control function according to an embodiment of the present application.

DETAILED DESCRIPTION

The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

It should be noted that in the following description, with reference to the accompanying drawings, several embodiments of the present application are described in the accompanying drawings. It should be understood that other embodiments may also be used, and mechanical composition, structure, electrical and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used here are only for describing specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.

In this application, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", "fix", "hold" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and/or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and/or groups. The terms "or" and "and/or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and/or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.

In order to solve the problems in the above-mentioned background technology, the present invention provides an audio codec device, method and system with automatic gain control function, aiming to solve the problems that the automatic gain control system is difficult to deploy, there are high technical barriers to parameter setting, and the automatic gain control system does not have an environmental monitoring function during the audio coding and decoding process, thereby reducing the clarity and reliability of the transmitted audio. At the same time, in order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

<1> Automatic Gain Control (AGC): It is used to automatically adjust the gain level of the audio signal to ensure that the amplitude of the output signal is within an appropriate range so that the signal is not too strong or too weak. AGC is usually used in audio processing, wireless communication and other fields.

<2> DC removal: refers to removing the DC component from the audio signal to ensure that the mean value of the signal is zero. This can prevent the signal from deviating from the zero point and facilitate signal processing and encoding and decoding.

<3>Short-time energy: An indicator used to measure the instantaneous energy of a signal, showing the instantaneous changes and characteristics used to detect the signal.

<4>Zero crossing rate: refers to the number of times a signal crosses the zero value per unit time, and is often used for feature extraction and analysis of sound in audio signals.

<5>Noise power spectral density: refers to the power distribution of the noise signal in the frequency domain, which is usually used to analyze and process the spectral characteristics of the noise signal.

<6> Spectral flatness: used to describe the flatness of the signal spectrum, usually used for feature extraction and analysis of audio signals.

<7> Spectral abstract: It is used to describe the degree of skewness of the signal spectrum and can be used for feature analysis and processing of audio signals.

An embodiment of the present invention provides an audio codec device with an automatic gain control function, a method for performing automatic gain control in an audio codec, and a system used by the audio codec device with an automatic gain control function. As far as the structure of the audio codec device with an automatic gain control function is concerned, the embodiment of the present invention will illustrate an exemplary implementation scenario of the audio codec device with an automatic gain control function.

As shown in Figure 1, a schematic diagram of the external connection structure of an audio codec device with automatic gain control function in an embodiment of the present invention is shown. The audio codec device with automatic gain control function in this embodiment mainly includes the following parts.

In one embodiment of the present invention, the external connection of the audio codec device with automatic gain control function includes audio input, audio codec, control interface, data interface, main processor and audio output. The audio codec and the main processor are connected through the control interface and the data interface/audio interface. The interaction between the audio codec and the environment is realized through audio input and audio output.

Furthermore, the audio input is used to obtain an audio acquisition device for the audio codec to obtain an audio stream from the outside, and the audio acquisition device includes a digital microphone, an analog microphone, a sensor, or other forms of sound-to-electric conversion devices. The audio input of the audio acquisition device includes a single-channel signal and a multi-channel signal. The audio input module will complete the function of converting the acoustic signal into an electrical signal. Its input is a sound wave signal and its output is a voltage signal.

Exemplarily, the audio codec is used to convert audio data from one format to another format to reduce data calculation, transmission and storage costs. It includes an uplink (recording) path, a downlink (playing) path and a control interface. Among them, the uplink path includes an analog-to-digital conversion unit, a data format conversion unit, a data cache unit, and a data interface; the downlink path includes a data cache unit, a data format conversion unit, a mixing unit (multiple audio streams), and a digital-to-analog conversion unit. The audio codec system of the present invention is based on the architecture of a conventional audio codec system, and adds an environmental noise detection and automatic gain control control module to realize automatic evaluation and adaptation to environmental noisiness, thereby performing automatic gain control.

Furthermore, the control interface is a channel for the main processor to send commands to the audio codec, and is also a channel for the audio codec to feedback status to the main processor. The communication protocols that can be used include: I2C, Soundwire, Slimbus, HDA and other interface protocols. Through the control interface, the user can communicate control instructions with the codec. In an audio codec device with an automatic gain control function, the main processor sends control information to the automatic gain control module through the control interface.

Exemplarily, the data interface is a channel for audio data interaction between the main processor and the audio codec. For example, the original audio stream is sent to the main processor through the interface, and the main processor transmits the remote audio signal or the local music signal to the audio codec through the interface. The protocols that can be used include but are not limited to: I2S/PCM/TDM, Soundwire, Slimbus, HDA and other interface protocols.

Furthermore, the main processor is a SOC (System On Chip) processing chip, which is responsible for the main control function of the device. Specifically, the main processor receives and processes the audio data of the audio codec upstream, or transmits the audio data to the audio codec downstream channel for further processing. At the same time, through the control interface, it monitors the status of the audio codec, or sends parameter configuration to it.

Exemplarily, the audio output is the audio codec outputting the audio stream to an external sound playing device, wherein the sound playing device includes a receiver, a speaker or an earphone, or a combination of the above three, which can be single-channel or multi-channel, to complete the function of converting electrical signals into sound signals. Its input is a voltage signal and its output is a sound wave signal.

The above describes the external connection structure of the audio codec device with automatic gain control function. The following will describe the internal structure of the audio codec device with automatic gain control function in detail in conjunction with Figure 2. As shown in Figure 2, the schematic diagram of the external connection structure of the audio codec device with automatic gain control function includes the following structure.

An analog-to-digital conversion unit is electrically connected to the environmental noise monitoring unit and the audio interface. The analog-to-digital conversion unit is used to receive a near-end audio analog signal and perform analog-to-digital conversion operations, data matching operations, and data caching operations to generate a corresponding near-end audio digital signal, and output the near-end audio digital signal to the environmental noise monitoring unit and the data interface.

In one embodiment of the present invention, the analog-to-digital conversion unit is used to convert an external audio input signal into a digital signal that meets the current system processing requirements. Specifically, as shown in FIG3 , the analog-to-digital conversion unit includes an analog-to-digital conversion module, a data matching module, and a data cache module. The unit inputs a near-end audio analog signal and outputs a near-end audio digital signal. The number of input signals of the unit is consistent with the number of output signals.

Exemplarily, the analog-to-digital conversion module samples and quantizes the analog audio stream collected by the audio input and converts it into a digital audio stream so that subsequent processing is based on digital discrete signals.

x(n)＝x(nT) (Formula 1)

_xq (n)＝Q[x(n)] (Formula 2)

Among them, formula 1 shows the sampling process in the analog-to-digital conversion module, where the input analog continuous signal x(t) is sampled in discrete time according to the sampling period T, and the output is x(n). Formula 2 shows the quantization process of the audio data, where the output x(n) of the sampling process is discretized by the quantization function Q, and the output is _xq (n). After the above two processes, the analog continuous signal is converted into an output digital discrete signal.

Furthermore, the data matching module performs certain format conversion on the converted digital audio stream as required, including matching of sampling rate, matching of signal bit width, etc.

Among them, Formula 3 is the filtering and sampling rate conversion of the input signal, x(n) is the output of the analog-to-digital conversion module, M is the downsampling change factor, I is the upsampling change factor, h(k) is the unit impulse response function, and the output is y _d (n) or _yu (n). Formula 4 shows the process of signal bit width matching. The data matching module determines whether the bit width matching operation is left shift to enlarge or right shift to reduce according to the positive and negative of the shift bit width parameter B. The input signal of this unit is the analog-to-digital intermediate signal, and the output is a digital audio signal.

Furthermore, the data buffer module is used to buffer a certain amount of data in the audio interface, thereby avoiding the problem of repeated sampling or loss of audio data due to jitter introduced by the system clock design defects on both sides of the interface. The input of this module is a digital signal, and there is no output data stream.

In one embodiment of the present invention, as shown in Figure 4, the audio codec device with automatic gain control function also includes an environmental noise monitoring unit, which is electrically connected to the automatic gain control unit, and the environmental noise monitoring unit is used to perform a noise monitoring operation on the received near-end audio digital signal to generate environmental noise data, and output the environmental noise data to the automatic gain control unit.

It is worth noting that the environmental noise monitoring unit in the present invention is used to analyze and evaluate the noise level of the surrounding environment in real time in order to optimize the audio processing algorithm, especially the adaptive automatic gain control (AGC) algorithm. The environmental noise monitoring unit uses a variety of signal processing techniques to accurately capture and quantify the characteristics of environmental noise. The environmental noise monitoring unit removes the DC component in the original audio signal and eliminates the long-term average deviation, thereby accurately capturing the instantaneous changes of the signal. Subsequently, by applying the Hanning window function, the time localization processing of the signal is realized to reduce spectrum leakage. Next, the short-time Fourier transform (STFT) is used to convert the processed time domain signal into a frequency domain representation, which provides the necessary data basis for subsequent analysis. More importantly, the method of extracting comprehensive noise features in the present invention provides a multi-dimensional perspective for environmental noise and improves the accuracy of capturing environmental noisiness.

In this embodiment, as shown in FIG. 4 , the environmental noise monitoring unit includes: a preprocessing module, a voice activity detection module, a noise power spectrum density estimation module and a noisiness measurement module. The preprocessing module is used to perform a frequency domain preprocessing operation on the received near-end audio digital signal to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to the voice activity detection module; the voice activity detection module is used to receive the near-end audio frequency domain signal, calculate the short-time energy and zero crossing rate of the near-end audio frequency domain signal, perform voice activity detection based on the short-time energy and zero crossing rate to generate a voice activity detection result, and send the voice activity detection result to the noise power spectrum density estimation module; the noise power spectrum density estimation module is used to receive the voice activity detection result, perform noise power spectrum density analysis based on the voice activity detection result to generate a noise power spectrum density, and send the noise power spectrum density to the noise feature extraction module; the noise feature extraction module is used to receive the noise power spectrum density, perform a noise feature extraction operation based on the noise power spectrum density to generate a comprehensive noise feature and send it to the annoyance measurement module; the annoyance measurement module is used to receive the comprehensive noise feature, calculate the environmental annoyance data based on the comprehensive noise feature and send it to the automatic gain control unit.

Furthermore, the preprocessing module includes the following modules to perform frequency domain preprocessing operations: a DC component removal processing module, which is used to receive a near-end audio digital signal and remove the DC component in the signal, and send the near-end audio digital signal with the DC component removed to a window function processing module; a window function processing module, which is used to filter the received near-end audio digital signal with the DC component removed using a Hanning window function, and send the signal processed by the Hanning window function to a short-time Fourier transform module; a short-time Fourier transform module, which is used to receive the signal processed by the window function, convert it from the time domain to the frequency domain to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to the voice activity detection module.

Exemplarily, the DC component in the original sound signal s(t) is removed to eliminate the long-term average deviation as shown in Formula 5.

Among them, s(t) is the original time domain sound signal; s′(t) is the signal after removing the DC component; T is the time length considered.

Then, as shown in Formula 6, the formula after removing the DC component uses the Hanning window function to localize the signal to reduce spectrum leakage.

Wherein, s _w (t) is the signal after the window function is applied; T is the length of the window function.

Subsequently, the processed time domain signal is converted into a frequency domain signal using Formula 7 for further analysis and processing.

Among them, S(f,τ) is the frequency domain representation of frequency f and time window τ.

In one embodiment of the present invention, the process of performing voice activity detection on the near-end audio frequency domain signal combines short-time energy and zero crossing rate to determine whether there is voice in the signal. Specifically, as shown in formulas 8 to 10. Among them, formula 8 shows the process of detecting the near-end audio frequency domain signal using short-time energy. Formula 9 shows the process of detecting the near-end audio frequency domain signal using zero crossing rate. Finally, the final VAD decision is made based on the short-time energy and zero crossing rate through formula 10.

Wherein θ _E represents the energy sum of the near-end audio signal, and θ _Z represents the preset threshold value of the zero-crossing rate.

In one embodiment of the present invention, the noise feature extraction operation includes: calculating the total noise energy, spectrum flatness and spectrum abstract based on the noise power spectral density; wherein the comprehensive noise feature includes the total noise energy, the spectrum flatness and the spectrum abstract.

Exemplarily, when no voice activity is detected in the voice activity detection unit, the process of estimating the noise power spectrum density for each frequency f and time window τ is as shown in Formula 11.

Where N _ns represents the number of samples that VAD judges to represent non-speech, and N(f,τ) represents the noise power spectral density at frequency f and time window τ.

Furthermore, in the environmental noise monitoring module, the following noise features are extracted for further analysis and processing including: calculating total energy, calculating spectrum flatness, and calculating spectrum abstracts.

Exemplarily, Equation 12 shows the process of total energy extraction from noise features.

E _noise (τ)＝∑ _f |N(f,τ)| ² (Formula 12)

Wherein, E _{noise (τ)} represents the sum of the squares of the noise power spectral density N (f, τ) of all frequencies f in the time window τ, that is, the total energy level of the noise in the time window. F (τ) represents the ratio of the geometric mean to the arithmetic mean of the noise power spectral density N (f, τ) in the time window τ, which is used to measure the uniformity of the spectrum. F represents the total number of frequency points considered in the analysis. For example, if the analyzed spectrum ranges from 0 Hz to 1000 Hz, and there is a frequency point every 100 Hz, then F is 10.

Exemplarily, Formulas 13 and 14 illustrate the process of extracting spectral flatness and spectral abstract from noise features.

Among them, H(τ) represents the spectrum abstract under the time window τ, that is, the distribution complexity of the power spectral density N(f,τ) of the frequency component f in the total power spectrum. A high value of the spectrum abstract indicates that the spectrum is more evenly distributed, while a low value indicates that the spectrum is more concentrated on certain specific frequencies.

After the total energy, spectrum flatness and spectrum extract are calculated, the noisiness of the ambient noise is further calculated according to the total energy, spectrum flatness and spectrum extract of the near-end audio signal using formula 15.

D(τ)＝α·E _noise (τ)+β·F(τ)+γ·H(τ) (Formula 15)

Where D(τ) represents the environmental noise level in the time window τ. α, β, and γ are preset weighting parameters.

The above describes in detail the process of the environmental noise monitoring unit monitoring the environmental noise. The following will describe the process of automatically controlling the gain of the output audio based on the environmental noise in conjunction with FIG. 5 .

In one embodiment of the present invention, as shown in FIG5 , the automatic gain control unit includes: a target volume determination module, which includes three input terminals and one output terminal, the three input terminals are respectively connected to the control interface, the audio interface and the environmental noise monitoring unit, and the output terminal is connected to the gain calculation module; the target volume determination module calculates the target volume value of the remote audio digital signal based on the control information and the environmental noisiness data; a gain calculation module, which includes an input terminal and an output terminal; the input terminal is connected to the target volume determination module, and the output terminal is connected to the gain smoothing adjustment module; the gain calculation module calculates the target volume value of the remote audio digital signal according to a preset gain coefficient and the target volume value of the remote audio digital signal The gain value is calculated and sent to a gain smoothing adjustment module; the gain smoothing adjustment module includes an input terminal and an output terminal; the input terminal is connected to the gain calculation module, and the output terminal is connected to the gain application module; the gain smoothing adjustment module performs a smoothing operation on the gain value of the remote audio digital signal based on a preset smoothing factor, and sends the smoothed gain value to the gain application module; the gain application module includes an input terminal and an output terminal, the input terminal is connected to the gain smoothing adjustment module, and the output terminal is connected to the digital-to-analog conversion unit; the gain application module obtains an automatic gain audio signal by calculating the smoothed gain value, and sends the automatic gain audio signal to the digital-to-analog conversion unit.

In this embodiment, the target volume value is first determined according to the environmental noise level to ensure that the output audio signal after adaptive gain processing does not exceed the safe maximum volume. The specific process is shown in Formula 16.

V _target (τ)=min(V _base +k·(D(τ)-D _min ),V _max ) (Formula 14)

Wherein: V _target (τ) represents the target volume value of the time window τ; V _base represents the reference volume value, that is, the standard volume level when at the lowest noisiness; D (τ) represents the noisiness of the time window τ; D _min represents the minimum threshold of noisiness; when the ambient noise is lower than this value, the target volume maintains the baseline level; k represents the coefficient for controlling the intensity of the target volume adjustment; V _max represents the set maximum volume limit for hearing protection.

After determining the safe maximum volume, the required gain parameters are calculated based on the target volume and the current volume using Formula 15.

Where G(τ) represents the gain coefficient of the time window τ. E _current (τ) represents the actual sound energy of the current time window τ.

In order to further improve the quality, stability and applicability of the automatic gain audio signal, the gain parameter obtained by Formula 15 is smoothly adjusted by Formula 16 to avoid sudden changes in volume.

G _smooth (τ)＝β·G(τ)+(1-β)·G _smooth (τ-1) (Formula 16)

Wherein, G _smooth (τ) represents the gain of the smoothed time window τ, and β represents the smoothness of the gain adjustment.

The smoothing parameter calculated by formula 16 is then added to the output audio to adjust the volume of the signal.

y(t)＝G _smooth (τ)·s(t) (Formula 17)

Among them, y(t) represents the signal after gain adjustment, and s(t) represents the output signal.

The above describes in detail the process of generating the automatic gain audio signal, and the following describes in detail the process of outputting audio based on the automatic gain audio signal in conjunction with Figure 6. The automatic gain control unit sends the automatic gain audio signal to the digital-to-analog conversion unit.

In one embodiment of the present invention, as shown in Figure 6, the digital-to-analog conversion unit includes: a data cache module, which is electrically connected to the automatic gain control unit and the data matching module, and is used to perform delay matching and data cache operations on the automatic gain audio signal, and send it to the data matching module; a data matching module, which is electrically connected to the mixing module, and is used to perform format conversion and matching operations on the automatic gain audio signal to generate a digital-to-analog intermediate signal and send it to the mixing module; a mixing module, which is electrically connected to the digital-to-analog conversion submodule, and is used to perform mixing and superimposing operations on the digital-to-analog intermediate signal with digital signals of other remote audio streams or locally stored digital audio streams to generate a mixed signal and send the mixed signal to the digital-to-analog conversion submodule; a digital-to-analog conversion submodule, which is used to perform a digital-to-analog conversion operation on the mixed signal, and generate a remote audio analog signal and output it.

In this embodiment, the digital-to-analog conversion submodule is mainly responsible for converting the digital signal into an audio output signal (usually an analog signal). The submodules included therein include a digital-to-analog conversion submodule, a mixing module, a data matching module, and a data buffer module. The data matching module is used to convert the digital audio stream into a certain format as required, including operations such as matching the sampling rate and matching the signal bit width, and the data buffer module is used to buffer a certain amount of data, thereby avoiding the problem of repeated sampling or loss of audio data due to jitter introduced by the system clock design defects on both sides of the interface.

In this embodiment, the audio mixing module is used to mix and superimpose multiple downstream audio streams, such as a remote audio stream transmitted from the audio codec main processor through a data interface or a locally stored audio stream.

Wherein, Formula 18 indicates that there are M input audio streams x(n), and after accumulating them, the output mixed audio stream y(n) is obtained.

Furthermore, the digital-to-analog conversion submodule is used to convert the digital audio stream processed by the codec into an analog audio stream.

v = ky(t) (Formula 20)

Among them, Formula 19 is the process of increasing the sampling rate to the analog processing sampling rate through the interpolation process, and the sampling process is realized by interpolation fitting between digital sample points. Formula 20 is the process of converting digital signals into analog signals. The input digital discrete signal y(n) is multiplied by the proportional factor k to obtain the output analog voltage signal v(t) to realize the function of converting discrete digital signals into continuous analog signals.

The audio codec device with automatic gain control function is described in detail above. The audio codec method with automatic gain control function will be described below in conjunction with FIG. 7 , wherein the method is applied to an audio codec.

Step S71: receiving a near-end audio analog signal and performing an analog-to-digital conversion operation, a data matching operation, and a data buffering operation to generate a corresponding near-end audio digital signal.

Step S72: performing a noise monitoring operation on the near-end audio digital signal to generate environmental noisiness data.

In one embodiment of the present invention, the noise monitoring operation includes: receiving a near-end audio digital signal and removing a DC component from the signal; filtering the near-end audio digital signal with the DC component removed using a Hanning window function; converting the near-end audio digital signal processed by the Hanning window function from the time domain to the frequency domain to generate a near-end audio frequency domain signal; calculating the short-time energy and zero-crossing rate of the near-end audio frequency domain signal, and performing voice activity detection based on the short-time energy and zero-crossing rate to generate a voice activity detection result; performing noise power spectral density analysis based on the voice activity detection result to generate a noise power spectral density; calculating the total noise energy, spectral flatness and spectral abstract based on the noise power spectral density to generate a comprehensive noise feature including the total noise energy, spectral flatness and spectral abstract, and obtaining environmental noisiness data through calculation through the comprehensive noise feature.

Step S73: receiving control information and a far-end audio digital signal, and performing an automatic gain control operation on the far-end audio digital signal based on the environmental noise data and the control information to generate an automatic gain audio signal.

In one embodiment of the present invention, the automatic gain control operation includes: calculating the target volume value of the far-end audio digital signal based on the control information and the environmental noise data; calculating the gain value of the far-end audio digital signal according to a preset gain coefficient and the target volume value of the far-end audio digital signal; smoothing the gain value of the far-end audio digital signal based on a preset smoothing factor; and obtaining an automatic gain audio signal by calculating the smoothed gain value.

Step S74: performing data caching operation, format conversion and matching operation, mixing and superposition operation and digital-to-analog conversion operation on the automatic gain audio signal, and generating and outputting a remote audio analog signal that has undergone automatic gain control.

It should be noted that: when the audio codec method with automatic gain control function provided in the above embodiment performs automatic gain control in the audio codec, only the division of the above program modules is used as an example. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the method is divided into different program modules to complete all or part of the processing described above. In addition, the audio codec method with automatic gain control function provided in the above embodiment and the audio codec device embodiment with automatic gain control function belong to the same concept. The specific implementation process is detailed in the device embodiment and will not be repeated here.

As shown in Fig. 8, a schematic diagram of the structure of an audio codec device with automatic gain control function in an embodiment of the present invention is shown. In this embodiment, the audio codec device with automatic gain control function 800 includes: a main processor 801 and the audio codec device with automatic gain control function 802 as described above.

In summary, the present application provides an audio codec device, method and system with an automatic gain control function. The present invention provides a method for improving the efficiency of adaptive gain control of audio signals within an audio codec. The present application innovatively integrates environmental noise monitoring and AGC functions within the audio codec, so that the AGC module can dynamically adjust the audio gain parameters according to real-time environmental noise data while ensuring the consistency and clarity of the audio output under various environmental conditions, thereby greatly improving the processing quality of the audio signal and the user's auditory experience. Secondly, integrating the AGC function into the audio codec significantly reduces the external hardware dependence and the complexity of system integration, reduces the need for additional equipment, simplifies the system deployment process, and reduces the technical barriers to using AGC technology. Finally, integrating the AGC function into the audio codec can make full use of the high sampling rate capability of the audio codec to process the audio signal with a lower algorithm delay, thereby achieving a faster and more accurate response. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims (9)

1. An audio codec device with automatic gain control function, comprising: an analog-to-digital conversion unit, electrically connected to the environmental noise monitoring unit and the audio interface, the analog-to-digital conversion unit being used to receive a near-end audio analog signal and perform an analog-to-digital conversion operation, a data matching operation, and a data caching operation to generate a corresponding near-end audio digital signal, and output the near-end audio digital signal to the environmental noise monitoring unit and the data interface; An environmental noise monitoring unit is electrically connected to the automatic gain control unit, and is used to perform a noise monitoring operation on the received near-end audio digital signal to generate environmental noisy data, and output the environmental noisy data to the automatic gain control unit; the environmental noise monitoring unit includes a noise feature extraction module, which is used to receive the noise power spectrum density, perform a noise feature extraction operation based on the noise power spectrum density, to generate a comprehensive noise feature and send it to the noisy measurement module; the noise feature extraction operation includes: calculating the total noise energy, spectrum flatness and spectrum abstract based on the noise power spectrum density; wherein the comprehensive noise feature includes the total noise energy, the spectrum flatness and the spectrum abstract; the spectrum flatness is the ratio of the geometric mean to the arithmetic mean of the noise power spectrum density in the time window, which is used to measure the uniformity of the spectrum, and the spectrum abstract is the distribution complexity of the power spectrum density of the frequency component in the total power spectrum; An automatic gain control unit is electrically connected to the control interface, the audio interface and the digital-to-analog conversion unit, and is used to receive environmental noise data from the environmental noise monitoring unit, receive a remote audio digital signal from the audio interface, and receive control information from the control interface, perform an automatic gain control operation on the remote audio digital signal based on the environmental noise data and the control information to generate an automatic gain audio signal, and output the automatic gain audio signal to the digital-to-analog conversion unit. 2. The audio codec device with automatic gain control function according to claim 1, wherein the environmental noise monitoring unit comprises: A preprocessing module, configured to perform a frequency domain preprocessing operation on the received near-end audio digital signal to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to the voice activity detection module; a voice activity detection module, configured to receive the near-end audio frequency domain signal, calculate the short-time energy and the zero-crossing rate of the near-end audio frequency domain signal, perform voice activity detection based on the short-time energy and the zero-crossing rate to generate a voice activity detection result, and send the voice activity detection result to the noise power spectral density estimation module; a noise power spectrum density estimation module, configured to receive the voice activity detection result, perform noise power spectrum density analysis based on the voice activity detection result to generate a noise power spectrum density, and send the noise power spectrum density to the noise feature extraction module; The noisiness measurement module is used to receive the comprehensive noise characteristics, calculate the environmental noise data based on the comprehensive noise characteristics and send it to the automatic gain control unit; the calculation method of the environmental noise data includes: D(τ)=α·E_noise(τ)+β·F(τ)+γ·H(τ); Wherein, D(τ) represents the environmental noise level in the time window τ; F(τ) represents the spectrum flatness, that is, the ratio of the geometric mean to the arithmetic mean of the noise power spectrum density N(f,τ) in the time window τ, which is used to measure the uniformity of the spectrum; H(τ) represents the spectrum abstract under the time window τ, that is, the distribution complexity of the power spectrum density N(f,τ) of the frequency component f in the total power spectrum; α, β, and γ represent the first preset weighting parameter, the second preset weighting parameter, and the third preset weighting parameter, respectively. 3. The audio codec device with automatic gain control function according to claim 2, characterized in that the preprocessing module includes the following modules to perform frequency domain preprocessing operations: A DC component removal processing module is used to receive a near-end audio digital signal and remove the DC component in the signal, and send the near-end audio digital signal with the DC component removed to the window function processing module; A window function processing module is used to filter the received near-end audio digital signal with the DC component removed using a Hanning window function, and send the signal processed by the Hanning window function to a short-time Fourier transform module; The short-time Fourier transform module is used to receive the signal processed by the window function, convert it from the time domain to the frequency domain to generate a near-end audio frequency domain signal, and send the near-end audio frequency domain signal to the voice activity detection module. 4. The audio codec device with automatic gain control function according to claim 1, characterized in that the automatic gain control unit comprises: A target volume determination module, comprising three input terminals and one output terminal, wherein the three input terminals are respectively connected to the control interface, the audio interface and the environmental noise monitoring unit, and the output terminal is connected to the gain calculation module; the target volume determination module calculates the target volume value of the remote audio digital signal based on the control information and the environmental noisiness data; A gain calculation module, comprising an input end and an output end; the input end is connected to the target volume determination module, and the output end is connected to the gain smoothing adjustment module; the gain calculation module calculates the gain value of the remote audio digital signal according to a preset gain coefficient and the target volume value of the remote audio digital signal, and sends the gain value to the gain smoothing adjustment module; A gain smoothing adjustment module, comprising an input end and an output end; the input end is connected to the gain calculation module, and the output end is connected to the gain application module; the gain smoothing adjustment module performs a smoothing operation on the gain value of the remote audio digital signal based on a preset smoothing factor, and sends the smoothed gain value to the gain application module; The gain application module includes an input end and an output end, the input end is connected to the gain smoothing adjustment module, and the output end is connected to the digital-to-analog conversion unit; the gain application module calculates the automatic gain audio signal through the smoothed gain value, and sends the automatic gain audio signal to the digital-to-analog conversion unit. 5. The audio codec device with automatic gain control function according to claim 1, characterized in that the digital-to-analog conversion unit comprises: A data caching module, electrically connected to the automatic gain control unit and the data matching module, for performing delay matching and data caching operations on the automatic gain audio signal and sending it to the data matching module; A data matching module, electrically connected to the mixing module, for performing a format conversion and matching operation on the automatic gain audio signal to generate a digital-analog intermediate signal and send it to the mixing module; A mixing module, electrically connected to the digital-to-analog conversion submodule, for performing a mixing and superimposing operation on the digital-to-analog intermediate signal and the digital signal of other remote audio streams or the locally stored digital audio streams to generate a mixed signal and send the mixed signal to the digital-to-analog conversion submodule; The digital-to-analog conversion submodule is used to perform a digital-to-analog conversion operation on the mixed audio signal, and generate and output a remote audio analog signal. 6. An audio codec method with automatic gain control function, characterized in that it is applied to an audio codec, and the method comprises: Receive a near-end audio analog signal and perform analog-to-digital conversion, data matching, and data buffering operations to generate a corresponding near-end audio digital signal; Performing a noise monitoring operation on the near-end audio digital signal to generate environmental noise data; receiving control information and a far-end audio digital signal, and performing an automatic gain control operation on the far-end audio digital signal based on the environmental noise data and the control information to generate an automatic gain audio signal; A data buffering operation, a format conversion and matching operation, a mixing and superposition operation, and a digital-to-analog conversion operation are performed on the automatic gain audio signal to generate and output a remote audio analog signal that has undergone automatic gain control. 7. The audio codec method with automatic gain control function according to claim 6, wherein the noise monitoring operation comprises: Receive near-end audio digital signal and remove DC component from the signal; Using the Hanning window function to filter the near-end audio digital signal to remove the DC component; Converting the near-end audio digital signal processed by the Hanning window function from the time domain to the frequency domain to generate a near-end audio frequency domain signal; Calculating the short-time energy and the zero-crossing rate of the near-end audio frequency domain signal, performing voice activity detection based on the short-time energy and the zero-crossing rate to generate a voice activity detection result, and performing noise power spectral density analysis based on the voice activity detection result to generate a noise power spectral density; The total noise energy, spectrum flatness and spectrum abstract are calculated based on the noise power spectrum density to generate a comprehensive noise feature including the total noise energy, spectrum flatness and spectrum abstract, and the environmental noisiness data is calculated by the comprehensive noise feature; the calculation method of the environmental noisiness data includes: D(τ)=α·E_noise(τ)+β·F(τ)+γ·H(τ); Wherein, D(τ) represents the environmental noise level in the time window τ; F(τ) represents the spectrum flatness, that is, the ratio of the geometric mean to the arithmetic mean of the noise power spectrum density N(f,τ) in the time window τ, which is used to measure the uniformity of the spectrum; H(τ) represents the spectrum abstract under the time window τ, that is, the distribution complexity of the power spectrum density N(f,τ) of the frequency component f in the total power spectrum; α, β, and γ represent the first preset weighting parameter, the second preset weighting parameter, and the third preset weighting parameter, respectively. 8. The audio encoding and decoding method with automatic gain control function according to claim 6, characterized in that the automatic gain control operation comprises: Calculate the target volume value of the remote audio digital signal based on the control information and the environmental noise data; Calculating a gain value of the far-end audio digital signal according to a preset gain coefficient and a target volume value of the far-end audio digital signal; Performing a smoothing operation on the gain value of the remote audio digital signal based on a preset smoothing factor; The smoothed gain value is calculated to obtain an automatic gain audio signal. 9. An audio codec system with an automatic gain control function, characterized in that it comprises: a main processor and the audio codec device with an automatic gain control function as claimed in any one of claims 1 to 5.