CN111402908A - Voice processing method, device, electronic device and storage medium - Google Patents

Abstract

The application provides a voice processing method, a voice processing device, electronic equipment and a storage medium, wherein the method comprises the following steps: decoding original coded data obtained by voice sampling to obtain decoded audio data; if the sampling rate and/or the sampling bit number of the decoded audio data are/is determined to be larger than the set threshold value, the decoded audio data are subjected to down-sampling to obtain target audio data; and sending the target audio data to the server to acquire the text obtained by voice recognition of the target audio data from the server. Therefore, the audio data with high sampling rate and/or high sampling digit is subjected to down-sampling processing, and then the down-sampled target audio data is transmitted to the server side so as to obtain the text obtained by voice recognition from the server side, so that the data transmission quantity is reduced, and the data transmission rate is improved.

Description

Voice processing method, device, electronic device and storage medium

technical field

The present application relates to the technical field of speech processing, and in particular, to a speech processing method, apparatus, electronic device and storage medium.

Background technique

A speech-to-text (STT) system is a way of converting spoken words into text files for subsequent use. For STT, the current common solution is to directly transfer the collected audio files (such as MP3, M4A, AMR and other format audio) to the server, and the server will perform voice conversion processing on the audio data, and return the converted text.

In order to ensure the sound quality, the sampling rate, the number of sampling bits and the bit rate will be greatly increased during the recording process, which will lead to an increase in the volume of the transmitted audio file, increase the burden of the audio file transmission process to the server, and reduce the transmission efficiency. .

SUMMARY OF THE INVENTION

The present application aims to solve one of the technical problems in the related art at least to a certain extent.

The embodiment of the first aspect of the present application proposes a speech processing method, including:

Decode the original encoded data obtained by the voice sampling to obtain decoded audio data;

If it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, down-sampling the decoded audio data to obtain target audio data;

The target audio data is sent to the server to obtain the text obtained by speech recognition of the target audio data from the server.

As a first possible implementation manner of the embodiment of the present application, the down-sampling of the decoded audio data includes:

The decoded audio data is down-sampled using a synchronous sampling rate conversion SSRC algorithm.

As a second possible implementation manner of the embodiment of the present application, the synchronous sampling rate conversion SSRC algorithm is used to downsample the decoded audio data, including:

A finite-length unit impulse response FIR filter is used to filter the set length sequence in the decoded audio data;

The target sequence of the set length is added to the set length sequence obtained after filtering to obtain the input sequence of the Fourier transform; wherein, each element in the target sequence takes a value of zero;

performing fast Fourier transform on the input sequence to obtain a frequency domain sequence;

After filtering the frequency domain sequence, perform inverse fast Fourier transform to obtain the time domain sequence;

The target audio data is obtained by resampling the time-domain sequence according to the set down-sampling rate.

As a third possible implementation manner of the embodiment of the present application, before the sending the target audio data to the server side, the method further includes:

If the target audio data includes binaural data, one channel data in the binaural data is eliminated.

As a fourth possible implementation manner of the embodiment of the present application, the removing one channel data from the binaural data includes:

Determine the data length occupied by single channel data in the target audio data;

For each interval of the data length of the target audio data, a piece of data that conforms to the data length is eliminated.

As a fifth possible implementation manner of the embodiment of the present application, before the sending the target audio data to the server side, the method further includes:

According to the target audio data, voice endpoint detection is performed to extract the voiced part and the unvoiced part from the target audio data, and remove the mute part;

Wherein, the energy value of the voiced portion is greater than the first energy threshold;

The energy value of the unvoiced portion is greater than the second energy threshold;

The first energy threshold is greater than the second energy threshold.

As a sixth possible implementation manner of the embodiment of the present application, before the sending the target audio data to the server side, the method further includes:

If the bit rate of the target audio data is lower than the set bit rate, the linear prediction encoding method is used for compression encoding;

If the bit rate of the target audio data is not lower than the set bit rate, a transform coding method is used to perform compression coding.

In the speech processing method of the embodiment of the present application, the original encoded data obtained by the speech sampling is decoded to obtain decoded audio data; if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data is greater than the set threshold, the decoded audio data is down-sampled , obtain the target audio data; send the target audio data to the server, so as to obtain the text obtained from the speech recognition of the target audio data from the server. Therefore, by performing down-sampling processing on the audio data with high sampling rate and/or high sampling bits, and then transmitting the down-sampled target audio data to the server, so as to obtain the text obtained by speech recognition from the server, thereby reducing The data transmission volume is increased, and the data transmission rate is improved.

The embodiment of the second aspect of the present application provides a voice processing apparatus, including:

The decoding module is used to decode the original encoded data obtained by the voice sampling to obtain the decoded audio data;

A downsampling module, configured to downsample the decoded audio data to obtain target audio data if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold;

The sending module is configured to send the target audio data to the server, so as to obtain the text obtained by speech recognition of the target audio data from the server.

The speech processing apparatus of the embodiment of the present application decodes the original encoded data obtained by speech sampling to obtain decoded audio data; if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, the decoded audio data is down-sampled , obtain the target audio data; send the target audio data to the server, so as to obtain the text obtained from the speech recognition of the target audio data from the server. Therefore, by performing down-sampling processing on the audio data with high sampling rate and/or high sampling bits, and then transmitting the down-sampled target audio data to the server, so as to obtain the text obtained by speech recognition from the server, thereby reducing The data transmission volume is increased, and the data transmission rate is improved.

An embodiment of a third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the implementation of the first aspect is implemented. The speech processing method described in the example.

Embodiments of the fourth aspect of the present application provide a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the voice processing method described in the first aspect of the embodiment.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic flowchart of a first voice processing method provided by an embodiment of the present application;

2 is a schematic flowchart of a second speech processing method provided by an embodiment of the present application;

3 is a schematic flowchart of a third voice processing method provided by an embodiment of the present application;

4 is a schematic flowchart of a fourth voice processing method provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a speech processing apparatus according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

In the related art, STT detection has a very high recognition rate on mono audio with a sampling rate of 16 kHz and a sampling number of 16 bits. Therefore, in the STT process, the sampling rate exceeds 16 kHz and the sampling number exceeds 16 bits. Audio has not greatly improved the recognition rate of STT, but will increase the resource consumption during audio transmission due to the size of the audio file.

In view of the above-mentioned technical problems, the present application proposes a voice processing method, which obtains decoded audio data by decoding the original encoded data obtained by voice sampling. Then, the decoded audio data is down-sampled to obtain target audio data, and the target audio data is sent to the server to obtain the text obtained by speech recognition of the target audio data from the server. The method performs down-sampling processing on the audio data with high sampling rate and/or high sampling bits, and then transmits the down-sampled target audio data to the server, so as to obtain the text obtained by speech recognition from the server, thereby reducing the number of The amount of data transfer increases the data transfer rate.

The speech processing method, apparatus, electronic device, and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a first voice processing method provided by an embodiment of the present application.

The embodiment of the present application is exemplified in that the voice processing method is configured in a voice processing apparatus, and the voice processing apparatus can be applied to any electronic device, so that the electronic device can perform a voice processing function.

The electronic device may be a personal computer (Personal Computer, PC for short), a cloud device, a mobile device, etc. The mobile device may be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, a vehicle-mounted device, etc. with various operating systems. hardware equipment.

As shown in Figure 1, the speech processing method includes the following steps:

Step 101: Decode the original encoded data obtained by the speech sampling to obtain decoded audio data.

Wherein, the original encoded data obtained by voice sampling refers to the original encoded data obtained by collecting a voice signal from a hardware device and performing analog-to-digital conversion on the voice signal.

There are two main categories of sources for communication systems: analog signals and digital signals. For example: the voice signal output by the microphone belongs to the analog signal; while the text and computer data belong to the digital signal. Compared with analog signals, digital signals have the advantages of strong anti-interference ability and no noise accumulation. Therefore, if the input is an analog signal, the input analog signal needs to be digitized in the source coding part of the digital communication system.

After the voice signal is collected from the hardware device, three steps are required to digitize the voice signal: sampling, quantization and encoding. Sampling refers to replacing the original continuous signal in time with a sequence of signal samples at regular intervals, that is, discretizing the analog signal in time. Quantization is to approximate the original continuously changing amplitude value with a finite amplitude value, and change the continuous amplitude of the analog signal into a finite number of discrete values with a certain interval. Coding is to encode the quantized signal into a code group composed of multi-bit binary codes to represent the sampled value, and complete the conversion of analog signal to digital signal.

It should be noted that, after encoding the collected raw data, audio in a data format is obtained and stored in an audio file, wherein the format of the audio file includes MP3, M4A, AMR, WAV, and so on.

As a possible implementation manner, pulse code modulation (Pulse Code Modulation, PCM for short) may be used to encode the collected raw data. Among them, the main process of coding is to sample the analog signals such as voice and image at regular intervals to make them discretized, and at the same time, the sampled value is rounded and quantized according to the hierarchical unit, and the sampled value is represented by a set of binary codes. The amplitude of the sampled pulse.

In the embodiment of the present application, after obtaining the original encoded data obtained by the speech sampling, the original encoded data obtained by the speech sampling needs to be decoded to obtain decoded audio data.

As a possible implementation, the audio file byte stream that stores the original encoded data can be filled into the input data buffer of MediaCodec. MediaCodec adopts the consumer mode to asynchronously read the byte stream from the data buffer. The decoding process is performed, and finally the decoded audio data is obtained.

Step 102, if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, down-sample the decoded audio data to obtain target audio data.

Among them, the sampling rate of the audio data refers to the number of times of sampling the sound signal in one second. The higher the sampling rate, the more realistic and natural the restoration of the sound. Downsampling, also known as downsampling, is a multi-rate digital signal processing technique or the process of reducing the sampling rate of a signal, usually used to reduce the data transmission rate or data size.

In the embodiment of the present application, when the sampling rate and/or the sampling number of bits of the decoded audio data are high, the volume of the audio file to be transmitted is large, which increases the transmission burden, thereby making the transmission efficiency low. Therefore, in the present application, after decoding the original encoded data obtained by voice sampling and obtaining the decoded audio data, if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data is greater than the set threshold, the decoded audio data is subjected to down-sampling processing , to get the target audio data. Therefore, by performing down-sampling processing on the decoded audio data, the size of the audio data is reduced, thereby helping to increase the data transmission rate.

For example, it is determined that the sampling rate of the decoded audio data is greater than 16 kHz, or when the number of sampling bits is greater than 16 bits, in order to reduce the transmission data of the audio data, the decoded audio data is down-sampled to obtain the target audio data.

As a possible implementation, it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, and a Synchronous Sample Rate Converter (SSRC) algorithm may be used to downsample the decoded audio data. , get the target audio data.

It should be noted that when the SSRC algorithm is used to downsample the decoded audio data, the sampling rate before and after the conversion must be integers, and the SSRC algorithm does not support conversion between arbitrary frequencies.

In the embodiment of the present application, when the SSRC algorithm is used to downsample the decoded audio data, first, a finite-length unit impulse response (Finite Impulse Response, FIR) filter is used to filter the set length sequence in the decoded audio data; The target sequence of set length is added to the target sequence of set length, and the input sequence of the Fourier transform is obtained; wherein, each element in the target sequence takes the value of zero; the fast Fourier transform is performed on the input sequence to obtain the frequency domain sequence; After the frequency domain sequence is filtered, inverse fast Fourier transform is performed to obtain the time domain sequence; the time domain sequence is resampled according to the set downsampling rate to obtain the target audio data.

As an example, the SSRC algorithm uses n-point fast Fourier transform to downsample the decoded audio data. First, pass the first n/2 encoded input samples through y(n)=a0*x(n)+a1*x(n-l)+...+a7*x(n-8), and perform a 9th-order FIR Digital filtering to obtain the input of fast Fourier transform (fast Fourier transform, referred to as FFT); add n/2 0s after the n/2 outputs obtained by filtering, and then perform fast Fourier transform on n data, in In the frequency domain, the frequency domain data is windowed and filtered in the complex domain, and then the inverse fast Fourier transform is performed. The inverse transform obtains the time domain data again, and then performs deletion and envelope processing according to the length requirements of the resampling output data, and finally outputs the The resampled target audio data.

Step 103: Send the target audio data to the server, so as to obtain the text obtained by speech recognition of the target audio data from the server.

In the embodiment of the present application, after down-sampling the decoded audio data to obtain the target audio data, the target audio data is sent to the server. Further, the server side performs speech recognition on the received target audio data, and after obtaining the text corresponding to the target audio data, the text corresponding to the target audio data can be obtained from the server side.

In the speech processing method of the embodiment of the present application, decoded audio data is obtained by decoding the original encoded data obtained by sampling the speech; if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, then the decoded audio data is reduced Sampling to obtain the target audio data; sending the target audio data to the server to obtain the text obtained from the speech recognition of the target audio data from the server. Therefore, by performing down-sampling processing on the audio data with high sampling rate and/or high sampling bits, and then transmitting the down-sampled target audio data to the server, so as to obtain the text obtained by speech recognition from the server, thereby reducing The data transmission volume is increased, and the data transmission rate is improved.

On the basis of the above-mentioned embodiment, after the decoded audio data is down-sampled in step 102 to obtain the target audio data, if the target audio data includes binaural data, it is also necessary to remove one channel of the binaural data to obtain the target audio data. The volume of the target audio data to be transmitted is reduced, thereby helping to improve the data transmission rate. For a specific implementation process, refer to FIG. 2 , which is a schematic flowchart of a second speech processing method provided by an embodiment of the present application.

As shown in Figure 2, the voice processing method may further include the following steps:

Step 201: Determine the data length occupied by single channel data in the target audio data.

The data length refers to the bytes occupied by the data.

In this embodiment of the present application, when down-sampling is performed on decoded audio data whose sampling rate and/or sampling number is greater than a set threshold, the obtained target audio data may include binaural data. Since the space occupied by the binaural data is twice that of the monophonic data, in order to reduce the amount of data transmission, one channel of data in the binaural data can be eliminated.

Specifically, after obtaining the target audio data, the data length occupied by the single channel data in the target audio data can be determined. For example, the data length occupied by one channel data may be 2 bytes or 1 byte.

Step 202: For each data length of the target audio data, remove a piece of data that conforms to the data length.

In the embodiment of the present application, after determining the data length occupied by the single channel data in the target audio data, a piece of data that conforms to the data length may be eliminated at every interval of the data length.

For example, assuming that the data length occupied by the single-channel data in the target audio data is 2 bytes, the 2-byte data can be removed every 2 bytes, so that a single-channel target audio data can be obtained separately.

For example, to remove the left channel data in the target audio data, the following formula can be used:

f(n)=f(0)+f(1)+f(4)+f(5)+...+f(2n-1)+f(2n);

The following formula can be used to remove the left channel data in the target audio data:

f(n)=f(2)+f(3)+f(6)+f(7)+...+f(2n-3)+f(2n-2).

In the speech processing method of the embodiment of the present application, if the target audio data includes two-channel data, when one channel of data in the two-channel data is excluded, the data length occupied by the single-channel data in the target audio data can be determined to determine the data length of the target audio data. For each data length of the audio data, remove a piece of data that matches the data length. Therefore, after the target audio data of two channels is eliminated, the target audio data of single channel is obtained, thereby reducing the volume of the target audio data to be transmitted, which is beneficial to improve the transmission rate of data.

In a possible case, after down-sampling the decoded audio data to obtain the target audio data, before sending the target audio data to the server, voice endpoint detection may also be performed on the target audio data to extract the target audio data from the target audio data. Voiced parts and unvoiced parts, and remove the silent parts. Thus, it is realized to identify and eliminate long silent periods from the target audio data, so as to achieve the effect of saving voice channel resources without reducing service quality. The above process will be described in detail below with reference to FIG. 3 , which is a schematic flowchart of a third speech processing method provided by an embodiment of the present application.

As shown in Figure 3, the voice processing method may further include the following steps:

Step 301: Decode the original encoded data obtained by the speech sampling to obtain decoded audio data.

Step 302, if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, down-sample the decoded audio data to obtain target audio data.

In this embodiment of the present application, for the implementation process of step 301 and step 302, reference may be made to the implementation process of step 101 and step 102 in the foregoing embodiment, and details are not described herein again.

Step 303: Perform voice endpoint detection according to the target audio data, so as to extract the voiced part and the unvoiced part from the target audio data, and remove the silent part.

Among them, Voice Activity Detection (VAD for short) is used to identify the appearance and disappearance of voice in the target audio data.

In the present application, the target audio data obtained by down-sampling the decoded audio data may include a voiced part, an unvoiced part and a silent part, in order to reduce the volume of the audio file when the target audio data is transmitted, the target audio data can be detected by voice endpoint detection, To extract the voiced part and the unvoiced part from the target audio data, and remove the silent part.

In this application, when the voice endpoint detection is performed on the target audio data, the audio data can also be divided into frames first, and then features are extracted from each frame of data, and a set of data frames in a known voice and silent signal area Train a classifier on the above to classify the unknown framed data to determine whether it belongs to the voiced part, the unvoiced part or the silent part.

As a possible situation, when the feature extraction is performed on the target audio data, the energy of each frame of audio data can be extracted. It should be explained that the energy value of the voiced part in the target audio data is greater than the first energy threshold, and the energy value of the unvoiced part is greater than the second energy threshold, wherein the first energy threshold is greater than the second energy threshold. Therefore, the voiced part and the unvoiced part in the target audio data can be extracted by setting the energy threshold.

As a possible implementation, when the voice endpoint detection is performed on the target audio data, the audio data can be processed into frames, the energy value of each frame of audio data can be extracted, and the energy value of each frame of audio data can be compared with the first energy value. thresholds are compared, and if the energy value of the audio data is greater than the first energy threshold, it is determined that the frame of audio data is a voiced part. Thus, the voiced portion can be extracted from the target audio data.

Further, the energy value of each frame of audio data of the target audio data after extracting the voiced portion is compared with the second energy threshold, if the energy value of the audio data is greater than the second energy threshold, then determine that this frame of audio data is the unvoiced portion. .

In the embodiment of the present application, the energy of the unvoiced and silent parts in the target audio data can be distinguished by the short-term zero-crossing rate threshold, and after the unvoiced parts are distinguished, the unvoiced parts can be removed, so that the silent audio segment can be discarded from the audio file itself. Reduce the volume of voice files that need to be transmitted during TTS.

Step 304: Send the target audio data to the server to obtain the text obtained by speech recognition of the target audio data from the server.

It should be noted that the target audio data sent to the server in step 304 is the data after removing the mute portion.

In this embodiment of the present application, for the implementation process of step 304, reference may be made to the implementation process of step 103 in the foregoing embodiment, and details are not described herein again.

The voice processing method of the embodiment of the present application decodes the original encoded data obtained by voice sampling to obtain decoded audio data, and if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data is greater than the set threshold, the decoded audio data is down-sampled , obtain the target audio data, and perform voice endpoint detection according to the target audio data to extract the voiced part and the unvoiced part from the target audio data, and remove the mute part, and send the target audio data to the server to obtain the target audio from the server. Text from audio data speech recognition. Therefore, the mute portion is removed before the target audio data is sent to the server, thereby reducing the amount of audio data transmission, which is beneficial to improve the data transmission rate.

In a possible case, after down-sampling the decoded audio data to obtain the target audio data, before sending the target audio data to the server, the bit rate of the target audio data can also be compared with the set bit rate to determine The target audio data is compressed and encoded using a corresponding encoding method, and then the compressed and encoded target audio data is sent to the server. The above process will be described in detail below with reference to FIG. 4 , which is a schematic flowchart of a fourth voice processing method provided by an embodiment of the present application.

As shown in FIG. 4, before the above step 103, the following steps may also be included:

Step 401: Decode the original encoded data obtained by the speech sampling to obtain decoded audio data.

Step 402, if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, down-sample the decoded audio data to obtain target audio data.

In this embodiment of the present application, for the implementation process of step 401 and step 402, reference may be made to the implementation process of step 101 and step 102 in the foregoing embodiment, and details are not described herein again.

Step 403: Compare the bit rate of the target audio data with the set bit rate.

The bit rate of the audio data refers to the amount of binary data per unit time after converting the analog sound signal into a digital sound signal, and is an indicator for indirectly measuring the audio quality. Among them, the higher the bit rate, the better the audio quality, but the larger the volume of the encoded audio file; the lower the bit rate, the worse the audio quality, but the smaller the volume of the encoded audio file.

It is understandable that if the target audio data is directly transmitted to the server without compression and encoding, it will occupy a huge amount of bandwidth, and the huge amount of data will put pressure on the transmission and storage of audio data. Therefore, after obtaining the target audio data, , the target audio data can be compressed and encoded to reduce the volume of the audio file during data transmission, thereby reducing the amount of data transmission.

For example, after down-sampling and compressing and encoding the decoded audio data, after the audio data is obtained, sending the audio data to the server reduces the data transmission amount by about 80% compared with directly sending the decoded audio data.

In the embodiment of the present application, after the target audio data is obtained, the bit rate of the target audio data is compared with the set bit rate to determine a manner of compressing and encoding the target audio data.

As a possible implementation manner, after obtaining the target audio data, the target audio data may be compressed and encoded by using Opus coding.

Among them, Opus coding is combined with two coding algorithms Silk and Celt, which has low algorithm delay and extremely high compression ratio. Both the coding end and the decoding end use the filters provided by Broadcom. Setting the filter can preserve the low-frequency part of the audio signal, attenuate the high-frequency part, and improve the coding efficiency.

Opus can seamlessly adjust high and low bitrates, and inside the encoder it uses linear predictive coding at lower bitrates and transform coding at high bitrates. Therefore, in this embodiment of the present application, the bit rate of the target audio data is compared with the set bit rate, so as to determine which encoding method is used for compression encoding.

Step 404, if the bit rate of the target audio data is lower than the set bit rate, a linear prediction encoding method is used to perform compression encoding.

In a possible situation, the bit rate of the target audio data is lower than the set bit rate, and the target audio data is compressed and encoded by using a linear predictive coding method.

Among them, linear predictive coding is mainly used in audio signal processing and speech processing to express the spectral envelope of digital speech signals in a compressed form according to the information of the linear prediction model.

Step 405 , if the bit rate of the target audio data is not lower than the set bit rate, a transform coding method is used to perform compression coding.

Among them, transform coding does not directly encode the spatial domain image signal, but firstly maps and transforms the spatial domain image signal to another orthogonal vector space (transform domain or frequency domain), generates a batch of transform coefficients, and then encodes these transform coefficients. deal with. Transform coding is an indirect coding method. The key problem is that when describing in the time domain or the space domain, the correlation between data is large and the data redundancy is large. The margin is reduced, the parameters are independent, and the amount of data is small, so that after quantization, the encoding can obtain a larger compression ratio.

Step 406: Send the compressed and encoded target audio data to the server, so as to obtain the text obtained by speech recognition of the target audio data from the server.

In the speech processing method of the embodiment of the present application, decoded audio data is obtained by decoding the original encoded data obtained by sampling the speech. Sampling to obtain the target audio data. If the bit rate of the target audio data is lower than the set bit rate, the linear prediction coding method is used for compression coding; if the bit rate of the target audio data is not lower than the set bit rate, the transformation coding is used. Compression coding is performed in the method, and the compressed and coded target audio data is sent to the server, so as to obtain the text obtained by speech recognition of the target audio data from the server. Therefore, the audio data is compressed and encoded before the audio data is transmitted, so as to realize the audio data transmission after the encoding, thereby reducing the data transmission volume and increasing the data transmission rate.

It should be noted that, on the basis of the above-mentioned embodiment, the original encoded data obtained by voice sampling is decoded to obtain decoded audio data. The audio data is down-sampled, and after the target audio data is obtained, after determining that the target audio data includes binaural data, one channel data in the binaural data is eliminated, and then voice endpoint detection is performed on the eliminated monophonic target audio data, to remove the silent part. Further, the audio data after the mute part is removed is compressed and encoded, and the compressed and encoded audio data is sent to the server. Therefore, the data transmission amount in the data transmission process is reduced, and the data transmission efficiency is improved.

In order to realize the above embodiments, the present application also proposes a voice processing apparatus.

FIG. 5 is a schematic structural diagram of a speech processing apparatus according to an embodiment of the present application.

As shown in FIG. 5 , the speech processing apparatus 500 may include: a decoding module 510 , a downsampling module 520 and a sending module 530 .

Wherein, the decoding module 510 is used for decoding the original encoded data obtained by voice sampling to obtain decoded audio data.

The down-sampling module 520 is configured to down-sample the decoded audio data to obtain target audio data if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold.

The sending module 530 is configured to send the target audio data to the server, so as to obtain the text obtained by speech recognition of the target audio data from the server.

As a possible situation, the downsampling module 520 can also be used for:

The synchronous sampling rate conversion SSRC algorithm is used to downsample the decoded audio data.

As another possible situation, the downsampling module 520 can also be used for:

The finite-length unit impulse response FIR filter is used to filter the set length sequence in the decoded audio data;

The set length sequence obtained after filtering is added to the set length target sequence, and the input sequence of the Fourier transform is obtained; wherein, each element in the target sequence takes a value of zero;

Perform fast Fourier transform on the input sequence to obtain the frequency domain sequence;

After filtering the frequency domain sequence, perform inverse fast Fourier transform to obtain the time domain sequence;

For the time domain sequence, resampling according to the set downsampling rate to obtain the target audio data.

As another possible situation, the voice processing apparatus 500 may further include:

The removing module is used for removing one channel data in the two channel data if the target audio data includes two channel data.

As another possible case, the culling module can also be used to:

Determine the data length occupied by the single channel data in the target audio data;

For each interval data length of the target audio data, a section of data that conforms to the data length is eliminated.

As another possible situation, the voice processing apparatus 500 may further include:

The detection module is used to perform voice endpoint detection according to the target audio data, so as to extract the voiced part and the unvoiced part from the target audio data, and remove the mute part;

Wherein, the energy value of the voiced part is greater than the first energy threshold;

The energy value of the unvoiced part is greater than the second energy threshold;

The first energy threshold is greater than the second energy threshold.

As another possible situation, the voice processing apparatus 500 may further include:

Compression coding module, used for compression coding by linear prediction coding if the bit rate of the target audio data is lower than the set bit rate; if the bit rate of the target audio data is not lower than the set bit rate, the transformation coding is used Compression encoding is performed.

It should be noted that, the foregoing explanations on the embodiment of the speech processing method are also applicable to the speech processing apparatus of this embodiment, and are not repeated here.

The speech processing apparatus of the embodiment of the present application obtains decoded audio data by decoding the original encoded data obtained by sampling the speech; if it is determined that the sampling rate and/or the number of sampling bits of the decoded audio data are greater than the set threshold, the decoded audio data is reduced Sampling to obtain the target audio data; sending the target audio data to the server to obtain the text obtained from the speech recognition of the target audio data from the server. Therefore, by performing down-sampling processing on the audio data with high sampling rate and/or high sampling bits, and then transmitting the down-sampled target audio data to the server, so as to obtain the text obtained by speech recognition from the server, thereby reducing The data transmission volume is increased, and the data transmission rate is improved.

In order to implement the above embodiments, the present application further proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the above-mentioned implementation is realized. Example speech processing method.

In order to implement the above-mentioned embodiments, the present application further provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the speech processing method according to the above-mentioned embodiments.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in conjunction with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (10)

1. A method of speech processing, the method comprising:

decoding original coded data obtained by voice sampling to obtain decoded audio data;

if the sampling rate and/or the sampling bit number of the decoded audio data are/is determined to be larger than a set threshold value, the decoded audio data are subjected to down-sampling to obtain target audio data;

and sending the target audio data to a server side so as to obtain a text obtained by voice recognition of the target audio data from the server side.

2. The speech processing method of claim 1, wherein the downsampling the decoded audio data comprises:

the decoded audio data is down-sampled using a Synchronous Sample Rate Conversion (SSRC) algorithm.

3. The speech processing method of claim 2 wherein the down-sampling the decoded audio data using a Synchronous Sample Rate Conversion (SSRC) algorithm comprises:

filtering the set length sequence in the decoded audio data by adopting a finite length single-bit impulse response FIR filter;

adding the target sequence with the set length to the sequence with the set length obtained after filtering to obtain an input sequence of Fourier transform; wherein, the value of each element in the target sequence is zero;

performing fast Fourier transform on the input sequence to obtain a frequency domain sequence;

filtering the frequency domain sequence, and performing inverse fast Fourier transform to obtain a time domain sequence;

and resampling the time domain sequence according to a set down-sampling rate to obtain the target audio data.

4. The speech processing method according to claim 1, wherein before sending the target audio data to the server, the method further comprises:

and if the target audio data comprises the double-channel data, one channel data in the double-channel data is removed.

5. The speech processing method of claim 4, wherein said removing one of the two-channel data comprises:

determining the data length occupied by single channel data in the target audio data;

and eliminating a section of data which accords with the data length for the target audio data at intervals of the data length.

6. The speech processing method according to claim 1, wherein before sending the target audio data to the server, the method further comprises:

performing voice endpoint detection according to the target audio data to extract a voiced part and an unvoiced part from the target audio data and remove a mute part;

wherein the energy value of the voiced parts is greater than a first energy threshold;

the energy value of the unvoiced part is greater than a second energy threshold;

the first energy threshold is greater than the second energy threshold.

7. The speech processing method according to any one of claims 1 to 6, wherein before sending the target audio data to the server, the method further comprises:

if the bit rate of the target audio data is lower than the set bit rate, performing compression coding by adopting a linear prediction coding mode;

and if the bit rate of the target audio data is not lower than the set bit rate, performing compression coding by adopting a transform coding mode.

8. A speech processing apparatus, comprising:

the decoding module is used for decoding the original coded data obtained by voice sampling to obtain decoded audio data;

the down-sampling module is used for down-sampling the decoded audio data to obtain target audio data if the sampling rate and/or the sampling bit number of the decoded audio data are/is determined to be greater than a set threshold value;

and the sending module is used for sending the target audio data to a server so as to obtain a text obtained by voice recognition of the target audio data from the server.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the speech processing method according to any of claims 1-7 when executing the program.

10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the speech processing method according to any one of claims 1 to 7.

Images