CN111105786B - Multi-sampling-rate voice recognition method, device, system and storage medium - Google Patents

Abstract

The invention discloses a multi-sampling rate voice recognition method, a device, a system and a storage medium. Firstly, under the condition of not changing the audio sampling rate, extracting the characteristics of the audio with different sampling rates in a corresponding configuration mode according to different sampling rates, and training a neural network model by using the extracted audio. The neural network model is provided with a general speech recognition label and a sampling rate classification label, and a gradient inversion method is used for carrying out countermeasure training on the sampling rate classification label when the neural network model is trained, so that the trained multi-sampling rate speech recognition model can be self-adapted to audios with different sampling rates. Then, the multi-sampling rate speech recognition model obtained by the training of the method can be used for speech recognition, and the aim of uniformly processing the audio input with various sampling rates by using the same speech recognition model is fulfilled.

Description

A multi-sampling rate speech recognition method, device, system and storage medium

technical field

The present invention relates to the field of artificial intelligence voice interaction, in particular to a multi-sampling rate voice recognition method, device, system and storage medium.

Background technique

With the continuous development and progress of artificial intelligence and electronic communication technology, intelligent voice interaction technology is increasingly popularized and applied in many product fields, including intelligent customer service, call center, smart speaker and smart watch, etc.

However, although the same is speech recognition, in different application scenarios, the speech sampling rate is not the same. If it is necessary to process speech samples of different multi-sampling rates in one system, the following schemes are mostly adopted: 1) Unify the sampling rates of the audio through up/down sampling, so as to unify into a speech recognition system. This scheme changes the nature of the original audio, resulting in a decrease in the accuracy of speech recognition. 2) Deploy multiple speech recognition systems, and screen the results according to the confidence or confusion after outputting the results, and select the most suitable result. This solution has the problems of low resource utilization efficiency and high operation and maintenance costs.

SUMMARY OF THE INVENTION

In view of the above problems, the present inventor creatively provides a method, device, system and storage medium for speech recognition with multiple sampling rates.

According to the first aspect of the embodiments of the present invention, a method for training a multi-sampling rate speech recognition model, the method includes: acquiring at least two audio features with different sampling rates; using the audio features as input to train a neural network model, wherein, Audio features are annotated with speech recognition labels and sample rate classification labels.

According to an embodiment of the present invention, acquiring at least two audio features with different sampling rates includes: receiving audio inputs with at least two different sampling rates; classifying and setting configuration information for feature extraction according to the sampling rate to which the audio input belongs; using The configuration information performs feature extraction on the audio to obtain at least two audio features with different sampling rates.

According to an embodiment of the present invention, the training of the neural network model includes: normal training of the neural network model for the speech recognition label, and adversarial training of the neural network model for the sampling rate classification label.

According to an embodiment of the present invention, the adversarial training of the neural network model with respect to the sampling rate classification label includes: performing adversarial training on the neural network model with respect to the sampling rate classification label according to a cross-entropy training criterion.

According to an embodiment of the present invention, performing adversarial training includes: performing adversarial training by inverting gradients and then performing backpropagation.

According to a second aspect of the embodiments of the present invention, a method for multi-sampling rate speech recognition, the method includes: receiving audio features; inputting the audio features into a multi-sampling rate speech recognition model to obtain a speech recognition result, wherein the multi-sampling rate speech recognition The model is obtained by performing any one of the above-mentioned training methods for the multi-sample speech recognition model.

According to a third aspect of the embodiments of the present invention, a multi-sampling rate speech recognition model training device includes: an audio feature acquisition module for acquiring at least two audio features with different sampling rates; a neural network model training module for using It is used to train a neural network model with audio features as input, where audio features are labeled with speech recognition labels and sample rate classification labels.

According to an embodiment of the present invention, the audio feature acquisition module includes: an audio input receiving unit for receiving at least two audio inputs with different sampling rates; a feature extraction configuration unit for classifying and setting according to the sampling rate to which the audio input belongs Configuration information for feature extraction; an audio feature extraction unit, configured to perform feature extraction on audio using the configuration information to obtain at least two audio features with different sampling rates.

According to an embodiment of the present invention, the neural network model training module includes: a speech recognition training unit for performing normal training on the neural network model for speech recognition labels; Adversarial training of the network model.

According to an embodiment of the present invention, the sampling rate classification training unit is specifically configured to perform adversarial training on the neural network model with respect to the sampling rate classification labels according to the cross-entropy training criterion.

According to an embodiment of the present invention, the sampling rate classification training unit includes: a gradient inversion subunit, which is used for adversarial training in a manner of performing backpropagation after gradient inversion.

According to a fourth aspect of the embodiments of the present invention, there is provided a multi-sampling rate speech recognition device, the device comprising: an audio feature receiving module for receiving audio features; a speech recognition module for inputting the audio features into the multi-sampling rate speech recognition The model obtains a speech recognition result, wherein the multi-sampling rate speech recognition model is obtained by performing any one of the above-mentioned training methods for the multi-sampling rate speech recognition model.

According to a fifth aspect of the embodiments of the present invention, a multi-sampling rate speech recognition system is provided, the system includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute any of the above when run by the processor. a method.

According to a sixth aspect of the embodiments of the present invention, a computer storage medium is provided, the storage medium includes a set of computer-executable instructions, which are used to perform any one of the above methods when the instructions are executed.

Embodiments of the present invention provide a multi-sampling rate speech recognition method, device, system, and storage medium. Firstly, under the condition of not changing the audio sampling rate, feature extraction is performed on audios with different sampling rates by configuring correspondingly according to different sampling rates, and the neural network model is trained by using the extracted audio. In addition to the general speech recognition label, the neural network model also adds a sampling rate classification label, and when training the neural network model, the gradient inversion method is used to conduct adversarial training on the sampling rate classification label, so that the training results The multi-sampling rate speech recognition model is able to autonomously adapt to different sampling rates of audio. Afterwards, the multi-sampling rate speech recognition model trained by the above method can be used for speech recognition, so as to achieve the goal of uniformly processing audio input with various sampling rates by the same speech recognition model. In this way, the properties of the original audio can be preserved, and the training cost and maintenance cost of the speech recognition system can be greatly saved. In addition, audio input data of different sampling rates can be fused with each other, further increasing the diversity of data and multiplying the available data.

Description of drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown by way of example and not limitation, wherein:

In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

FIG. 1 is a schematic flowchart of an implementation of a training method for a multi-sampling rate speech recognition model according to an embodiment of the present invention;

2 is a schematic flow chart of a specific implementation of a training method using a multi-sampling rate speech recognition model according to the present invention;

FIG. 3 is a schematic flowchart of an implementation of a multi-sampling rate speech recognition method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a specific implementation flow diagram of an application of a multi-sampling rate speech recognition method according to the present invention;

5 is a schematic diagram of the composition and structure of a training device for a multi-sampling rate speech recognition model according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of the composition and structure of a multi-sampling rate speech recognition apparatus according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

FIG. 1 shows an implementation flow of a method for training a multi-sampling rate speech recognition model according to an embodiment of the present invention. Referring to FIG. 1, the method includes: operation 110, obtaining audio features of at least two different sampling rates; operation 120, using the audio features as input to train a neural network model, wherein the audio features are marked with speech recognition labels and sampling rate classifications Label.

In operation 110, the audio feature data here is training data for training the neural network model. It can be obtained by feature extraction of audio input, or it can be obtained from audio feature suppliers or raw material library. It should be noted that the training data is different from the data in practical applications, it is the data that has been labelled. The label here is the content to be predicted, and the value of the label is the expected value. By comparing the actual predicted value with the expected value, the model can be continuously revised to minimize the error between the actual predicted value and the expected value, thereby obtaining a model with a high accuracy rate that can be put into practical application. In order to enable the trained speech recognition system to recognize a variety of audio data with different sampling rates, at least two audio inputs with different sampling rates should be used when training the neural network model. Moreover, the training data used to train the neural network model should preferably use the same sampling rate as the audio data that needs to be recognized in practical applications, so that in subsequent applications, the accuracy of speech recognition for audio input with different sampling rates will be higher. high, the application effect is also better. Using the training method of the multi-sampling rate speech recognition model according to the embodiment of the present invention, there is no requirement for the data volume distribution of various sampling rates in the training data, and it can be data of different sampling rates in any proportion.

In operation 120, the neural network can be trained using the audio features obtained in operation 110. As mentioned earlier, the training data carries labels, and the value of the label is the expected value. By comparing the actual predicted value and the expected value obtained from the training, the model can be continuously revised to minimize the actual predicted value and the expected value until a model with high accuracy and practical application is obtained. This is the process of training the neural network. . Different from other neural network models for speech recognition, the embodiment of the present invention also adds a sampling rate classification label. This also means that this neural network model will not only predict the outcome of speech recognition, but also the sample rate classification to which the audio input belongs. Here, the use of this tag is to better integrate audio data of various sampling rates, so that the amount of data available to the model is larger, and the data from different sources also makes the data more diverse, and the prediction results are relatively better. precise.

According to an embodiment of the present invention, acquiring at least two audio features with different sampling rates includes: receiving audio inputs with at least two different sampling rates; classifying and setting configuration information for feature extraction according to the sampling rate to which the audio input belongs; using The configuration information performs feature extraction on the audio to obtain at least two audio features with different sampling rates.

The audio input here can be the original audio collected by the audio collection device, but preferably the audio input that has been processed by the voice signal. The audio processed by the speech signal is clearer, easier to recognize, and has better training effect. These audio inputs can also be obtained from data providers or audio databases. The sample rate class of the audio input is preset, and the sample rate class to which the audio input belongs is known. Here, just specify the sample rate classification to which the audio input belongs through some input parameters or configuration information. The definition of the sample rate classification is done during modeling, and the value specified here should be consistent with the value defined when the sample rate classification label is established. For example, assuming that the classification of the first sampling rate audio is defined as 1 during modeling, when training with the first sampling rate audio here, the specified sampling rate classification is 1. The process and parameters for extracting features from audio inputs with different audio sampling rates are slightly different. To this end, it is also necessary to set some audio processing parameters for the audio input of different audio sampling rates in order to obtain the unique and most representative features of the audio of the sampling rate. The setting values of these parameters are set for feature extraction. configuration information. For example, in the process of audio feature extraction, a maximum frequency needs to be set. For audio with a sampling rate of 8K, this parameter can be set to 4000, and for audio with a sampling rate of 16K, this parameter can be set. is 8000. The configuration used for a specific sampling rate audio can be preset according to the audio properties of the sampling rate audio, and the preset value can be obtained in this operation for configuration. The audio feature extraction may use any applicable method, and the embodiment of the present invention mainly adopts the current relatively common F-bank audio feature extraction method. The audio features extracted here include speech recognition features used for speech recognition, such as phonemes, words, etc., and also include sampling rate classification values.

According to an embodiment of the present invention, the training of the neural network model includes: normal training of the neural network model for the speech recognition label, and adversarial training of the neural network model for the sampling rate classification label.

Here, the normal training for speech recognition labels is no different from the implementation process of other speech recognition systems. Generally, the neural network-based connectionist temporal classification (CTC) criterion and the maximum mutual information (Maximum Mutual Information, MM) criterion are sampled. Or any other applicable criteria for training. The adversarial training of the neural network model for the sampling rate classification label is a prominent feature of the embodiment of the present invention. The adversarial training here, which can also be called interference training, aims to make the model unable to accurately distinguish the sampling rate classification, so as to make full use of the audio data of different sampling rates and make predictions on the basis of more diverse data. Such prediction results have automatic It is more adaptable and more accurate, so that the same model can be used to uniformly process audio input with multiple sampling rates and output speech recognition results.

According to an embodiment of the present invention, the adversarial training of the neural network model with respect to the sampling rate classification label includes: performing adversarial training on the neural network model with respect to the sampling rate classification label according to a cross-entropy training criterion.

Cross-entropy is mainly used in the field of disambiguation, by calculating the cross-entropy of prior information and posterior information, and using the cross-entropy to guide the identification and elimination of ambiguity. Especially suitable for computer adaptive implementation. Using the cross-entropy criterion to train the neural network model for the sampling rate classification labels, the multi-sampling rate speech recognition model obtained through this training can autonomously adapt to multiple sampling rate data.

According to an embodiment of the present invention, performing adversarial training includes: performing adversarial training by inverting gradients and then performing backpropagation.

The gradient here can be simply understood as the error between the actual predicted value and the expected value, which can be used as a basis for adjusting the parameters of the neural network model. The smaller the gradient, the more accurate the prediction is, and the more mature the neural network model is. When using the neural network model to predict the sampling rate classification, in order to make it ignore the difference of the sampling rate, the gradient inversion operation will be performed. That is, multiply the gradient by negative one to get its inverse, then pass back that inverse to train the model. This is the interference method used in adversarial training. In this way, the neural network model cannot accurately predict the sampling rate classification, and it will not be treated differently because of different sampling rates, thus greatly improving the contribution of data with different sampling rates. rate, and can uniformly process audio features of different sampling rates. This is also the key to adversarial training, which allows the speech recognition system to ignore the differences caused by the sampling rate and achieve a good recognition effect.

In the following, a specific flow of a method for training a multi-sampling rate speech recognition model according to the present invention will be described in detail with reference to FIG. 2 . In the application scenario shown in FIG. 2 , the system mainly receives audio data of three sampling rates, namely audio data of the first sampling rate, audio data of the second sampling rate and audio data of the third sampling rate. When training a multi-sample rate speech recognition model for this system, the following steps are mainly used:

Step 201, receiving audio input;

The audio input here is training data with training labels and at least two or more different sampling rates of training data.

Step 220, judging the sampling rate classification;

The sampling rate classification to which the audio input belongs is known information and can be obtained through a certain parameter or configuration information. Here, it is necessary to determine which classification the audio input belongs to, and determine the next step according to the classification. If the sampling rate of the audio input is the first sampling rate, proceed to step 230; if the sampling rate of the audio input is the second sampling rate, proceed to step 240; if the sampling rate of the audio input is the third sampling rate, then Proceed to step 250.

Step 230, first sampling rate audio feature extraction;

The received audio input is the first sample rate audio, and the configuration used for the first sample rate audio is used for feature extraction. The configuration used for the audio of the first sampling rate may be preset according to the audio properties of the audio of the first sampling rate. In this step, only a preset value is required for configuration. After configuration, audio feature extraction can be performed to obtain the required audio features and input them to the neural network model for training.

Step 240, second sampling rate audio feature extraction;

The received audio input is the second sample rate audio, and the configuration used for the second sample rate audio is used for feature extraction. The configuration used for the audio of the second sampling rate may be preset according to the audio properties of the audio of the second sampling rate. In this step, only a preset value is required for configuration. After configuration, audio feature extraction can be performed to obtain the required audio features and input them to the neural network model for training.

Step 250, the third sampling rate audio feature extraction;

The received audio input is the third sample rate audio, and the configuration used for the third sample rate audio is used for feature extraction. The configuration used for the audio of the third sampling rate may be preset according to the audio properties of the audio of the third sampling rate. In this step, only the preset value is required for configuration. After configuration, audio feature extraction can be performed to obtain the required audio features and input them to the neural network model for training.

Step 260, train the neural network model;

Here is mainly the process of using the audio features obtained in the above steps to train the neural network, and adjusting the parameters of the neural network model according to the prediction results, so that the prediction error value is continuously reduced. In this application of the embodiment of the present invention, the gradient descent method is mainly used.

Step 270, inverting the gradient for the sampling rate classification label and backpropagating it;

As before, in order to fuse audio data with different sampling rates, the gradient is inverted during training, and an adversarial learning method is used.

In step 280, gradient backpropagation is performed for the speech recognition label.

The gradient backpropagation here is normal backpropagation, and no inversion operation is performed.

It should be noted that the process of training the neural network model is an infinite loop process, and the mature standard of the model can be determined according to the actual application needs, and the multi-sampling rate speech recognition model that is considered to be relatively mature is applied to the actual production environment for speech recognition. identify.

Embodiments of the present invention further provide a speech recognition method after a relatively mature multi-sampling rate speech recognition model is obtained by executing the above multi-sampling rate speech recognition model training method. As shown in FIG. 3 , the method includes: operation 310, receiving audio features; operation 320, inputting the audio features into a multi-sampling rate speech recognition model to obtain a speech recognition result, wherein the multi-sampling rate speech recognition model is to execute the above-mentioned multi-sampling speech recognition model. It is obtained by training any one of the training methods of the recognition model.

In operation 310, the audio features here are extracted from unlabeled new data in an actual production environment. The new data mentioned here is the audio data classified by a certain sampling rate in the used training data, for example, the audio data of the first sampling rate.

In operation 320, the trained multi-sampling rate speech recognition model is used here, and a prediction result, that is, a speech recognition result, can be directly obtained according to the input.

It should be noted that after using the newly input unlabeled audio features in the actual production environment for prediction, the gradient back-propagation operation will not be performed, but the labels can be labeled according to the prediction results to become new training data for later use. Semi-supervised learning.

The following describes in detail a method for performing multi-sampling rate speech recognition using the multi-sampling rate speech recognition model trained in the steps shown in FIG. 2 using an application example of the present invention. As shown in Figure 4, multi-sample rate speech recognition can be performed using the following steps:

Step 410, receiving audio input;

The audio input here is the newly input audio data in the production environment without training labels.

Step 420, judging the sampling rate classification;

The sampling rate classification to which the audio input belongs is known information and can be obtained through a certain parameter or configuration information. Here, it is necessary to determine which classification the audio input belongs to, and determine the next step according to the classification. If the sampling rate of the audio input is the first sampling rate, proceed to step 430; if the sampling rate of the audio input is the second sampling rate, proceed to step 440; if the sampling rate of the audio input is the third sampling rate, then Proceed to step 450.

Step 430, first sampling rate audio feature extraction;

The received audio input is the first sample rate audio, and the configuration used for the first sample rate audio is used for feature extraction. The configuration used for the audio of the first sampling rate may be preset according to the audio properties of the audio of the first sampling rate. In this step, only a preset value is required for configuration. After the configuration is complete, audio feature extraction can be performed to obtain the required audio features and input them to the multi-sampling rate speech recognition model for speech recognition.

Step 440, second sampling rate audio feature extraction;

The received audio input is the second sample rate audio, and the configuration used for the second sample rate audio is used for feature extraction. The configuration used for the audio of the second sampling rate may be preset according to the audio properties of the audio of the second sampling rate. In this step, only a preset value is required for configuration. After the configuration is complete, audio feature extraction can be performed to obtain the required audio features and input the multi-sampling rate speech recognition model for speech recognition.

Step 450, the third sampling rate audio feature extraction;

The received audio input is the third sample rate audio, and the configuration used for the third sample rate audio is used for feature extraction. The configuration used for the audio of the third sampling rate may be preset according to the audio properties of the audio of the third sampling rate. In this step, only the preset value is required for configuration. After the configuration is complete, audio feature extraction can be performed to obtain the required audio features and input them to the multi-sampling rate speech recognition model for speech recognition.

Step 460, using a multi-sampling rate speech recognition model to perform speech recognition;

The prediction model used here is a multi-sample rate speech recognition model that has been trained and can be used for speech recognition in production environments.

Step 470, output the speech recognition result.

Here, according to the audio features extracted in step 430, step 440 or step 450, a prediction result can be obtained by using the sampling rate speech recognition model to obtain a prediction result, and the prediction result is the speech recognition result.

Further, the embodiment of the present invention also provides a training device for a multi-sampling rate speech recognition model. As shown in FIG. 5 , the device 50 includes: an audio feature acquisition module 501 for acquiring at least two audio features with different sampling rates; a neural network model training module 502 for training the neural network model by using the audio features as input , where the audio features are annotated with speech recognition labels and sample rate classification labels.

According to an embodiment of the present invention, the audio feature acquisition module 501 includes: an audio input receiving unit, configured to receive audio inputs with at least two different sampling rates; a feature extraction configuration unit, configured to classify and set the audio input according to the sampling rate to which the audio input belongs. The configuration information for feature extraction is determined; the audio feature extraction unit is configured to perform feature extraction on audio using the configuration information to obtain at least two audio features with different sampling rates.

According to an embodiment of the present invention, the neural network model training module 502 includes: a speech recognition training unit for normally training the neural network model for speech recognition labels; a sampling rate classification training unit for classifying label pairs according to the sampling rate Adversarial training of neural network models.

According to an embodiment of the present invention, the sampling rate classification training unit is specifically configured to perform adversarial training on the neural network model with respect to the sampling rate classification labels according to the cross-entropy training criterion.

According to an embodiment of the present invention, the sampling rate classification training unit includes: a gradient inversion subunit, which is used for adversarial training in a manner of performing backpropagation after gradient inversion.

In addition, an embodiment of the present invention also provides a multi-sampling rate speech recognition device. As shown in FIG. 6 , the device 60 includes: an audio feature receiving module 601 for receiving audio features; a speech recognition module 602 for converting the audio features Input to the multi-sampling rate speech recognition model to obtain a speech recognition result, wherein the multi-sampling rate speech recognition model is obtained by performing any one of the above-mentioned training methods for the multi-sampling rate speech recognition model.

According to a fifth aspect of the embodiments of the present invention, a multi-sampling rate speech recognition system is provided, the system includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute any of the above when run by the processor. a method.

According to a sixth aspect of the embodiments of the present invention, a computer storage medium is provided, the storage medium includes a set of computer-executable instructions, which are used to perform any one of the above methods when the instructions are executed.

It should be pointed out here that: the above description of the embodiment of the training apparatus for the multi-sampling rate speech recognition model, the description of the embodiment of the multi-sampling rate speech recognition apparatus, the above description of the embodiment of the multi-sampling rate speech recognition system and the above The description of the embodiment of the computer storage medium is similar to the description of the foregoing method embodiment, and has similar beneficial effects as the foregoing method embodiment, so it will not be repeated. For the description of the embodiment of the multi-sampling rate speech recognition model training apparatus, the description of the embodiment of the multi-sampling rate speech recognition apparatus, the description of the embodiment of the multi-sampling rate speech recognition system and the description of the embodiment of the computer storage medium of the present invention For technical details that have not yet been disclosed, please refer to the description of the foregoing method embodiments of the present invention to understand, and to save space, it will not be repeated.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated. to another device, or some features can be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.

The unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integration The unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, the execution includes: The steps of the above method embodiments; and the aforementioned storage medium includes: a removable storage medium, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.

Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium and include several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is caused to execute all or part of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: a removable storage medium, a ROM, a magnetic disk, or an optical disk and other mediums that can store program codes.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present invention, and should cover within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (9)

1. A method for training a multiple sampling rate speech recognition model, the method comprising:

receiving audio inputs of at least two different sampling rates; setting configuration information extracted by the characteristic according to the sampling rate classification of the audio input; performing feature extraction on the audio by using the configuration information to obtain audio features of at least two different sampling rates;

and training a neural network model by taking the audio features as input, wherein the audio features are marked with a voice recognition label and a sampling rate classification label.

2. The method of claim 1, wherein training the neural network model comprises:

and normally training the neural network model according to the voice recognition label, and carrying out countermeasure training on the neural network model according to the sampling rate classification label.

3. The method of claim 2, wherein the training the neural network model against the sample rate class labels comprises:

and carrying out countermeasure training on the neural network model according to a cross entropy training criterion and aiming at the sampling rate classification label.

4. The method of any one of claims 2 or 3, wherein the performing counter training comprises:

and performing countermeasure training by adopting a mode of reversing transmission after gradient reversal.

5. A method of multiple sample rate speech recognition, the method comprising:

receiving an audio feature;

inputting the audio features into a multi-sampling rate speech recognition model to obtain a speech recognition result, wherein the multi-sampling rate speech recognition model is obtained by performing the method of any one of claims 1 to 4.

6. An apparatus for training a multiple sampling rate speech recognition model, the apparatus comprising:

the audio characteristic acquisition module is used for receiving audio input of at least two different sampling rates; setting the configuration information extracted by the characteristic according to the sampling rate classification of the audio input; performing feature extraction on the audio by using the configuration information to obtain audio features of at least two different sampling rates;

and the neural network model training module is used for training the neural network model by taking the audio features as input, wherein the audio features are marked with voice recognition labels and sampling rate classification labels.

7. A multi-sample rate speech recognition apparatus, the apparatus comprising:

the audio characteristic receiving module is used for receiving the audio characteristics;

a speech recognition module, configured to input the audio features into a multi-sampling rate speech recognition model to obtain a speech recognition result, where the multi-sampling rate speech recognition model is trained by performing the method according to any one of claims 1 to 4.

8. A multi-sampling rate speech recognition system comprising a processor and a memory, wherein the memory has stored therein computer program instructions for execution by the processor to perform the method of any of claims 1 to 4;

or to perform the method of claim 5.

9. A storage medium having stored thereon program instructions for performing, when executed, the method of any one of claims 1 to 4;

or, performing the method of claim 5.

Images