CN115206296A - Method and device for speech recognition - Google Patents

Abstract

Embodiments of the present disclosure disclose methods and apparatuses for speech recognition. The specific implementation of the method includes: obtaining the keywords to be identified; searching for the audio containing the keywords from the original scene training set to form the first training set, and obtaining the audio containing the keywords but not the original scene training set to form the first test set Carry out the first round of incremental training to the first speech recognition model of the original scene based on the first training set, obtain the second speech recognition model to identify the first test set, and add the audio that the keywords in the first test set are correctly identified. Go to the first training set, obtain a second training set and a second test set, and perform a second round of incremental training to obtain a third speech recognition model; input the second test set into the third speech recognition model to obtain an initial recognition result. This implementation can not only greatly reduce the training time and the required amount of data, but also solve the overfitting problem of incremental learning and the limitation of hot word decoding.

Description

Method and device for speech recognition

technical field

The embodiments of the present disclosure relate to the field of computer technology, and in particular, to methods and apparatuses for speech recognition.

Background technique

End-to-end deep neural networks have become a popular framework in the field of speech recognition, which can simplify the model building and training process compared to traditional speech recognition frameworks. In practical applications, many occasions require that the existing speech recognition model can not only recognize the speech input in the new scene, but also maintain the recognition accuracy of the original scene. For example, the speech recognition model trained on the original data set needs to strengthen the ability to recognize new keywords. Alternatively, for a cold start of a new speech recognition scenario, it is necessary to adapt the model to a new domain based on a small new dataset while inheriting the recognition capabilities of the old speech recognition model. Since new keywords or new scenes are usually not in past training datasets, the recognition performance will be very suboptimal when using old models directly on new tasks. To solve this problem, a feasible approach is to retrain the speech recognition model by mixing old and new scene datasets. However, this method may suffer from imbalanced training data, as new datasets are usually much smaller than older datasets. At the same time, due to data security and privacy concerns, past datasets may not be available for training. Another method is to use new scene data for transfer learning, although this method can reduce the time cost, but it will lead to the problem of overfitting the speech recognition model. Using hot word decoding is also a feasible way, but using hot word decoding can only operate the path when the keyword appears in the decoding path, so as to achieve keyword recall. When the keyword does not exist in the decoding path or When the keyword probability is low, the hot word decoding method cannot achieve keyword recall.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure propose methods and apparatuses for speech recognition.

In a first aspect, an embodiment of the present disclosure provides a method for speech recognition, including: obtaining a keyword to be recognized; searching for audio containing the keyword from an original scene training set to form a first training set, and obtaining a first training set containing the keyword. The audio of the keyword and the non-original scene training set constitutes the first test set; the first round of incremental training is performed on the first speech recognition model of the original scene based on the first training set to obtain the second speech recognition model; The second speech recognition model recognizes the first test set, and adds the audio of the correctly identified keywords in the first test set to the first training set to obtain a second training set and a second test set; Perform a second round of incremental training on the second speech recognition model based on the second training set to obtain a third speech recognition model; input the second test set into the third speech recognition model to obtain an initial recognition result .

In some embodiments, the method further includes: adjusting the initial recognition result through a hot word decoder to calculate the recall rate of the keyword.

In some embodiments, adding the audio with correctly identified keywords in the first test set to the first training set to obtain a second training set and a second test set includes: using a hot word decoder to The second speech recognition model adjusts the recognition result obtained by recognizing the first test set; determines the audio for which the keyword is correctly recognized according to the adjusted recognition result; adds the determined audio to the first training set , to obtain a second training set; delete the determined audio from the first training set to obtain a second test set.

In some embodiments, the number of audios in the first training set is greater than a first threshold.

In some embodiments, the method further includes: if the number of audios containing the keyword in the original scene training set is not greater than a first threshold, recording the audio containing the keyword and adding it to the first training set.

In some embodiments, the number of audios in the first test set is greater than a second threshold.

In some embodiments, the method further includes: if the keywords in the first test set are all correctly identified or the number of audios in the second test set is less than a third threshold, recording audios containing the keywords and adding to the second test set.

In a second aspect, an embodiment of the present disclosure provides an apparatus for speech recognition, including: an acquisition unit configured to acquire a keyword to be recognized; a composition unit configured to search for a keyword containing the keyword from an original scene training set The audio of the original scene constitutes the first training set, and obtains the audio containing the keywords and the non-original scene training set constitutes the first test set; the first training unit is configured to be based on the first training set. The speech recognition model performs a first round of incremental training to obtain a second speech recognition model; the recognition unit is configured to use the second speech recognition model to recognize the first test set, and the first test set is the key The audio whose words are correctly recognized is added to the first training set to obtain a second training set and a second test set; a second training unit is configured to perform a second training set on the second speech recognition model based on the second training set. The second round of incremental training obtains a third speech recognition model; the output unit is configured to input the second test set into the third speech recognition model to obtain an initial recognition result.

In some embodiments, the apparatus further includes a computing unit configured to: adjust the initial recognition result through a hot word decoder to calculate the recall rate of the keyword.

In some embodiments, the recognizing unit is further configured to: adjust the recognition result obtained by recognizing the first test set by the second speech recognition model by using the hot word decoder; determine according to the adjusted recognition result The audios whose keywords are correctly identified; the determined audios are added to the first training set to obtain the second training set; the determined audios are deleted from the first training set to obtain the second test set.

In some embodiments, the number of audios in the first training set is greater than a first threshold.

In some embodiments, the apparatus further includes a first recording unit configured to: if the number of audios including the keyword in the original scene training set is not greater than a first threshold, record audio including the keyword Audio is added to the first training set.

In some embodiments, the number of audios in the first test set is greater than a second threshold.

In some embodiments, the apparatus further includes a second recording unit, configured to: if the keywords in the first test set are all correctly identified or the number of audios in the second test set is less than a third threshold, record Audio containing the keyword is added to the second test set.

In a third aspect, embodiments of the present disclosure provide an electronic device for speech recognition, including: one or more processors; and a storage device on which one or more programs are stored, when the one or more programs are The one or more processors execute such that the one or more processors implement the method of any one of the first aspects.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, wherein the program implements the method according to any one of the first aspects when the program is executed by a processor.

The method and apparatus for speech recognition provided by the embodiments of the present disclosure perform the first round of incremental training by using a small amount of training data with keywords. After the first round of training is completed, the model obtained from the first round of training is used to add the correctly recalled data in the test set to the training set for the second round of incremental training. It can not only greatly reduce the training time and the amount of data required, but also solve the overfitting problem of incremental learning and the limitations of hot word decoding.

Description of drawings

Other features, objects and advantages of the present disclosure will become more apparent upon reading the detailed description of non-limiting embodiments taken with reference to the following drawings:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure may be applied;

2 is a flowchart of one embodiment of a method for speech recognition according to the present disclosure;

3 is a schematic diagram of an application scenario of the method for speech recognition according to the present disclosure;

4 is a flow chart of yet another embodiment of a method for speech recognition according to the present disclosure;

Fig. 5 is the hot word decoding flow chart of the method for speech recognition according to the present disclosure;

6 is a schematic structural diagram of an embodiment of an apparatus for speech recognition according to the present disclosure;

7 is a schematic structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure.

Detailed ways

The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the related invention, but not to limit the invention. In addition, it should be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that the embodiments of the present disclosure and the features of the embodiments may be combined with each other under the condition of no conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

FIG. 1 shows an exemplary system architecture 100 to which the speech recognition method and speech recognition apparatus according to the embodiments of the present application can be applied.

As shown in FIG. 1 , the system architecture 100 may include terminals 101 and 102 , a network 103 , a database server 104 and a server 105 . The network 103 is used as a medium for providing a communication link between the terminals 101 , 102 , the database server 104 and the server 105 . The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user 110 can use the terminals 101, 102 to interact with the server 105 through the network 103 to receive or send messages and the like. Various client applications may be installed on the terminals 101 and 102, such as model training applications, speech recognition applications, shopping applications, payment applications, web browsers, and instant messaging tools.

The terminals 101 and 102 here may be hardware or software. When the terminals 101 and 102 are hardware, they can be various electronic devices with microphones, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, moving picture experts compressed Standard Audio Layer 3), Laptops and Desktops, etc. When the terminals 101 and 102 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (eg, to provide distributed services), or as a single software or software module. There is no specific limitation here.

When the terminals 101 and 102 are hardware, an audio collection device may also be installed thereon. The audio collection device may be various devices that can realize the function of collecting audio, such as a microphone and so on. The user 110 can use the audio collection device on the terminals 101 and 102 to collect the voice of himself or others.

Database server 104 may be a database server that provides various services. For example, a sample set may be stored in a database server. The sample set contains a large number of samples. The samples may include audio and annotation information corresponding to the audio. In this way, the user 110 can also select samples from the sample set stored in the database server 104 through the terminals 101 and 102 .

The server 105 may also be a server that provides various services, such as a background server that provides support for various applications displayed on the terminals 101 and 102 . The background server can use the samples in the sample set sent by the terminals 101 and 102 to train the initial model, and can send the training results (such as the generated speech recognition model) to the terminals 101 and 102 . In this way, the user can apply the generated speech recognition model for speech recognition.

The database server 104 and the server 105 here can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software or software modules (eg, to provide distributed services), or as a single software or software module. There is no specific limitation here.

It should be noted that the speech recognition method provided by the embodiment of the present application is generally executed by the server 105 . Correspondingly, the apparatus for speech recognition is generally also provided in the server 105 .

It should be noted that, in the case where the server 105 can implement the relevant functions of the database server 104, the database server 104 may not be provided in the system architecture 100.

It should be understood that the numbers of terminals, networks, database servers and servers in FIG. 1 are merely illustrative. There can be any number of terminals, networks, database servers, and servers according to implementation needs.

Continuing to refer to FIG. 2 , a flow 200 of one embodiment of the method for speech recognition according to the present application is shown. The method for speech recognition may include the following steps:

Step 201, acquiring the keyword to be identified.

In this embodiment, the execution body of the speech recognition method (for example, the server 105 shown in FIG. 1 ) can acquire the keywords to be recognized in various ways. For example, the execution subject may acquire the existing keywords to be identified stored therein from a database server (eg, the database server 104 shown in FIG. 1 ) through a wired connection or a wireless connection. For another example, the user may collect keywords to be identified through a terminal (eg, the terminals 101 and 102 shown in FIG. 1 ). In this way, the execution body can receive the keywords to be identified collected by the terminal, and store the keywords to be identified locally.

The keyword to be recognized is applied to a new scene, and the speech recognition model trained on the original scene has a poor recognition effect on the keyword to be recognized. Keywords are generally professional terms or product names, such as "ETF", "XY Loan", etc.

Step 202: Search for audios containing keywords from the original scene training set to form a first training set, and acquire audios containing keywords but not in the original scene training set to form a first test set.

In this embodiment, the present application performs incremental learning based on the first speech recognition model of the original scene. Use the original speech recognition model to train on a small batch of new scene training sets to maximize the recognition performance on the original scene, and at the same time fit the new scene. Incremental learning means that the model uses new data to learn new task or scene knowledge without accessing the original data without forgetting the task or scene knowledge that has been learned. It can not only optimize the performance of the model in the new scene, but also preserve the accuracy of the model in the original scene to the greatest extent.

The first speech recognition model of the original scene is trained through the original scene training set. The original scene training set includes a large number of audios, each audio corresponds to the annotation information, and the annotation information is used to annotate the content of the audio. Some audios in the original scene training set include keywords, and these audios including keywords can be picked out to form the first training set. Then collect the audio that contains keywords and is not in the original scene training set to form the first test set, which is used to verify the performance of the trained model.

In some optional implementations of this embodiment, the number of audios in the first training set is greater than the first threshold (for example, 30 pieces), and if the training data is insufficient, the training effect is not ideal, and it is necessary to record the audio files containing the keyword. Audios are added to the first training set such that the total number of audios containing the keyword is greater than a first threshold.

In some optional implementations of this embodiment, the number of audios in the first test set is greater than a second threshold (for example, 15). The second threshold may be smaller than the first threshold. There needs to be enough test sets to verify the performance of the model, and in order to allow some of the data in the test set to be added to the training set, the test set still has a sufficient number of audios to verify the performance of the model.

Step 203 , perform a first round of incremental training on the first speech recognition model of the original scene based on the first training set to obtain a second speech recognition model.

In this embodiment, incremental training does not require a large amount of training data to be provided before the training process starts, but continuously uses new training data for training over time. The audio and label information in the first training set are respectively used as the input and expected output of the first speech recognition model, and the first speech recognition model is trained in a supervised manner to obtain the second speech recognition model. The specific training process is in the prior art, so it is not repeated here.

Step 204 , use the second speech recognition model to recognize the first test set, and add the audios whose keywords are correctly recognized in the first test set to the first training set to obtain the second training set and the second test set.

In this embodiment, the trained second speech recognition model is used to recognize the first test set. For the audio whose keywords are correctly identified, add it to the first training set to obtain the second training set, and delete the audio from the first test set to obtain the second test set. That is to say, after the first incremental training, the number of audios in the training set is increased, and the number of audios in the test set is reduced.

In some optional implementations of this embodiment, if all the keywords in the first test set are correctly identified or the number of audios in the second test set is less than a third threshold, the audio containing the keywords is recorded added to the second test set. The third threshold (eg, 5 bars) may be less than the second threshold (eg, 15 bars) and also less than the first threshold (eg, 30 bars). If the keywords of all audios in the test set are correctly recognized by the model, or there are fewer than 5 audios in the new test set, record new audios containing keywords so that there are no less than 5 audios in the new test set.

In some optional implementations of this embodiment, the recognition result obtained by recognizing the first test set by the second speech recognition model is adjusted by the hot word decoder; the keyword is determined according to the adjusted recognition result. Correctly recognized audio; adding the determined audio to the first training set to obtain a second training set; deleting the determined audio from the first training set to obtain a second testing set. In the model recognition process, this method uses the hot word decoder shown in Figure 5. During the speech recognition decoding process, the hot word decoder matches the end of the decoding path at each time step. If the end of the decoding path is the specified hot word, the path will be given corresponding bonus points. As shown in Figure 5, the specified decoding hot word is "ETF". In a certain time step, "ETF" appears at the end of a decoding path. After matching by the hot word decoder, the score is improved. When the decoding search is completed, the decoder outputs the decoding path with the highest score as the recognition result, which is the path with the keyword that obtains the hot word reward. In this way, through the adjustment of the acoustic model and the adjustment of hot words, the method effectively improves the recall rate of keywords.

Step 205 , perform a second round of incremental training on the second speech recognition model based on the second training set to obtain a third speech recognition model.

In this embodiment, the audio and label information in the second training set are used as the input and expected output of the second speech recognition model, respectively, and the second speech recognition model is trained in a supervised manner to obtain the third speech recognition model. The third speech recognition model has significantly improved the recognition effect of keywords.

The speech recognition method in this embodiment uses incremental learning to adjust the speech recognition model and improves the recall rate of keywords by combining with hot words when there are few training samples. The method can effectively retain the recognition performance of the new model on the old scene, while solving the problem that hot word decoding is limited by the acoustic probability distribution.

Step 206: Input the second test set into the third speech recognition model to obtain an initial recognition result.

In this embodiment, the performance of the third speech recognition model is verified by using the second test set to obtain an initial recognition result, that is, the accuracy of keyword recognition. Referring further to FIG. 3 , FIG. 3 is a schematic diagram of an application scenario of the method for speech recognition according to this embodiment. The specific process is as follows:

1) The user can input the keyword "ETF" to be identified to the server through the terminal device.

2) For the specified keyword, find the training data containing the keyword in the original scene training set to form the training set for the first incremental training. If the training set contains less than 30 audios with keywords, it needs to be recorded. Audio containing keywords should be no less than 30 in the training set. At the same time, collect no less than 15 data containing keywords and not in the original scene training set as the test set.

3) Use the speech recognition model of the original scene for incremental training, and use the training set in 1) to perform the first round of incremental training.

4) After the training is completed, use the new model obtained in 3) to identify the test set, and for the audio whose keywords are correctly identified, add it to the training set in 2); for the audio that has not been correctly identified, add it to the new in the test set. If the keywords of all audios in the test set are correctly recognized by the model, or there are fewer than 5 audios in the new test set, record new audios containing keywords so that there are no less than 5 audios in the new test set.

In the model recognition process, this method uses the hot word decoder shown in Figure 4. During the speech recognition decoding process, the hot word decoder matches the end of the decoding path at each time step. If the end of the decoding path is the specified hot word, the path will be given corresponding bonus points. As shown in Figure 4, the specified decoding hot word is "ETF", and in a certain time step, "ETF" appears at the end of a certain decoding path, and after matching by the hot word decoder, the score is improved. When the decoding search is completed, the decoder outputs the decoding path with the highest score as the recognition result, which is the path with the keyword that obtains the hot word reward. In this way, through the adjustment of the speech recognition model (especially the acoustic model) and the adjustment of hot words, the method effectively improves the recall rate of keywords.

5) Using the training set in 4), perform a second round of incremental training on the model obtained in 3).

6) After the training is completed, the model in 5) is obtained, which is the speech recognition model.

Continuing to refer to FIG. 4 , as an implementation of the methods shown in the above figures, the present application provides a flowchart of another embodiment of a method for speech recognition.

As shown in FIG. 4 , the method 400 for speech recognition in this embodiment may include:

Step 401, acquiring the keyword to be identified.

Step 402 , searching for audios containing keywords from the original scene training set to form a first training set, and acquiring audios containing keywords but not in the original scene training set to form a first test set.

Step 403 , perform a first round of incremental training on the first speech recognition model of the original scene based on the first training set to obtain a second speech recognition model.

Step 404 , use the second speech recognition model to recognize the first test set, and add the audios whose keywords are correctly recognized in the first test set to the first training set to obtain the second training set and the second test set.

Step 405: Perform a second round of incremental training on the second speech recognition model based on the second training set to obtain a third speech recognition model.

Steps 401-405 are basically the same as steps 201-205, and thus are not repeated here.

Step 406: Input the second test set into the third speech recognition model to obtain an initial recognition result.

In this embodiment, the performance of the third speech recognition model is verified by using the second test set to obtain an initial recognition result, that is, the accuracy of keyword recognition.

In step 407, the initial recognition result is adjusted by the hot word decoder, and the recall rate of the keyword is calculated.

In this embodiment, the hot word decoder shown in FIG. 5 is used. During the speech recognition decoding process, the hot word decoder matches the end of the decoding path at each time step. If the end of the decoding path is the specified hot word, the path will be given corresponding bonus points. As shown in Figure 5, the specified decoding hot word is "ETF". In a certain time step, "ETF" appears at the end of a decoding path. After matching by the hot word decoder, the score is improved. When the decoding search is completed, the decoder outputs the decoding path with the highest score as the recognition result, which is the path with the keyword that obtains the hot word reward. In this way, through the adjustment of the acoustic model and the adjustment of hot words, the method effectively improves the recall rate of keywords.

Optionally, if the recall rate does not reach the desired threshold, steps 404 to 407 can be repeatedly executed, and the audio that accurately identifies the keyword is continued to be selected from the test set and added to the training set, and the training model is repeated with the updated training set until The recall of the trained model reaches the desired threshold.

As can be seen from FIG. 4 , compared with the embodiment corresponding to FIG. 2 , the process 400 of the speech recognition method in this embodiment embodies the step of adjusting the recognition result of the speech recognition model through the hot word decoder. Therefore, the solution described in this embodiment can improve the recall rate of keywords.

Continuing to refer to FIG. 6 , as an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for speech recognition. This apparatus embodiment corresponds to the method embodiment shown in FIG. 2 , and the apparatus can be specifically applied to various electronic devices.

As shown in FIG. 6 , the apparatus 600 for speech recognition in this embodiment may include: an acquisition unit 601 , a composition unit 602 , a first training unit 603 , a recognition unit 604 , a second training unit 605 and an output unit 606 . Wherein, the obtaining unit 601 is configured to obtain the keyword to be identified; the composition unit 602 is configured to search for the audio containing the keyword from the original scene training set to form the first training set, and obtain the first training set containing the keyword and The audio of the non-original scene training set constitutes the first test set; the first training unit 603 is configured to perform the first round of incremental training on the first speech recognition model of the original scene based on the first training set, to obtain the second voice A recognition model; the recognition unit 604 is configured to use the second speech recognition model to recognize the first test set, and add the audio of the correctly recognized keywords in the first test set to the first training set , to obtain a second training set and a second test set; the second training unit 605 is configured to perform a second round of incremental training on the second speech recognition model based on the second training set to obtain a third speech recognition model ; The output unit 606 is configured to input the second test set into the third speech recognition model to obtain an initial recognition result.

In this embodiment, the specific processing of the acquiring unit 601 , the composition unit 602 , the first training unit 603 , the identification unit 604 , the second training unit 605 and the output unit 606 of the apparatus 600 for speech recognition may refer to the corresponding embodiment in FIG. 2 Step 201, Step 202, Step 203, Step 204, Step 205 and Step 206.

In some optional implementations of this embodiment, the apparatus 600 further includes a computing unit (not shown in the drawings), configured to: adjust the initial recognition result through a hot word decoder, calculate the key word recall.

In some optional implementations of this embodiment, the recognizing unit 604 is further configured to: adjust the recognition result obtained by recognizing the first test set by the second speech recognition model through a hot word decoder; According to the adjusted recognition result, determine the audio for which the keyword is correctly recognized; add the determined audio to the first training set to obtain a second training set; delete the determined audio from the first training set to obtain Second test set.

In some optional implementations of this embodiment, the number of audios in the first training set is greater than a first threshold.

In some optional implementations of this embodiment, the apparatus 600 further includes a first recording unit (not shown in the drawings), configured to: if the original scene training set contains the audio of the keyword The number of is not greater than the first threshold, then the audio containing the keyword is recorded and added to the first training set.

In some optional implementations of this embodiment, the number of audios in the first test set is greater than a second threshold.

In some optional implementations of this embodiment, the apparatus 600 further includes a second recording unit (not shown in the drawings), configured to: if all the keywords in the first test set are correctly identified or If the number of audios in the second test set is less than the third threshold, the audios containing the keyword are recorded and added to the second test set.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

FIG. 7 shows a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 7 , the device 700 includes a computing unit 701 that can be executed according to a computer program stored in a read only memory (ROM) 702 or loaded into a random access memory (RAM) 703 from a storage unit 708 Various appropriate actions and handling. In the RAM 703, various programs and data necessary for the operation of the device 700 can also be stored. The computing unit 701 , the ROM 702 , and the RAM 703 are connected to each other through a bus 704 . An input/output (I/O) interface 705 is also connected to bus 704 .

Various components in the device 700 are connected to the I/O interface 705, including: an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc. ; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the device 700 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 701 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 701 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as method speech recognition. For example, in some embodiments, the method speech recognition may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 708 . In some embodiments, part or all of the computer program may be loaded and/or installed on device 700 via ROM 702 and/or communication unit 709 . When a computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the method speech recognition described above may be performed. Alternatively, in other embodiments, computing unit 701 may be configured to perform method speech recognition by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a distributed system server, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server can be a distributed system server, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (10)

1. A method of speech recognition, comprising:

acquiring a keyword to be identified;

searching audio containing the keywords from an original scene training set to form a first training set, and acquiring audio containing the keywords and not forming a first test set from the original scene training set;

performing a first round of incremental training on a first voice recognition model of an original scene based on the first training set to obtain a second voice recognition model;

using the second speech recognition model to recognize the first test set, and adding the audio with the correctly recognized keywords in the first test set into the first training set to obtain a second training set and a second test set;

performing a second round of incremental training on the second voice recognition model based on the second training set to obtain a third voice recognition model;

and inputting the second test set into the third speech recognition model to obtain an initial recognition result.

2. The method of claim 1, wherein the method further comprises:

and adjusting the initial recognition result through a hotword decoder, and calculating the recall rate of the keyword.

3. The method of claim 1, wherein the adding the audio of the first test set with correctly recognized keywords to the first training set to obtain a second training set and a second test set comprises:

adjusting a recognition result obtained by recognizing the first test set by a second voice recognition model through a hot word decoder;

determining the audio frequency with the correctly recognized keywords according to the adjusted recognition result;

adding the determined audio into the first training set to obtain a second training set;

and deleting the determined audio from the first training set to obtain a second test set.

4. The method of claim 1, wherein the amount of audio in the first training set is greater than a first threshold.

5. The method of claim 4, wherein the method further comprises:

and if the number of the audios containing the keywords in the original scene training set is not larger than a first threshold value, recording the audios containing the keywords and adding the audios into the first training set.

6. The method of claim 1, wherein the number of tones in the first test set is greater than a second threshold.

7. The method of claim 1, wherein the method further comprises:

and if all the keywords in the first test set are correctly identified or the number of the audios in the second test set is smaller than a third threshold value, recording the audios containing the keywords and adding the audios into the second test set.

8. An apparatus for speech recognition, comprising:

an acquisition unit configured to acquire a keyword to be recognized;

the component unit is configured to search an audio component first training set containing the keywords from an original scene training set, and acquire an audio component first test set containing the keywords and not the original scene training set;

a first training unit configured to perform a first round of incremental training on a first speech recognition model of an original scene based on the first training set to obtain a second speech recognition model;

the recognition unit is configured to recognize the first test set by using the second speech recognition model, and add the audio with correctly recognized keywords in the first test set into the first training set to obtain a second training set and a second test set;

a second training unit configured to perform a second round of incremental training on the second speech recognition model based on the second training set, resulting in a third speech recognition model;

and the output unit is configured to input the second test set into the third speech recognition model to obtain an initial recognition result.

9. An electronic device for speech recognition, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.

Images