CN111477220B - Neural network voice recognition method and system for home spoken language environment - Google Patents

Abstract

The invention discloses a neural network speech recognition method and system oriented to a home oral environment. The method includes: model construction: adding a long-term and short-term memory network to a deep neural network to construct a combined neural network DNN-LSTM model; The data set is preprocessed to obtain the feature vector set, and the feature vector set is used as the input of the DNN-LSTM model for iterative training to train to the optimal acoustic model; an input speech signal of an unknown language is trained to the DNN-LSTM model. The LSTM model obtains the Chinese output probability vector set and the English output probability vector set respectively; performs language matching according to the Chinese output probability vector set and the English output probability vector set, and outputs the judgment result; the present invention can quickly and accurately identify The content of the speaker can be widely used in actual home scenes.

Description

A Neural Network Speech Recognition Method and System Oriented to Home Spoken Language Environment

technical field

The invention belongs to the technical field of intelligent recognition, and in particular relates to a neural network speech recognition method and system for home spoken language environment.

Background technique

The key object of speech recognition research is speech, which converts speech signals into information that can be recognized by computers, so as to recognize the speaker's speech commands and text content. Speech recognition methods can basically be divided into three types: three methods based on linguistics and acoustics, model matching and neural networks. Although the first method appeared earlier, due to the limitations of its complex model, it has not yet reached a more practical stage; the second method is mostly used in the hidden Markov model, which can be used to label the probability model of the problem, And it shows that the model randomly generates observation sequences, which greatly improves the speech recognition technology. The third method uses shallow neural network learning and training to easily cause gradient instability, and manual extraction of sample features is time-consuming and laborious, and the recognition effect is not very good. In the traditional speech recognition system, the acoustic modeling method of GMM-HMM is the most widely used in practice, but when dealing with some complex speech signal problems in the home environment, the application scenario of the traditional model is relatively simple.

Contents of the invention

Purpose of the invention: In order to overcome the deficiencies of the prior art, the present invention provides a neural network speech recognition method oriented to the home spoken language environment, which can solve the problems of low speech recognition rate and poor recognition efficiency. The present invention also provides a home-oriented A Neural Network Speech Recognition System for Spoken Language Environments.

Technical solution: On the one hand, the neural network speech recognition method for home spoken language environment according to the present invention, the method includes:

Model construction: add a long-term short-term memory network to the deep neural network to build a combined neural network DNN-LSTM model;

Model training:

Chinese voice data training: preprocessing the collected Chinese voice data set to obtain a Chinese feature vector set, and using the Chinese feature vector set as the input of the DNN-LSTM model for iterative training to train to an optimal Chinese acoustic model;

English speech data training: preprocessing the collected English speech data set to obtain the English feature vector set, and using the English feature vector set as the input of the DNN-LSTM model for iterative training to train to the optimal English acoustic model;

Model test:

An input speech signal voice0 of an unknown language is passed through the Chinese acoustic model and the English acoustic model respectively to obtain a Chinese output probability vector set and an English output probability vector set;

Language matching is performed according to the Chinese output probability vector set and the English output probability vector set, and a judgment result is output.

Further, include:

The combined neural network DNN-LSTM model includes an input layer, a long short-term memory network, a second hidden layer, a third hidden layer, a fourth hidden layer and an output layer, and the long short-term memory network is used as the first hidden layer.

Further, include:

The number of nodes of the first hidden layer is 512, and its activation function selects sigmoid function and tanh function, and the number of nodes of the second hidden layer, the third hidden layer and the fourth hidden layer is 1024, and the activation function selects sigmoid function .

Further, include:

The Chinese feature vector set is carried out iterative training as the input of the DNN-LSTM model, and the training steps include:

(1) Initialize the weight matrix W and bias vector b in the model structure as a random value;

(2) Start iterations from the 1st to the maximum number of times; in each iteration, start traversing from the first voice data training sample to the last training sample;

(3) During the training process of each training sample, input the corresponding feature vector to the input layer; traverse from the first hidden layer to the output layer, and use the forward propagation algorithm to indicate the corresponding layer being traversed, and then according to The loss function represents the output layer; after the forward propagation algorithm is completed, it starts to traverse from the fourth hidden layer to the first hidden layer, and uses the back propagation algorithm to represent the corresponding first hidden layer;

(4) After the backpropagation algorithm is completed, it traverses from the first hidden layer to the output layer, and updates the weight matrix and bias vector W ⁿ , b ⁿ of the corresponding layer. The training is over; at this time, if the sample has not been traversed, continue to traverse the sample; if the sample has been traversed, proceed to the next iteration;

(5) When the change values of all W and b do not exceed the iteration threshold, stop the iterative cycle;

(6) Save the optimal weight matrix W and bias vector b of each layer.

Further, include:

Perform language matching according to the Chinese output probability vector set and the English output probability vector set, including:

Use the information entropy formula to calculate the information entropy corresponding to the Chinese output probability vector set P and the English output probability vector set P', which are respectively recorded as H and H'; where, P={p ₁ , p ₂ ,...,p _q }, P'={p ₁ ', p' ₂ ,..., p _t '}, q is the total number of Chinese acoustic model output categories, and t is the total number of English acoustic model output categories;

If there are probability values in the probability vector set output from the Chinese acoustic model that p _i is significantly greater than other probability values, the probability values in the probability vector set output from the English acoustic model are not much different, where 1≤i≤q;

And if the information entropy H corresponding to the Chinese acoustic model is smaller than the information entropy H' corresponding to the English acoustic model, the corresponding unknown language input voice signal voice0 is Chinese, and the output probability of the Chinese acoustic model is used as the final output result;

If there is a probability value in the probability vector set output from the English acoustic model that p' _j is significantly larger than other probability values, the probability values in the probability vector set output from the Chinese acoustic model are not much different.

And if the information entropy H' corresponding to the input to the English acoustic model is smaller than the information entropy H corresponding to the input to the Chinese acoustic model, the corresponding unknown language input voice signal voice0 is English, and the output probability of the English acoustic model is taken as the final output result.

On the other hand, the present invention also provides a system realized by the above-mentioned neural network speech recognition method for home oral environment, the system includes:

A model building block for adding a long short-term memory network to a deep neural network to construct a combined neural network DNN-LSTM model;

Model training module, it comprises Chinese model training unit and English model training unit again, described Chinese model training unit is used for the Chinese speech data set preprocessing of collection, obtains Chinese feature vector set, and described Chinese feature vector set Carry out iterative training as the input of described DNN-LSTM model, train to optimal Chinese acoustic model;

The English model training unit is used to preprocess the collected English speech data set to obtain the English feature vector set, and use the English feature vector set as the input of the DNN-LSTM model for iterative training to achieve the optimal English acoustics Model;

Model testing module, it comprises speech input unit and speech type judging unit again, and described speech input unit is used for the input speech signal voice0 of an unknown language, passes through described Chinese acoustic model and described English acoustic model respectively, obtains respectively The Chinese output probability vector set and the English output probability vector set; the speech type judgment unit is used to perform language matching according to the Chinese output probability vector set and the English output probability vector set, and output the judgment result.

Beneficial effects: Compared with the prior art, the present invention has the remarkable advantages that: the present invention combines the feature that LSTM utilizes the memory unit to record very long historical information and the feature that DNN can effectively extract high-level information in the data, The idea of adding LSTM to the first layer of the hidden layer of DNN was proposed, and a combined neural network combining DNN and LSTM was constructed for acoustic modeling, and the Chinese and English data sets were trained and tested, and the Chinese acoustic model and English Acoustic model, and compare the output results of the input speech signal in the Chinese acoustic model and the English acoustic model by referring to the concept of entropy, and use the result with a smaller entropy value as the output result of the acoustic model, so as to achieve the purpose of simple language recognition and improve The overall speech recognition rate is improved, and then it can quickly and accurately identify the content of the speaker in the home scene, which can be widely used in the actual home scene.

Description of drawings

Fig. 1 is the overall structural block diagram of the combined neural network speech recognition algorithm facing home spoken language environment of the present invention;

Fig. 2 is the DNN-LSTM model structural diagram of the present invention;

Figure 3 is the overall structure diagram of LSTM.

Detailed ways

In order to describe in more detail the combined neural network speech recognition algorithm oriented to the home spoken language environment proposed by the present invention, an example is illustrated as follows in conjunction with the accompanying drawings.

As shown in Figure 1, the overall structural block diagram of the combined neural network speech recognition algorithm for home spoken language environment, firstly combine the characteristics of DNN and LSTM to construct the DNN-LSTM model; then, use the DNN-LSTM model to train the Chinese data set and the English data set , save the Chinese acoustic model and the English acoustic model; finally, output the results through language matching, so as to achieve the purpose of language recognition and speech recognition.

DNN is a deep neural network, Deep Neural Networks, LSTM is a long-term short-term memory network, LongShort-Term Memory, as shown in Figure 3 is a three-door logic calculation structure diagram inside LSTM, the core element of LSTM is the cell state, which represents the information of the cell state over time transfer. It runs in a straight line along the entire chain, with only a few minor linear interactions, and information flows easily without drastic changes. During the transfer process, the information in the cell state is added or deleted through the input at the current moment, the hidden layer state at the previous moment, the cell state at the previous moment, and the gate structure. In speech recognition, the memory unit of the LSTM model is mainly used to store and process speech features. It implements three calculations, namely, the forgetting gate, the input gate and the output gate, and protects and controls the neuron state at the current moment through three gates. c _t , the details are as follows:

(1) Input gate: The function of this gate is to determine how much information in the input x _t is retained in c _t , and the realization formula is:

i _t =σ(W _i ·[h _t-1 , x _t ]+b _i ) (3)

保留下来。W_i、W_c表示权值矩阵，b_i、b_c表示偏置项，x_t-1,x_t,x_t+1分别表示上一个时刻、当前时刻和下一个时刻的输入；h_t-1,h_t,h_t+1分别表示上一个时刻、当前时刻和下一个时刻的神经元状态，σ表示sigmoid函数。Among them, it is the input of the input gate at time _t , through the input gate, the corresponding state of the input gate

Keep it. W _i , W _c represent the weight matrix, b _i , b _c represent the bias items, x _t-1 , x _t , x _t+1 represent the input of the previous moment, the current moment and the next moment respectively; h _{t- 1} , h _t , h _t+1 represent the neuron state at the previous moment, the current moment and the next moment respectively, and σ represents the sigmoid function.

(2) Forget gate: The function of this gate is to determine how many components of c _t-1 in the input at time t remain in c _t . The realization formula is:

f _t = σ(W _f ·[h _t-1 , x _t ]+b _f ) (5)

Among them, W _f represents the weight matrix, and b _f represents the bias term.

(3) Output gate: the function of this gate is to use the current output value h _t of the control unit c _t to LSTM. First, the state after passing through the input gate and the forget gate, that is, the realization formula of c _t is:

Among them, the first half is the component retained in c _t after the information passes through the forget gate, and the second half is the component retained in c _t after the information passes through the input gate. Then, to determine how much of c _t remains in h _t , the output is implemented as:

o _t =σ(W _o [h _t-1 , x _t ]+b _o ) (7)

Among them, o _t is the state of the output layer at time t, W _o represents the weight matrix, and b _o represents the bias item. Finally, after the output gate, the final output of the hidden layer is:

h _t ＝o _t *tanh(c _t ) (8)

Specifically: the combined neural network speech recognition method for home oral environment includes:

First, build the model: add a long-term short-term memory network to the deep neural network to build a combined neural network DNN-LSTM model;

As shown in Figure 2, the DNN-LSTM model has the following structure: layer 0 is the input layer, layers 1 to 4 are hidden layers, layer 5 is the output layer, and its activation function is the softmax function. In the hidden layer, the first layer is an LSTM network structure with 512 nodes. Its activation function selects sigmoid function and tanh function. In order to prevent excessive learning of data in the network, a Dropout strategy is added inside the neural unit; the last 3 layers Both are DNN network structures, the number of nodes in each layer is 1024, and the activation function selects the sigmoid function; that is, the combined neural network DNN-LSTM model includes an input layer, a long-term short-term memory network, a second hidden layer, a third hidden layer, and a fourth hidden layer. A hidden layer and an output layer, the long short-term memory network is used as the first hidden layer.

The model has 6 layers, the input vector of neurons in each layer is z ⁿ , and the output vector is y ⁿ , then:

z ⁿ = W ⁿ z ^n-1 + b ⁿ , n = 1,2,3,4,5 (1)

In the formula, W ⁿ is the weight matrix from the n-1th layer to the nth layer, b ⁿ is the bias of the nth layer. According to the input vector, the output can be obtained as:

y ⁿ =f _n (z ⁿ ) (2)

In the formula, f _n is the activation function of the nth layer.

Second, perform model training:

Chinese voice data training: preprocessing the collected Chinese voice data set to obtain the Chinese feature vector set vector0, and using the Chinese feature vector set as the input of the DNN-LSTM model for iterative training to train to the optimal Chinese acoustic model;

Among them, the preprocessing operations include: sampling, pre-emphasis, windowing and framing, endpoint detection, and the feature vector vector0 is used as the input of the DNN-LSTM model for iterative training to the optimal acoustic model China_model.

The training steps are as follows:

(1) Initialize the weight matrix W and bias vector b in the network structure to a random value.

(2) Start iterations from the first to the maximum number of times; in this embodiment, the maximum number of times is set to 50, and in each iteration, it is traversed from the first training sample to the last training sample, where i is used to Indicates the training samples being traversed;

(3) During the training process of each sample, the input vector is used as the first layer input of DNN, denoted by a ¹ ; and then traversed from the first layer of the hidden layer to the output layer, and n is denoted by the traversing layer layer, each layer performs forward propagation algorithm to calculate a ^{i, n} = f(z ^{i, n} ) = f(W ⁿ a ^{i, n-1} + b ⁿ ), which means the i-th traversal of the layer being traversed The input layer to which the sample corresponds.

；即正在遍历的该层正在遍历的第i个的训练样本对应的输出层，T为转置f′为求导，⊙表示同或运算。Calculate the output layer δ ^{i according to the loss function, L} is the output layer; after the forward propagation algorithm is completed, start traversing from the last layer of the hidden layer to the first layer of the hidden layer, and perform the back propagation algorithm to calculate δ ^{i, n} ＝(W ⁿ⁺¹ ) ^T δ ^{i, n+1} ⊙

; That is, the output layer corresponding to the i-th training sample that the layer is traversing is traversing, T is the transpose f' is the derivative, ⊙ represents the same OR operation.

(4) After the backpropagation algorithm is completed, start traversing from the first layer of the hidden layer to the output layer, and update W ⁿ and b ⁿ of the nth layer being traversed, then:

In this way, the training for a certain sample in one iteration process is over; at this time, if the sample has not been traversed, continue to traverse the sample; if the sample has been traversed, the next iteration will be performed, where m is the total number of training samples, α is the iteration step size;

(5) When the change values of all W and b do not exceed the iteration threshold, stop the iterative cycle;

(6) Save the optimal weight matrix W and bias vector b of each layer.

English speech data training: Preprocess the collected English speech data set to obtain the English feature vector set vector1, and use the English feature vector set as the input of the DNN-LSTM model for iterative training to train to the optimal English acoustic model ; Among them, the preprocessing operation includes: sampling, pre-emphasis, windowing and framing, endpoint detection, and the feature vector vector1 is used as the input of the DNN-LSTM model for iterative training, training to the optimal acoustic model English_model, the specific traversal steps and Chinese speech The steps of data training are the same and will not be repeated here.

Finally, perform model testing, the specific steps are:

An input speech signal voice0 of an unknown language is passed through the Chinese acoustic model and the English acoustic model respectively to obtain a Chinese output probability vector set and an English output probability vector set;

Language matching is performed according to the Chinese output probability vector set and the English output probability vector set, and a judgment result is output.

Use the information entropy formula to calculate the information entropy corresponding to the Chinese output probability vector set P and the English output probability vector set P', which are respectively recorded as H and H'; where, P={p ₁ , p ₂ ,...,p _q }, P'={p ₁ ', p' ₂ ,..., p _t '}, q is the total number of Chinese acoustic model output classifications, and t is the total number of English acoustic model output classifications;

The information entropy formula is:

If there is a probability value in the probability vector set output from the Chinese acoustic model that p _i is significantly greater than other probability values, the difference between each probability value in the probability vector set output from the English acoustic model is not obvious, where 1≤i≤q;

In the embodiment of the present invention, whether the difference between the probability values is significantly related to the output classification of the softmax output layer, the more output classifications, the smaller the corresponding range, the range is set to β, that is, the difference between the probability values is greater than or equal to β is the difference Obviously, if the difference range of each probability value is less than β, the difference is not obvious. During the experiment, when the output classification is 5 categories, the range β is about 0.2, and the more output classifications, the smaller the range.

And if the information entropy H corresponding to the Chinese acoustic model is smaller than the information entropy H' corresponding to the English acoustic model, the corresponding unknown language input voice signal voice0 is Chinese, and the output probability of the Chinese acoustic model is used as the final output The result; that is: according to the nature of information entropy, the greater the entropy, the greater the amount of information in the system, and the higher the uncertainty, when p ₁ =p ₂ =,...,=p _q , the maximum value is taken, and the Chinese output The information entropy of the probability is much smaller than the information entropy of the English output probability, that is, the matching degree of the Chinese speech signal in the Chinese acoustic model is higher, so the output probability of the Chinese acoustic model is taken as the final output result.

If there is a probability value in the probability vector set output from the English acoustic model that p' _j is significantly greater than other probability values, the probability values in the probability vector set output from the Chinese acoustic model are not much different, where 1≤j≤t;

And if the information entropy H' corresponding to the input to the English acoustic model is smaller than the information entropy H corresponding to the input to the Chinese acoustic model, the corresponding unknown language input voice signal voice0 is English, and the output probability of the English acoustic model is taken as the final output result.

On the other hand, the present invention also provides a kind of neural network speech recognition system facing home spoken language environment, and this system comprises:

A model building block for adding a long short-term memory network to a deep neural network to construct a combined neural network DNN-LSTM model;

Model training module, it comprises Chinese model training unit and English model training unit again, described Chinese model training unit is used for the Chinese speech data set preprocessing of collection, obtains Chinese feature vector set, and described Chinese feature vector set Carry out iterative training as the input of described DNN-LSTM model, train to optimal Chinese acoustic model;

The English model training unit is used to preprocess the collected English speech data set to obtain the English feature vector set, and use the English feature vector set as the input of the DNN-LSTM model for iterative training to achieve the optimal English acoustics Model;

Model testing module, it comprises speech input unit and speech type judging unit again, and described speech input unit is used for the input speech signal voice0 of an unknown language, passes through described Chinese acoustic model and described English acoustic model respectively, obtains respectively The Chinese output probability vector set and the English output probability vector set; the speech type judgment unit is used to perform language matching according to the Chinese output probability vector set and the English output probability vector set, and output the judgment result.

As for the system/apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to part of the description of the method embodiments.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or one operation from another entity or another operation, and do not necessarily require or imply that these entities Or any such actual relationship or order between operations.

Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely application embodiment, or an embodiment combining application and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

While preferred embodiments of the invention have been described, additional changes and modifications to these embodiments can be made by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment as well as all changes and modifications which fall within the scope of the invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (5)

1. A neural network speech recognition method facing home oral environment, characterized in that the method comprises: Model construction: add a long-term short-term memory network to the deep neural network to build a combined neural network DNN-LSTM model; Model training: Chinese voice data training: preprocessing the collected Chinese voice data set to obtain a Chinese feature vector set, and using the Chinese feature vector set as the input of the DNN-LSTM model for iterative training to train to an optimal Chinese acoustic model; English speech data training: preprocessing the collected English speech data set to obtain the English feature vector set, and using the English feature vector set as the input of the DNN-LSTM model for iterative training to train to the optimal English acoustic model; Model test: An input speech signal voice0 of an unknown language is passed through the Chinese acoustic model and the English acoustic model respectively to obtain a Chinese output probability vector set and an English output probability vector set; Perform language matching according to the Chinese output probability vector set and the English output probability vector set, and output a judgment result; Perform language matching according to the Chinese output probability vector set and the English output probability vector set, including: Use the information entropy formula to calculate the information entropy corresponding to the Chinese output probability vector set P and the English output probability vector set P', which are respectively recorded as H and H'; where, P={p ₁ ,p ₂ ,...,p _q }, P'={p′ ₁ ,p′ ₂ ,...,p′ _t }, q is the total number of output categories of the Chinese acoustic model, and t is the total number of output categories of the English acoustic model; If there are probability values in the probability vector set output from the Chinese acoustic model that p _i is significantly greater than other probability values, the probability values in the probability vector set output from the English acoustic model are not much different, where 1≤i≤q; And if the information entropy H corresponding to the Chinese acoustic model is smaller than the information entropy H' corresponding to the English acoustic model, the corresponding unknown language input voice signal voice0 is Chinese, and the output probability of the Chinese acoustic model is used as the final output result; If there is a probability value in the probability vector set output from the English acoustic model that p′ _j is significantly greater than other probability values, the probability values in the probability vector set output from the Chinese acoustic model are not much different, where 1≤j≤t; And if the information entropy H' corresponding to the input to the English acoustic model is smaller than the information entropy H corresponding to the input to the Chinese acoustic model, the corresponding unknown language input voice signal voice0 is English, and the output probability of the English acoustic model is taken as the final output result. 2. the neural network speech recognition method facing home spoken language environment according to claim 1, is characterized in that, described combined neural network DNN-LSTM model comprises input layer, long short-term memory network, the second hidden layer, the third hidden layer layer, the fourth hidden layer and the output layer, and the long short-term memory network is used as the first hidden layer. 3. the neural network speech recognition method facing home spoken language environment according to claim 2, is characterized in that, the node number of the first hidden layer is 512, and its activation function selects sigmoid function and tanh function, and the second hidden layer The number of nodes in the first layer, the third hidden layer and the fourth hidden layer is 1024, and the activation function selects the sigmoid function. 4. the neural network speech recognition method facing home spoken language environment according to claim 3, is characterized in that, described Chinese feature vector set carries out iterative training as the input of described DNN-LSTM model, and training step comprises: (1) Initialize the weight matrix W and bias vector b in the model structure as a random value; (2) Start iterations from the 1st to the maximum number of times; in each iteration, start traversing from the first voice data training sample to the last training sample; (3) During the training process of each training sample, input the corresponding feature vector to the input layer; traverse from the first hidden layer to the output layer, and use the forward propagation algorithm to indicate the corresponding layer being traversed, and then according to The loss function represents the output layer; after the forward propagation algorithm is completed, it starts to traverse from the fourth hidden layer to the first hidden layer, and uses the back propagation algorithm to represent the corresponding first hidden layer; (4) After the backpropagation algorithm is completed, start traversing from the first hidden layer to the output layer, and update the weight matrix and bias vector W ⁿ , b ⁿ of the corresponding layer, n is the layer being traversed, n=1, 2, 3, 4, 5, so far the training for one sample in one iteration process is over; at this time, if the sample has not been traversed, continue to traverse the sample; if the sample has been traversed, proceed to the next training iteration; (5) When the change values of all W and b do not exceed the iteration threshold, stop the iterative cycle; (6) Save the optimal weight matrix W and bias vector b of each layer. 5. A system realized by the neural network speech recognition method facing the home spoken language environment according to any one of claims 1-4, characterized in that the system comprises: A model building block for adding a long short-term memory network to a deep neural network to construct a combined neural network DNN-LSTM model; Model training module, it comprises Chinese model training unit and English model training unit again, described Chinese model training unit is used for the Chinese speech data set preprocessing of collection, obtains Chinese feature vector set, and described Chinese feature vector set Carry out iterative training as the input of described DNN-LSTM model, train to optimal Chinese acoustic model; The English model training unit is used to preprocess the collected English voice data set to obtain the English feature vector set, and use the English feature vector set as the input of the DNN-LSTM model for iterative training to achieve the best English acoustics Model; Model test module, it comprises speech input unit and speech type judging unit again, and described speech input unit is used for the input speech signal voice0 of an unknown language, passes through described Chinese acoustic model and described English acoustic model respectively, obtains respectively The Chinese output probability vector set and the English output probability vector set; the speech type judging unit is used to perform language matching according to the Chinese output probability vector set and the English output probability vector set, and output the judgment result; Perform language matching according to the Chinese output probability vector set and the English output probability vector set, including: Use the information entropy formula to calculate the information entropy corresponding to the Chinese output probability vector set P and the English output probability vector set P', which are respectively recorded as H and H'; where, P={p ₁ ,p ₂ ,...,p _q }, P'={p′ ₁ ,p′ ₂ ,...,p′ _t }, q is the total number of output categories of the Chinese acoustic model, and t is the total number of output categories of the English acoustic model; If there are probability values in the probability vector set output from the Chinese acoustic model that p _i is significantly greater than other probability values, the probability values in the probability vector set output from the English acoustic model are not much different, where 1≤i≤q; And if the information entropy H corresponding to the Chinese acoustic model is smaller than the information entropy H' corresponding to the English acoustic model, the corresponding unknown language input voice signal voice0 is Chinese, and the output probability of the Chinese acoustic model is used as the final output result; If there is a probability value in the probability vector set output from the English acoustic model that p′ _j is significantly greater than other probability values, the probability values in the probability vector set output from the Chinese acoustic model are not much different, where 1≤j≤t; And if the information entropy H' corresponding to the input to the English acoustic model is smaller than the information entropy H corresponding to the input to the Chinese acoustic model, the corresponding unknown language input voice signal voice0 is English, and the output probability of the English acoustic model is taken as the final output result.

Images