CN103310272B - Based on the DIVA neural network model manner of articulation that sound channel action knowledge base is improved - Google Patents

Abstract

The present invention relates to a kind of manner of articulation, be based especially on the DIVA neural network model manner of articulation that sound channel action knowledge base is improved. The described DIVA neural network model manner of articulation improved based on sound channel action knowledge base utilizes the DIVA neural network model after the improvement that with the addition of sound channel action knowledge base, for the voice not having in voice mapping ensemblen, revised auditory feedback information is obtained in conjunction with disturbance factor, recycle revised auditory feedback information training neutral net, decrease the model frequency of training when producing pronunciation, improve pronunciation accuracy.

Description

Pronunciation method based on improved DIVA neural network model based on vocal tract action knowledge base

technical field

The invention relates to a pronunciation method, in particular to an improved DIVA neural network model pronunciation method based on vocal tract action knowledge base.

Background technique

Neuro-computational speech model (Neuro-computational speech model) is a model that uses computer simulation to realize a series of complex processes such as speech generation, perception and acquisition. The composition of the neurocomputational speech model is very complex, including at least a cognitive part, a motor processing part and a sensory processing part: the role of the cognitive part is to generate neural activation (or generate phoneme representations) during the speech generation and speech perception stages; The motor processing part starts with the activation planning movement according to the generated phoneme representation, and ends with the movement of the vocal organs corresponding to the specific phoneme item; the sensory processing part includes generating the corresponding auditory representation according to the external sound signal and activating the corresponding phoneme representation.

So far, research on neural computing speech models has achieved many results, among which DIVA (Directions Intoof Articulators) model is a relatively advanced neural computing speech model for speech generation, perception and acquisition.

The DIVA model was developed by Professor Frank. Guenther and his team at the Speech Laboratory at Boston University. Among the neural computing speech models that are truly biophysically meaningful, the DIVA model is the most thoroughly defined and tested, and it is also the only adaptive neural network model that uses pseudo-inverse control technology. The DIVA model can describe the relevant processing processes in the process of speech acquisition, perception and generation, and can generate phonemes, syllables or words by controlling the analog vocal tract. Figure 1 shows the block diagram of the DIVA model.

Features of the DIVA model include:

The model includes two subsystems of feedforward control and feedback control;

The target area of the model is composed of the fundamental frequency F0, the first three formant frequencies and the corresponding somatosensory targets;

The input to the model is words, syllables or phonemes. Although the models have so far focused on short and simple phonetic sequences, their impact on language (i.e. prosody and prosodic structure, morphology and word boundaries, etc.) necessarily involves longer and more complex structures that has been considered in the model;

The model's explanation of co-articulation and its associated phenomena is similar to Keating's window model, but it has more advantages than the window model in explaining how the target is learned;

The DIVA model has achieved unprecedented success by fully applying the learning of the perception system. It is based on categorizing auditory sounds that already exist, without explaining how they are learned.

There are some defects in the DIVA model, which are mainly manifested in the following points: For the model, it is assumed that all state information given at a given point is available instantaneously; the model is assumed to have no neural delay and the system uses instantaneous feedback control; The reference frame for control can only choose the sensory reference frame space of the articulation organ or the auditory space reference frame, and the two cannot coexist at the same time; the description of the division of cortical and subcortical processing processes and the correlation of brain region components is relatively rough.

Contents of the invention

The technical problem to be solved by the present invention is to provide an improved DIVA neural network model pronunciation method based on the vocal tract action knowledge base for the deficiency of the above-mentioned background technology.

The present invention adopts following technical scheme for realizing above-mentioned purpose of the invention:

The improved DIVA neural network model pronunciation method based on vocal tract action knowledge base comprises the following steps:

Step 1, build an improved DIVA neural computing speech model: add the vocal tract action knowledge base that acts on the simulated pronunciation organs to the DIVA neural computing speech model,

In the vocal tract action knowledge base, the activation starts from the activation of the phoneme representation of the phonetic item, the planned motion of the high-frequency syllable has been obtained when the high-frequency syllable is processed, the voice mapping set activates the planned motion, and the vocal tract action corresponding to each syllable is generated motoneuron activation patterns, neuromuscular processing leading to movement of articulatory organs and allowing the generation of speech signals through articulatory-auditory models, activation of planned movements by activating phonetic plans of similar syllables through phonetic maps when processing low-frequency syllables;

Step 2, collecting the formant frequency of the articulation unit as the input of the DIVA neural computing speech model;

Step 3, the input volume of the DIVA neural network model is mapped to the voice mapping set, and all phoneme units in the voice mapping set are initialized to be inactive;

Step 4, input the vibration peak frequency of any pronunciation unit, and train the improved DIVA neural computing speech model based on the vocal tract action knowledge base:

When there are factor units with the same vibration peak frequency in the input pronunciation unit in the phonetic mapping, the simulated pronunciation organ sends out the input pronunciation unit directly through feed-forward control;

Otherwise, the simulated articulation organ learns to emit the input articulation unit through feedback control.

The improved DIVA neural network model pronunciation method based on the vocal tract action knowledge base, the simulated pronunciation organ described in step 4 sends out the pronunciation unit of input through feedback control The specific implementation is as follows:

Step A: Disturb the pronunciation unit of the simulated pronunciation organ, collect auditory feedback information and somatosensory feedback information of the DIVA model, and obtain somatosensory feedback commands from the somatosensory target area and somatosensory feedback information in the somatosensory error mapping set;

Step B, mapping the auditory feedback information and perturbed pronunciation units of the DIVA model to the auditory state mapping set;

Step C, the auditory error mapping set obtains an auditory feedback command according to the input amount of the DIVA neural network model and the auditory feedback information of the simulated pronunciation organ;

Step D, the pronunciation organ velocity and position mapping set obtains the training amount of the simulated pronunciation organ according to the somatosensory feedback command and the auditory feedback command, and the simulated pronunciation organ pronounces under the function of the vocal tract action knowledge base.

The present invention adopts the above-mentioned technical scheme, and has the following beneficial effects: reducing the training times of the model when producing pronunciation, and improving the accuracy of pronunciation.

Description of drawings

Figure 1 is a block diagram of the DIVA model.

Fig. 2 is a block diagram of the vocal tract vibration knowledge base.

Figure 3 is a block diagram of the improved DIVA model.

detailed description

Below in conjunction with accompanying drawing, the technical scheme of invention is described in detail:

Figure 2 shows the composition block diagram of the vocal tract action knowledge base model. The vocal tract movement knowledge base includes sensory-motor, speaking skills and comparable mental syllabary.

The workflow of the vocal tract action knowledge base model is divided into two stages: speech generation and classification perception:

The workflow of the speech production stage is as follows: the activation of the vocal tract action knowledge base model starts with the activation of the phoneme representation of the phonetic item, and this speech mode is to process syllables one by one. In the case of processing high-frequency syllables, the model has captured the planned motion of high-frequency syllables, first the planned motion is activated through the phonetic map set, and then the corresponding vocal tract action of each syllable produces a motoneuron activation pattern. Subsequent neuromuscular processing results in the movement of the vocal organs and allows the generation of speech signals through the articulation-auditory model. The previously acquired sensory states of the same syllable are simultaneously activated by the set of phonetic maps. The state TS in Fig. 3 corresponds to the state ES, and then generates the current syllable. In the presence of significant discrepancies, auditory and somatosensory error signals are passed through the phonetic map to alter the planned movement of a new or updated syllable. In the case of low-frequency syllables, the phonetic planning of similar syllables is activated by the phonetic map set to activate the planned motion module to generate the planned motion.

The workflow of the classification perception stage model is as follows: Speech perception begins with the generation of external sound signals. If it is aimed at phoneme recognition, it must be a signal of high-frequency syllables to achieve it. For this purpose, signals are preprocessed in peripheral and subcortical areas to load short-term memory into external auditory states. Its neural activation pattern is then passed to the training state map, leading first to the co-activation of neuronal regions at the level of the phonetic map, followed by the co-activation of specific neurons at the level of the phoneme map; the first represents the pronunciation of the syllable , the second indicates the phonology of the syllable. This neural pathway also co-activates a planned movement for high-frequency syllables through phonological mapping, also known as phonological perception of the dorsal tract. A second neural tract in speech perception, such as the ventral tract, directly links auditory activation patterns with speech processing modules. The dorsal tract is postulated to be important in speech acquisition, whereas the ventral tract is later dominant in speech perception in adults.

The improved DIVA model of the present invention is shown in Fig. 3, adding a vocal tract action knowledge base module and a disturbance module acting on the simulated vocal organs.

The model is trained on 200 instances using initializations from different speech-to-phoneme, sensory, and planned-motion mapping sets. The "knowledge" acquired by each instance of the model during the babble and imitation phases is stored in a bidirectional neural map from the speech map to the other map. A neuron is represented in a speech map set as:

(a) realization of vowel or vowel-consonant phoneme states;

(b) a planned motion state;

(c) an auditory state;

(d) A somatosensory state.

The training experiment included a babbling phase and an imitation phase (represented in the DIVA model). During the babbling phase, the model associates planned motor states with auditory states. Based on this, the model is able to generate planned motion during the imitation training phase.

During the imitation training phase, phoneme regions emerged at the phonetic map level. After these initial experiments, we moved on to more complex model languages, including vowel--, consonant-vowel--, and consonant-vowel--syllables, based on a larger set of consonants. Training again revealed a strict ordering of phonetic map sets with respect to phonetic properties, phonemic arrangement properties, and clustered consonant types.

In order to understand the workflow and pronunciation effects of the improved DIVA process, we conducted the following learning experiments using the improved DIVA model:

1. A five-vowel system /i,e,a,o,u/

2. A small consonant system (simple syllables composed of voiced stops /b, d, g/ and the 5 vowels acquired earlier)

3. A small language model, including five vowel systems, voiced and voiceless stops /b, d, g, p, t, k/, nasal /m, n/, lateral /l/ and three syllables Type (V, CV, CCV)

4. Test the most common 200 syllables in English with the test standard of a 6-year-old child.

Step 1, constructing an improved DIVA neural computing speech model: adding a vocal tract action knowledge base that acts on the simulated pronunciation organs to the DIVA neural computing speech model;

Step 2, collecting the formant frequency of the articulation unit as the input of the DIVA neural computing speech model;

Step 3, the input volume of the DIVA neural network model is mapped to the voice mapping set, and all phoneme units in the voice mapping set are initialized to be inactive;

Step 4, input the vibration peak frequency of any pronunciation unit, and train the improved DIVA neural computing speech model based on the vocal tract action knowledge base:

When there are factor units with the same vibration peak frequency in the input pronunciation unit in the phonetic mapping, the simulated pronunciation organ sends out the input pronunciation unit directly through feed-forward control;

Otherwise, the simulated articulation organ learns to emit the input articulation unit through feedback control.

In step 4, the specific implementation of the pronunciation unit that simulates the pronunciation organ to issue input through feedback control is as follows:

Step A: Disturb the pronunciation unit of the simulated pronunciation organ, collect auditory feedback information and somatosensory feedback information of the DIVA model, and obtain somatosensory feedback commands from the somatosensory target area and somatosensory feedback information in the somatosensory error mapping set;

Step B, mapping the auditory feedback information and perturbed pronunciation units of the DIVA model to the auditory state mapping set;

Step C, the auditory error mapping set obtains an auditory feedback command according to the input amount of the DIVA neural network model and the auditory feedback information of the simulated pronunciation organ;

Step D, the pronunciation organ velocity and position mapping set obtains the training amount of the simulated pronunciation organ according to the somatosensory feedback command and the auditory feedback command, and the simulated pronunciation organ pronounces under the function of the vocal tract action knowledge base.

The purpose of mapping the perturbed pronunciation unit to the auditory state mapping set is to further improve the auditory state mapping set. The addition of the vocal tract action knowledge base aims to enrich the actions of the simulated vocal organs, thereby improving the pronunciation accuracy and improving the learning efficiency of the entire DIVA model.

The modified model integrates sensory-motor and cognitive aspects. A serious problem faced by speech or sensorimotor models during phonological processing is the failure to model the development of phoneme maps during speech acquisition. We improve this problem and introduce a feasible solution: it is to directly couple the behavioral knowledge base and the mental vocabulary at the beginning of speech acquisition without explicitly introducing a phonetic mapping set. In this way, our modified DIVA model has a smaller pronunciation delay and higher accuracy than the original model.

Compared with the prior art, the present invention has the following significant advantages: the present invention is based on the DIVA neural network model, describes and simulates the relevant functions of pronunciation at the level of neuroanatomy and neurophysiology, and adds a disturbance module to the model, so that The model can produce pronunciation more efficiently and accurately; adding the vocal channel action knowledge base module to the model enriches the original vocal channel configuration of the DIVA model, reduces the number of training times for the model when producing pronunciation, and improves the accuracy of pronunciation. The DIVA neural network model can eventually be combined with the brain-computer interface (BCI) to construct a neural computing model for the generation and acquisition of Chinese speech that conforms to the law of Chinese speech and has real physiological significance, thereby further constructing a Chinese thinking characteristic. The Thought Reader lays the theoretical and practical foundations.

Claims (2)

1. the DIVA neural network model manner of articulation improved based on sound channel action knowledge base, it is characterised in that comprise the steps:

Step 1, builds the DIVA neurocomputing speech model improved: add the sound channel action knowledge base acting on simulation phonatory organ in DIVA neurocomputing speech model,

Described sound channel action knowledge base activates and starts from the activation that the phoneme of speech items characterizes, the programming movement of high frequency syllable has been obtained when processing high frequency syllable, voice mapping ensemblen activates programming movement, the sound channel action that each syllable is corresponding produces motor neuron and activates pattern, neuromuscular processes the motion that result in phonatory organ and allows to generate voice signal by pronunciation-auditory model, and the planning activated in similar syllable verbal audio by voice mapping ensemblen when processing low frequency syllable activates programming movement;

Step 2, gathers the formant frequency of pronunciation unit, as the input quantity of DIVA neurocomputing speech model;

Step 3, is mapped in voice mapping ensemblen by the input quantity of DIVA neural network model, and initializing all of phoneme unit in voice mapping ensemblen is unactivated state;

Step 4, the peak frequency of shaking of input arbitrarily pronunciation unit, train the DIVA neurocomputing speech model improved based on sound channel action knowledge base:

When being present in the identical factor unit of peak frequency of shaking of pronunciation unit of input in voice mapping ensemblen, simulation phonatory organ are directly over the feedforward and send the pronunciation unit of input;

Otherwise, simulation phonatory organ send the pronunciation unit of input through feedback control study.

2. the DIVA neural network model manner of articulation improved based on sound channel action knowledge base according to claim 1, it is characterised in that the pronunciation unit detailed description of the invention that the simulation phonatory organ described in step 4 send input through feedback control is as follows:

Simulation phonatory organ are imposed disturbance pronunciation unit, gather the auditory feedback information of DIVA model, somatesthesia feedback information by step A, and somatesthesia error map collection is obtained somatesthesia feedback command by somatesthesia target area and somatesthesia feedback information;

Step B, is mapped to hearing status mapping ensemblen by the auditory feedback information of DIVA model, disturbance pronunciation unit;

Step C, audition error map collection obtains auditory feedback order according to input quantity and the described simulation phonatory organ auditory feedback information of described DIVA neural network model;

Step D, phonatory organ speed and position mapping ensemblen obtain the training burden of described simulation phonatory organ according to somatesthesia feedback command, auditory feedback order, and simulation phonatory organ pronounce under the effect of sound channel action knowledge base.