JP6721165B2 - Input sound mask processing learning device, input data processing function learning device, input sound mask processing learning method, input data processing function learning method, program - Google Patents

Description

The present invention relates to a learning technique that can be used to generate a mask for masking an input sound and a processing function for processing input data.

The sound source enhancement technology is a technology that emphasizes a desired target sound from the observation signal buried in noise, and it has been used for many years due to its wide range of applications such as preprocessing for voice recognition, sound collection for highly realistic sound, and hearing aid. Being researched. As one example of its implementation, there is a process based on a time-frequency mask such as Wiener filtering.

In order to formulate the sound source enhancement, the observed signal is first modeled. The signal X _{ω, τ} ∈ C ^{Ω × Τ} which is obtained by cutting out the observation signal of the m-th microphone with a length of several tens of ms and performing a short-time Fourier transform (STFT) for each time frame is the desired source. The signal S _{ω, τ} ∈ C ^{Ω × Τ} and the noise N _{ω, τ} ∈ C ^{Ω × Τ} are described as superposed as follows.

Here, ω ∈ {1,..., Ω} and τ ∈ {1,..., Τ} are variables representing frequency and time indexes.

Non-linear filtering is processing based on a time-frequency mask that adjusts the gain for each time-frequency component. In the sound source enhancement based on the time-frequency mask, the source signal S _ω,τ is emphasized by multiplying the observed signal X _ω,τ by the time-frequency mask G _ω,τ ∈ [0,1] having a value of 0 to 1. The obtained signal S ^ _{ω, τ} ∈ C ^{Ω × T} is obtained (see FIG. 1).

A Wiener mask is a typical method for calculating the time-frequency mask G _ω,τ . The Wiener mask is a mask that minimizes the mean squared error (MSE) of S _ω,τ and S ^ _ω,τ when the source signal and all noises are uncorrelated and stationary with each other. However, since the source signal and noise are often non-stationary, the following time-varying Wiener mask G ^WF _ω,τ is often used in practice.

In order to calculate the Wiener mask, it is necessary to estimate both the amplitude spectrum of the source signal |S _ω,τ | and the amplitude spectrum of noise |N _ω,τ | In order to reduce the number of values, it is assumed that the additivity of the source signal and noise holds in the power spectrum region as follows,

Often, either the source signal or the noise is estimated and the Wiener mask is calculated approximately. For example, when estimating the amplitude spectrum |S _ω,τ | of the source signal, the Wiener mask can be calculated as follows.

In recent years, a deep neural network (DNN) has been applied to time-frequency mask estimation as a projection function for non-linearly projecting an observed signal onto parameters of the time-frequency mask (Non-Patent Document 1). Estimate the vector y^ _τ by the following formula, where x _{τ is} the vector in which the time-frequency elements of the observed signal X _ω,τ are arranged and vector y _{τ in} which the parameters for calculating the time-frequency mask are arranged (see Fig. 2). ). For example, the vector x _{τ in} FIG. 2 is a frame-combined amplitude spectrum or MFCC (Mel-Frequency Cepstrum Coefficients), and the vector y^ _τ is the amplitude spectrum of the source signal.

Here, L is the number of layers of the neural network, and W ^(j) and b ^(j) are the weight matrix and bias vector of the jth layer, respectively. That is, the parameter Θ _Μ of the DNN is Θ _Μ = {W ^(j) , b ^(j) |j=2,..., L}. Further, σ _θ is a non-linear function called an activation function, and a sigmoid function or a ramp function is used. Note that z _τ ⁽¹⁾ =x _τ . In order to consider both the frequency information and the time information of the observed signal, the vector x _{τ that} is the input of the DNN has the time-frequency elements of the observed signal X _{ω, τ} ∈ C ^{Ω × T} arranged as follows, for example. Vector.

Here, t on the right shoulder of the parentheses in Expression (9) represents transposition. Further, P _b and P _f are the number of time frames before and after considering, and are called context windows.

When the time-frequency mask is calculated from the amplitude spectrum |S _ω,τ | of the source signal, the vector y _{τ that} is the output of the DNN is, for example, as follows.

For the DNN parameter Θ _Μ , prepare a large amount of data (in this example, vector x _τ and vector y _τ paired) in which the observed signal and label data (time-frequency mask parameter) are paired, It is generated by supervised learning using error back propagation so as to minimize a differentiable evaluation value such as a squared error.

However, in order to suppress the number of dimensions of the input/output vector, the source signal S _ω,τ and the noise N _ω,τ can be compressed by a mel filter bank of about 64 dimensions. When such a compression is performed, the mel filter bank compression is regarded as a matrix operation, and the inverse matrix is used to restore the output of the DNN to the original frequency domain and design the time-frequency mask.

Y. Xu, J. Du, LR Dai and CH Lee, “A regression approach to speech enhancement based on deep neural networks”, IEEE/ACM Trans. Audio, Speech and Language Processing, Vol.23, No.1, pp. 7-19, 2015.

Conventionally, the evaluation value that can be used for error back propagation has been limited to a differentiable value such as a squared error. However, the performance evaluation value of the sound source enhancement is not only a differentiable one such as a squared error, depending on the application of the sound source enhancement, but also PESQ (perceptual evaluation of speech quality) and STOI (short-time objective intelligibility measure). Those that cannot be differentiated are also used (reference non-patent document 1, reference non-patent document 2).
(Reference Non-Patent Document 1: ITU-T Recommendation P.862, "Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs", 2001. )
(Reference Non-Patent Document 2: CHTaal, RCHendriks, R. Heusdens, and J. Jensen, “An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech”, IEEE Transactions on Audio, Speech and Language Processing, Vol.19, pp. .2125-2136, 2011.)

Therefore, in order to learn DNN appropriately according to the application, a DNN learning framework that optimizes DNN parameters using non-differentiable evaluation values such as PESQ is required.

Therefore, the present invention provides a learning technique applicable to the purpose of generating a mask for masking an input sound using various evaluation values including non-differentiable evaluation values and a processing function for processing input data. The purpose is to do.

According to one aspect of the present invention, a mask G _τ ( _τ ∈ {1,..., Τ}) is generated when an input vector x _τ ( _τ ∈ {1,..., Τ}) based on an input sound is input. An input vector based on N input sounds (N is an integer between 1 and τ inclusive) based on the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,…, Τ}) modeling the generation probability A mask generation unit that generates _N masks _GN from x _N and the mask _GN is used to generate N output sounds by masking the N input sounds from the N input sounds. A mask processing unit that obtains a reward coefficient of the mask G _N for the N output sounds, the reward coefficient, and the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ Using the generation probability q(G _N |x _N ) at which the mask G _N is generated when the input vector x _N based on {1,...,Τ}) is input, the posterior probability distribution p( G _τ |x _τ )( _τ ∈ {1,...,Τ}) is updated, and the reward coefficient is generated when the evaluation value of the output sound and the input sound are input. It is determined from the certainty factor that is the certainty of the mask G _N.

According to one aspect of the present invention, a processing function G _τ ( _τ ∈ {1,..., Τ}) is generated when an input vector x _τ ( _τ ∈ {1,..., Τ}) based on input data is input. Based on the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,…, Τ}) that models the generation probability of Using the processing function generator that generates _N processing functions G _N from the vector x _N and the processing function G _N , the N input data are processed by the processing function N from the N input data. Processing function applying unit for generating output data, a reward coefficient acquisition unit for obtaining the reward coefficient of the processing function G _N for the N output data, the reward coefficient, and the posterior probability distribution p(G _tau | using the _{x N) | x τ) (} τ∈ {1, ..., Τ} process when an input said input vector x _N based on) the function G _N is generated probability q generated (G _N And an update unit for updating the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}), wherein the reward coefficient is the evaluation value of the output data and the input value. It is determined from the certainty factor, which is the certainty of the processing function G _N generated when data is input.

According to the present invention, by updating the posterior probability distribution using various evaluation values including non-differentiable evaluation values, a mask for masking the input sound and a processing function for processing the input data are provided. It is possible to learn the posterior probability distribution for generation.

The block diagram which shows an example of a structure of the sound source emphasis apparatus 900. The block diagram which shows an example of a structure of the time frequency mask generation part 910 using DNN. The block diagram which shows an example of a structure of the sound source emphasis learning apparatus 100. The flowchart which shows an example of operation|movement of the sound source emphasis learning apparatus 100. 3 is a block diagram showing an example of the configuration of a DNN parameter initial value generation unit 110. FIG. 6 is a flowchart showing an example of the operation of the DNN parameter initial value generation unit 110. 3 is a block diagram showing an example of the configuration of a DNN-RL parameter generation unit 120. FIG. 6 is a flowchart showing an example of the operation of the DNN-RL parameter generation unit 120. 3 is a block diagram showing an example of the configuration of a DNN-RL time domain output signal generation unit 124. FIG. 9 is a flowchart showing an example of the operation of the DNN-RL time domain output signal generation unit 124. 3 is a block diagram showing an example of the configuration of a sound source enhancement device 200. FIG. 6 is a flowchart showing an example of the operation of the sound source enhancement device 200. The block diagram which shows an example of a structure of the input sound mask process learning apparatus 300. The flowchart which shows an example of operation|movement of the input sound mask process learning apparatus 300. The block diagram which shows an example of a structure of the input sound mask process learning apparatus 301. The flowchart which shows an example of operation|movement of the input sound mask process learning apparatus 301. The block diagram which shows an example of a structure of the input data processing function learning apparatus 400. The flowchart which shows an example of operation|movement of the input data processing function learning apparatus 400.

Hereinafter, embodiments of the present invention will be described in detail. It should be noted that components having the same function are denoted by the same reference numeral, and redundant description will be omitted.

<Technical background>
Evaluation values such as PESQ and STOI cannot be differentiated like the error between the estimated value of the time-frequency mask (or its parameter) and the label data (see equation (12)). Therefore, here, instead of the non-linear projection approach that directly estimates the time-frequency mask (or its parameters) as in the past, the posterior probability distribution (or the Its parameters). The property that this posterior probability distribution should satisfy is described as an objective function, and DNN (DNN parameter Θ _Μ ) is learned using this objective function.

Previously, the time-frequency mask itself and its parameters were output as a DNN, and the DNN was learned so as to minimize the differentiable evaluation values such as the squared error between the label data prepared in advance and the DNN output (Fig. 2, see equation (12)). However, here, the probability density function (or its parameter) of the time-frequency mask that maximizes the evaluation value under the observation signal is output by DNN. Then, the DNN is learned using a new objective function T _ar that maximizes the evaluation function R that outputs an evaluation value that cannot be differentiated. That is, conventionally, the DNN was learned using the equation (12), but here, the DNN is learned using the equation (26) described later.

The details will be described below.
《 _Deriving the objective function T _ar 》
The evaluation value to be maximized in the embodiment of the present invention includes the evaluation value that can be calculated from the output signal S^ _ω,τ of speech enhancement such as PESQ and STOI. Further, it may be an evaluation value that can be obtained from the output signal S^ _{ω,τ by} a method other than calculation, such as a result of subjective evaluation such as a MOS value or a binary value indicating good or bad. Further, for example, if it is desired to optimize the sound source enhancement for voice recognition, a binary value indicating whether or not the result of voice recognition is correct may be used as the evaluation value.

Further, in the embodiment of the present invention, since the sound source enhancement is performed by the time frequency mask processing, the evaluation value can be regarded as a function of the time frequency mask sequence G. That is,

Is.

Here, the evaluation function that outputs the evaluation value is R, and the evaluation value to be maximized is R(G). Then, the problem results in finding the parameter Θ _Μ of the DNN that outputs the time-frequency mask that maximizes the evaluation value R(G).

Conventionally, the output of the DNN has been to output the time-frequency mask itself and its parameters like the vector y ^ _τ in equation (6). Here, Μ(x _τ | Θ _Μ ) is defined as the posterior probability that the time-frequency mask G _τ maximizes the evaluation value as follows.

Then, the time-frequency mask G _τ is obtained by the following posterior probability maximization estimation.

Here, since the posterior probability p(G _τ |x _τ ,Θ _Μ ) is a continuous probability distribution for the time-frequency mask G _τ , Eq. (16) maximizes p(G _τ |x _τ ,Θ _Μ ). It can be regarded that the G _τ to be _converted is directly obtained, that is, the time-frequency mask G _τ is generated.

In order to realize the sound source enhancement that maximizes the evaluation value R(G), the objective function T _ar is designed as the expected value of the evaluation value R(G) as follows.

However,

Where p(X) is the probability of obtaining the observed signal sequence X, and p(G|X, Θ _Μ ) is the probability density function that maximizes the evaluation value of the time-frequency mask sequence G under the observation signal sequence X. Represent

Further, in the sound source enhancement, the time-frequency masking does not affect the observation signal at the next time, and the design of the time-frequency mask at the time τ is performed independently of other times. Considering this, the probability density function p(G|X, Θ _Μ ) in the sound source enhancement can be described in the following simple form.

Therefore, the objective function T _ar can be described as follows.

In order to investigate the property of this objective function T _ar, the gradient with respect to Θ _Μ of the objective function T _ar appearing in equation (21) is obtained. The gradient of the objective function T _ar with respect to Θ _Μ can be calculated as follows.

Here, the expected value of equation (23) is replaced with the arithmetic mean of I (I is an integer of 1 or more) episodes. Then, equation (23) can be rewritten as follows.

Where R _ew (i)=R(G ⁽ⁱ⁾ )p(G ⁽ⁱ⁾ |X ⁽ⁱ⁾ ,Θ _Μ ), the superscript/subscript i or (i) is the i-th episode Indicates that it is a variable. Hereinafter, _{Rew is referred} to as a reward coefficient.

Qualitatively consider the meaning of the reward coefficient R _ew (i). The first term R(G ⁽ⁱ⁾ ) is a term related to the evaluation value. If the generated time-frequency mask is a good evaluation, the value is positive, and if it is a bad evaluation, the value is negative. Evaluation of frequency mask". Also, the second term p(G ⁽ⁱ⁾ |X ⁽ⁱ⁾ ,Θ _Μ ) is a term related to the generation probability of the time-frequency mask, and the confidence of the time-frequency mask generated by itself in the current DNN parameter Θ _Μ It represents the “confidence of the time-frequency mask” that it was generated with. Since the reward coefficient R _ew (i) is the product of these two terms, if the time-frequency mask generated with confidence improves the evaluation value, the generation probability lnp(G ⁽ⁱ⁾ _τ |X ⁽ⁱ⁾ _τ , Θ _Μ ) is greatly increased, and the probability of generation lnp(G ⁽ⁱ⁾ _τ |X ⁽ⁱ⁾ _τ , Θ _Μ ) is greatly reduced if the time-frequency mask generated with confidence lowers the evaluation value. ing. Further, when the evaluation value is improved or decreased by the time-frequency mask generated without confidence, the result may be accidental, and thus the generation probability lnp(G ⁽ⁱ⁾ _τ | It has the function of suppressing the increase or decrease of X ⁽ⁱ⁾ _τ , Θ _Μ ).

In summary, the objective function T _ar for generating a time-frequency mask that maximizes non-differentiable evaluation values such as PESQ, STOI, and MOS values is evaluated value R(G ⁽ⁱ⁾ ) and confidence p( ^{G (i) | X (i} ), the weighted with theta _Micromax), log likelihood lnp against the generated time-frequency mask _{^{(G (i) τ | X}} (i) τ, the arithmetic mean of theta _Micromax).

Note that the second term p(G ⁽ⁱ⁾ _τ |X ⁽ⁱ⁾ _τ , Θ _Μ ) of the reward coefficient R _ew (i) is not differentiated by Θ _Μ .

In the derivation of the objective function T _ar defined by the equation (26), the discussion has been made on the evaluation value which is not differentiable, but this discussion is not limited to the evaluation value which is not differentiable. That is, the objective function T _ar defined by the equation (26) and the reward coefficient R _ew (i) defined by the equation (26′) can be applied to differentiable evaluation values.

《DNN parameter Θ _Μ learning algorithm》
Below, using the objective function T _ar of Eq. (26), the DNN parameters that output the distribution parameters of the posterior probability p(G _τ |x _τ , Θ _Μ ) that the time-frequency mask G _τ maximizes the evaluation value. An algorithm for learning Θ _Μ will be described.

(Design of distribution parameters of DNN output p(G _τ |x _τ , Θ _Μ ))
First, p(G _τ |x _τ ,Θ _Μ ) is expressed as a distribution that can be differentiated by the DNN parameter Θ _Μ , and the distribution parameter of p(G _τ |x _τ , Θ _Μ ) is estimated and output by a neural network. ..

Therefore, p(G _τ |x _τ , Θ _Μ ) is modeled as a complex Gaussian distribution that is easy to differentiate and numerically easy to handle with the DNN parameter Θ _Μ .

Here, the small circle on the right side of equation (27) represents the Hadamard product, and R and I on the right side of equation (29) represent the real and imaginary parts of the complex number, respectively.

Then, the average vector μ(x _τ ) and the variance vector σ(x _τ ) that are distribution parameters of the complex Gaussian distribution p(G _τ |x _τ , Θ _Μ ) are set as the output of the DNN.

Here, a sigmoid function is used as an activation function in order to use the average vector μ(x _τ ) as an estimated value of the time-frequency mask G _τ ∈ [0,1].

Here, the posterior probability p(G _τ |x _τ ,Θ _Μ ) is modeled by a complex Gaussian distribution, and the output of the DNN is the mean vector μ(x _τ ) and the variance vector σ(x _τ ). The posterior probability p(G _τ |x _τ , Θ _Μ ) itself may be used as the output of the DNN (see FIG. 2).

(Design of evaluation function R)
The values of PESQ and STOI, which are typical evaluation values, vary depending on not only the performance of sound source enhancement but also the SNR of the observed signal and the type of noise. Therefore, the evaluation value of the output sound emphasized by the sound source using the time-frequency mask obtained from the parameter Θ _Μ learned by the above-mentioned DNN (DNN using the objective function T _ar of Equation (26)) and the conventional MMSE (minimum The evaluation value obtained by comparing the evaluation values of the output sound (Non-Patent Document 1) emphasized by the sound source using the time frequency mask obtained from the parameter Θ _Μ learned by the DNN using the mean squared error criterion (hereinafter, Calculate the comparative reward).

Hereinafter, for simplicity, the DNN learned by using the objective function T _ar of Expression (26) is referred to as DNN-RL. A DNN learned using the MMSE criterion is called a DNN-MMSE. Similarly, for simplification, the output sound obtained by DNN-RL and the time-frequency mask obtained by DNN-MMSE are the output sounds emphasized by using the time-frequency mask obtained by DNN-RL. The output sound whose sound source is emphasized by using it is called the output sound obtained by DNN-MMSE.

Note that DNN learning using the objective function T _ar of Expression (26) is called DNN-RL learning, and DNN learning using the MMSE criterion is called DNN-MMSE learning.

Let Z ^{RL be} the evaluation value of the output sound obtained by DNN-RL, and Z ^{MMSE be} the evaluation value of the output sound obtained by DNN-MMSE. Then, a comparative reward R(G), which is an evaluation value obtained by comparing these two evaluation values, is obtained as follows.

Where α(>0) is the scaling factor of the comparative reward and tanh is the hyperbolic tangent function for clipping the comparative reward.

This comparative reward R(G) is a value inspired by the outcome of the game. Z ^RL that is greater than Z ^MMSE is that high evaluation value of output sounds obtained by DNN-RL than the evaluation value of output sounds obtained by DNN-MMSE, in order to obtain the Z ^RL It can be judged that the sound source enhancement performed was correct (at this time, R(G)>0). On the other hand, it Z ^RL is called Z ^MMSE smaller is that the evaluation value of output sounds obtained by DNN-RL than the evaluation value of output sounds obtained by DNN-MMSE is low, obtains the Z ^RL It can be determined that the sound source enhancement performed for the purpose was wrong (at this time, R(G)<0). As described above, by providing the sound source emphasizing means called DNN-MMSE which is to be compared with the DNN-RL, it is possible to reduce the influence on the evaluation value from other than the performance of the sound source emphasizing. In addition, it is possible to learn the DNN parameters for sound source enhancement, which have higher evaluation values than the sound source enhancement based on the MMSE criterion.

In addition, the second term p(G|X, Θ _Μ ) of the reward coefficient _Rew is a product of probabilities and thus has a very small value (see equation (20)). In order to avoid underflow, the reward coefficient R _ew is _calculated by the following formula.

Here, β and γ are coefficients for avoiding underflow of p(G|X).

When the evaluation value of the output sound obtained by DNN-RL is lower than the evaluation value of the output sound obtained by DNN-MMSE (R(G)<0), DNN-MMSE is better than the time-frequency mask of DNN-RL. It is considered that the evaluation value becomes higher in the time frequency mask of. Therefore, the following processing is performed in order to increase the generation probability of the MMSE-based time-frequency mask.

<First embodiment>
Here, a sound source emphasis learning device configured based on the contents described in <Technical background> will be described.

Hereinafter, the sound source emphasis learning device 100 will be described with reference to FIGS. 3 to 4. FIG. 3 is a block diagram showing the configuration of the sound source emphasis learning device 100. FIG. 4 is a flowchart showing the operation of the sound source emphasis learning device 100. As shown in FIG. 3, the sound source emphasis learning device 100 includes a frequency domain signal generation unit 105, a DNN parameter initial value generation unit 110, a DNN-RL parameter generation unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the sound source emphasis learning device 100.

The sound source emphasis learning device 100 is connected to the target sound learning data recording unit 910 and the noise learning data recording unit 920. The target sound learning data recording unit 910 and the noise learning data recording unit 920 record the target sound and noise collected in advance as learning data. The target sound learning data and the noise learning data are time domain signals. For example, when a voice is the target sound, the target sound learning data is utterance data recorded in an anechoic room or the like. It is desirable that this utterance data collects utterances for about 8 seconds and more than about 5000 utterances. Further, the noise learning data is noise recorded in an environment in which it is supposed to be used.

Various parameters (for example, parameters used for DNN-MMSE learning, DNN-RL learning, etc.) used in each component of the sound source emphasis learning device 100 are input from the outside like the target sound learning data and the noise learning data. Alternatively, it may be set in each component in advance.

The operation of the sound source emphasis learning device 100 will be described with reference to FIG. The frequency domain signal generation unit 105 calculates the frequency domain target sound signal S _ω,τ , the frequency domain noise signal N _ω,τ , and the frequency domain observation signal X _ω,τ (ω∈{1 from the target sound learning data and the noise learning data. ,..., Ω}, τε{1,..., Τ}, Ω, Τ respectively generate integers of 1 or more determined by the target sound learning data and the noise learning data (S105). Specifically, first, one target sound learning data (in the above example, utterance data for about 8 seconds) is randomly selected, and noise learning data having the same length as the target sound learning data is randomly selected. Choose one. Further, a time domain observation signal is generated by superimposing the target sound learning data and the noise learning data at a random SNR (signal-to-noise ratio). The SNR range may be set to, for example, about -6 dB to 12 dB. Next, from these target sound learning data, noise learning data, and time domain observation signal, frequency domain target sound signal S _ω,τ , frequency domain noise signal N _ω,τ , frequency domain observation signal X _ω,τ ( _ω ∈ ( 1,...,Ω}, τ ∈ {1,...,Τ}) is generated. Short-time Fourier transform or the like may be used to generate these frequency domain signals.

The DNN parameter initial value generating unit 110 generates the frequency domain target sound signal S _ω,τ generated in S105, the frequency domain noise signal N _ω,τ , and the frequency domain observation signal X _ω,τ (ωε{1,...,Ω}. ,τε{1,...,Τ}), the initial value Θ ^MMSE _ini of the DNN- ^MMSE parameter and the initial value Θ ^RL _ini of the DNN-RL parameter are generated (S110). The DNN-RL parameter generation unit 120 uses the DNN-MMSE parameter initial value Θ ^MMSE _ini and the DNN-RL parameter initial value Θ ^RL _ini generated in S110 to generate the frequency domain target sound signal S _{ω, τ} , The DNN-RL parameter Θ ^RL is generated from the frequency domain observation signal X _ω,τ ( _ω ε{1,..., Ω}, τ ε{1,..., Τ}) (S120).

The process of S105 is appropriately executed as many times as necessary for the processes of S110 and S120 (DNN-MMSE learning and DNN-RL learning). Therefore, the process of S105 necessary for the process of S120 may be executed between S110 and S120 in FIG.

Hereinafter, the DNN parameter initial value generation unit 110 will be described with reference to FIGS. FIG. 5 is a block diagram showing the configuration of the DNN parameter initial value generation unit 110. FIG. 6 is a flowchart showing the operation of the DNN parameter initial value generation unit 110. As shown in FIG. 5, the DNN parameter initial value generation unit 110 includes a DNN-MMSE parameter initial value generation unit 111 and a DNN-RL parameter initial value generation unit 112.

The operation of the DNN parameter initial value generation unit 110 will be described with reference to FIG. The DNN-MMSE parameter initial value generation unit 111 generates the frequency domain target sound signal S _ω,τ generated in S105, the frequency domain noise signal N _ω,τ , and the frequency domain observation signal X _ω,τ ( _ω ∈ {1,..., Ω}, τε{1,..., Τ}), an initial value Θ ^MMSE _ini of the DNN-MMSE parameter is generated (S111). For example, Non-Patent Document 1 can be used to generate the initial value Θ ^MMSE _ini . Specifically, first, from the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ∈{1,...,Τ}), the input vector x of the DNN-MMSE is calculated by the equation (9). Generate _τ ( _τ ∈ {1,..., Τ}). Further, from the frequency-domain target sound signal S _ω,τ and the frequency-domain noise signal N _ω,τ , the time-frequency mask G ^IRM _ω,τ (ω∈{1,...,Ω},τ∈{1,... , Τ}) is generated.

This time-frequency mask G ^IRM _ω,τ becomes label data.

Next, the DNN-MMSE is learned using equations (42) to (44).

Specifically, first, for an input vector x _τ of DNN-MMSE, μ(x _τ ) ( _τ ε{1,..., Τ}) that is the output of DNN-MMSE is generated. Next, assuming that G _τ =μ(x _τ ), the back-propagation method is used to minimize the square error of the label data G ^IRM _ω,τ and G _ω,τ (=μ _ω,τ ), and DNN is used. -Learn the MMSE parameter Θ _M. The equations (42) to (44) that determine the structure of this DNN-MMSE are the equations (32) to (35) that determine the structure of the DNN-RL excluding the estimation of the variance vector of equation (33). equal.

An initialization method such as discriminative pre-training (Reference Non-Patent Document 3) can be used for this learning. An algorithm such as Adam (Reference Non-Patent Document 4) can be used to implement the error back propagation method.
(Reference Non-Patent Document 3: F.Seide, G.Li, X.Chen and D.Yu, “Feature engineering in context-dependent deep neural networks for conversational speech transcription”, In Proc. IEEE Automatic Speech Recognition and Understanding Workshop( ASRU), pp. 24-29, 2011.)
(Reference Non-Patent Document 4: D. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization”, In Proc. of the 3rd International Conference for Learning Representations(ICLR), pp.1-15, 2015.)

The DNN-MMSE parameter Θ _M at the end of learning is output as the DNN-MMSE parameter initial value Θ ^MMSE _ini . Since the DNN-MMSE parameter initial value Θ ^MMSE _ini is used in the process of the DNN-RL parameter generating unit 120, it is recorded in the recording unit 190.

The DNN-RL parameter initial value generation unit 112 generates the frequency domain target sound signal S _ω,τ generated in S105, the frequency domain noise signal N _ω,τ , and the frequency domain observation signal X _ω,τ ( _ω ε{1,..., Ω}, τε{1,..., Τ}), an initial value Θ ^RL _ini of the DNN-RL parameter is generated (S112). Specifically, first, from the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ε{1,...,Τ}) generated in S105, the DNN-RL is calculated by the equation (9). Generate an input vector x _τ ( _τ ∈ {1,..., Τ}) of.

Next, the DNN-RL is learned using equations (32) to (35). Specifically, first, for the input vector x _τ of the DNN-RL, the average vector μ(x _τ ) and the variance vector σ(x _τ ) ( _τ ∈ {1,...,Τ }) is generated. Next, the DNN-RL parameter Θ _M is learned by using the error backpropagation method so as to maximize the likelihood function as shown in Expression (45).

However,

Is.

Note that Adam can be used to implement the error back-propagation method as before.

The DNN-RL parameter Θ _M at the end of learning is output as the DNN-RL parameter initial value Θ ^RL _ini . Since the DNN-RL parameter initial value Θ ^RL _ini is used in the process of the DNN-RL parameter generation unit 120, it is recorded in the recording unit 190.

Hereinafter, the DNN-RL parameter generation unit 120 will be described with reference to FIGS. 7 to 8. FIG. 7 is a block diagram showing the configuration of the DNN-RL parameter generation unit 120. FIG. 8 is a flowchart showing the operation of the DNN-RL parameter generation unit 120. As shown in FIG. 7, the DNN-RL parameter generation unit 120 includes a DNN-RL time domain output signal generation unit 124, a DNN-MMSE time domain output signal generation unit 125, a reward coefficient calculation unit 126, and a DNN-RL parameter. The optimizing unit 127 and the convergence condition determining unit 128 are included.

The operation of the DNN-RL parameter generator 120 will be described with reference to FIG. The DNN-RL time-domain output signal generator 124 generates the frequency-domain target sound signal S _ω,τ generated in S105 and the frequency-domain observation signal X _ω,τ (ωε{1,...,Ω},τε{1, , Τ}), a DNN-RL time domain output signal is generated (S124).

Hereinafter, the DNN-RL time domain output signal generation unit 124 will be described with reference to FIGS. 9 to 10. FIG. 9 is a block diagram showing the configuration of the DNN-RL time domain output signal generation unit 124. FIG. 10 is a flowchart showing the operation of the DNN-RL time domain output signal generation unit 124. As shown in FIG. 9, the DNN-RL time domain output signal generation unit 124 includes a posterior probability distribution parameter generation unit 121, a time frequency mask generation unit 122, and a time frequency mask processing unit 123.

The posterior probability distribution parameter generation unit 121, the time-frequency mask generation unit 122, and the time-frequency mask processing unit 123 correspond to the non-linear mapping unit 912, the mask calculation unit 913, and the filtering unit 920 in the related art, respectively.

The operation of the DNN-RL time domain output signal generation unit 124 will be described with reference to FIG. The posterior probability distribution parameter generation unit 121 is a posterior probability distribution parameter from the frequency domain observation signal X _ω,τ (ω∈{1,...,Ω},τ∈{1,...,Τ}) generated in S105. An average vector μ(x _τ ) and a variance vector σ(x _τ ) (τε{1,..., Τ}) are generated (S121). Specifically, first, from the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ∈ {1,...,Τ}), the input vector x of the DNN-RL is calculated by the equation (9). Generate _τ ( _τ ∈ {1,..., Τ}).

Next, using the current DNN-RL parameter Θ _M , the average vector, which is the posterior probability distribution parameter, is calculated from the input vector x _τ ( _τ ∈ {1,…, Τ}) using Equations (32) to (35). Generate μ(x _τ ) and variance vector σ(x _τ ) ( _τ ∈ {1,..., Τ}). The DNN-RL parameter used in the first process of the posterior probability distribution parameter generation unit 121 is the DNN-RL parameter initial value Θ ^RL _ini .

The time-frequency mask generator 122 calculates the time-frequency mask G from the mean vector μ(x _τ ) and the variance vector σ(x _τ ) ( _τ ∈ {1,..., Τ}) that are posterior probability distribution parameters generated in S121. _τ (τε{1,..., Τ}) is generated (S122). Specifically, the time-frequency mask G _τ (τ∈{1,..., Τ}) is generated using the following ε-greedy algorithm.

Here, ~ in Expression (50) represents that a random number is generated from the probability distribution on the right side. The probability ε(0<ε<1) may be set to about 0.05, for example.

Of course, G _τ =μ(x _τ ) ( _τ ∈ {1,..., Τ}) may be simply used.

The time-frequency mask processing unit 123 uses the time-frequency mask G _τ ( _τ ∈ {1,..., Τ}) generated in S122 to observe the frequency domain observation signal X _ω,τ ( _ω ∈ {1,..., Ω}. , τ ∈ {1,..., T}), a DNN-RL time domain output signal is generated (S123). Specifically, using the time-frequency mask G _τ , the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ∈{1,...,Τ}) DNN-RL frequency domain output signal S^ _τ =(G _1,τ X _1,τ ,…, G _Ω,τ X _Ω,τ )( _τ ∈ {1,…,Τ}) is generated and inverse Fourier transform is generated. A DNN-RL time domain output signal is generated by transforming into a time domain waveform using.

The DNN-MMSE time domain output signal generation unit 125 uses the frequency domain observation signals X _ω,τ ( _ω ε{1,...,Ω}, τε{1,...,Τ}) generated in S105 to obtain the DNN-MMSE. A time domain output signal is generated (S125). Specifically, first, from the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ∈{1,...,Τ}), the input vector x of the DNN-MMSE is calculated by the equation (9). _τ ( _τ ∈ {1,…, Τ}) is generated, and using the DNN-MMSE parameter initial value Θ ^MMSE _ini , the average vector μ( which is the output of the DNN-MMSE is given by Equations (42) to (44). x _τ )( _τ ∈ {1,..., Τ}) is generated. Then, as the time-frequency mask _{G τ = μ (x τ)} , by using a time-frequency mask G _tau, the equation (2), the frequency domain observed signal _{X ω, τ (ω∈ {1} , ..., Ω}, τ ∈ {1,…, Τ}) generates DNN-MMSE frequency domain output signal S^ _τ =(G _{1, τ} X _{1, τ} ,…, G _{Ω, τ} X _{Ω, τ} ) A DNN-MMSE time domain output signal is generated by transforming into a time domain waveform using transformation or the like.

The reward coefficient calculation unit 126 uses the DNN-RL time domain output signal generated in S124 and the DNN-MMSE time domain output signal generated in S125 to generate the time-frequency mask G _τ ( _τ ∈ {1,...,Τ). }) is calculated (S126). Specifically, calculates an evaluation value Z ^MMSE evaluation value Z ^RL and DNN-MMSE time domain output signal of the DNN-RL time domain output signal, the comparison compensation using equation (36) calculates the formula (37) ~ Calculate the reward coefficient using equation (38). Each parameter used to calculate the reward coefficient is preferably tuned according to the evaluation value used to calculate the comparative reward. For example, when PESQ is used as the evaluation value, it can be set to about α=1.0, β=10.0, γ=0.01.

A reward coefficient is calculated for a set of I target sound learning data and noise learning data. That is, the processing from S124 to S126 is repeated I times. Here, I should be set to about 5.

The DNN-RL parameter optimizing unit 127 updates the DNN-RL parameter Θ _M so as to maximize the value of the objective function T _ar in Expression (26) (S127). The value of the objective function T _ar of the equation (26) is obtained by using the equations (27) to (31) and the input vector x _τ generated in the processing step of S121 and the average vector μ(x _τ ) generated in S121. It can be obtained from the variance vector σ(x _τ ), the reward coefficient calculated in S126. It should be noted that (i) and i appearing in the objective function T _ar of the equation (26) are indexes representing the number of repetitions. Further, similar to the DNN-RL parameter initial value generation unit 112, the DNN-RL parameter Θ _M is updated by the error back propagation method so as to be optimized. Note that Adam may be used for the error back propagation method.

The convergence condition determination unit 128 determines a convergence condition set in advance as a learning end condition, ends the process if the convergence condition is satisfied, and repeats the processes of S124 to S127 if the convergence condition is not satisfied ( S128). The DNN-RL parameter Θ _M at the end of learning is output as the DNN-RL parameter Θ ^RL . As the convergence condition, for example, a condition that the number of times the processes of S124 to S127 are executed reaches a predetermined number can be adopted. In this case, the predetermined number of times can be set to about 100,000 times.

According to the invention of this embodiment, by optimizing the DNN parameters using various evaluation values including non-differentiable evaluation values, the DNN for generating the time-frequency mask for sound source enhancement of the input sound is generated. Can learn. For example, if you want to optimize the sound source enhancement for speech recognition, by constructing the objective function with the binary value of whether the result of the speech recognition is correct or not as an evaluation value, the DNN in a form suitable for the sound source enhancement for speech recognition. It will be possible to optimize the parameters.

<Second embodiment>
Here, a sound source enhancement device using the DNN parameter generated by the sound source enhancement learning device of the first embodiment will be described.

Hereinafter, the sound source enhancement device 200 will be described with reference to FIGS. 11 to 12. FIG. 11 is a block diagram showing the configuration of the sound source enhancement device 200. FIG. 12 is a flowchart showing the operation of the sound source emphasizing device 200. As shown in FIG. 11, the sound source enhancement apparatus 200 includes a frequency domain observation signal generation unit 210, a posterior probability distribution parameter generation unit 121, a time frequency mask generation unit 122, a time frequency mask processing unit 123, and a recording unit 290. Including. The recording unit 290 is a component that appropriately records information necessary for the processing of the sound source enhancement device 200. For example, the DNN-RL parameter Θ ^RL generated by the sound source emphasis learning device 100 is recorded.

The operation of the sound source emphasizing device 200 will be described with reference to FIG. The frequency domain observation signal generation unit 210 calculates the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ε{1,...,Τ}, Ω, and Τ from the time domain observation signal, respectively) An integer greater than or equal to 1 determined by the region observation signal) is generated (S210). For example, using the short-time Fourier transform, the time-domain observation signal picked up by the microphone is transformed into the frequency domain, and the frequency-domain observation signal X _ω,τ (ω∈{1,...,Ω},τ∈{1, …, Τ}) is generated. The posterior probability distribution parameter generation unit 121 outputs the frequency domain observation signal X _ω,τ ( _ω ∈ {1,...,Ω}, τ ∈{1,...,Τ}) generated in S210 as an output of the DNN-RL. An average vector μ(x _τ ) and a variance vector σ(x _τ ) (τε{1,..., Τ}) which are posterior probability distribution parameters are generated (S121). At that time, the DNN-RL parameter Θ ^RL is used. The time-frequency mask generator 122 calculates the time-frequency mask G from the mean vector μ(x _τ ) and the variance vector σ(x _τ ) ( _τ ∈ {1,..., Τ}) that are posterior probability distribution parameters generated in S121. _τ (τε{1,..., Τ}) is generated (S122). The time-frequency mask processing unit 123 uses the time-frequency mask G _τ ( _τ ∈ {1,..., Τ}) generated in S122 to observe the frequency domain observation signal X _ω,τ ( _ω ∈ {1,..., Ω}. , τ ∈ {1,..., Τ}) to generate a time domain output signal (S123).

According to the invention of the present embodiment, the sound source enhancement can be performed by the time-frequency mask generated based on the DNN in which the optimized DNN parameters are set using various evaluation values including the evaluation values that cannot be differentiated. For example, it becomes possible to perform sound source enhancement using DNN parameters optimized in a form suitable for sound source enhancement for speech recognition. In addition, by adopting PESQ, which has a high correlation with subjective sound quality evaluation, as the evaluation value, sound source enhancement using DNN parameters generated by a criterion (objective function) suitable for sound information processing technology for sound quality evaluation. Is possible.

<Third embodiment>
In the first embodiment, the DNN-RL learning for sound source enhancement has been described. However, the framework described in <Technical Background>, that is, the learning (optimization) of the DNN-RL parameter Θ _Μ is expressed by Equation (15). The framework formulated by DNN-RL that outputs the posterior probability distribution p(G _τ |x _τ , Θ _Μ ) as described above can be generally applied to mask processing (filtering) of sounds.

Furthermore, the learning dealt with in the first embodiment is not limited to DNNs, but can be applied to more general neural networks.

Therefore, an embodiment relating to learning by a general neural network, which is not limited to sound source emphasis, will be described here. In the following, the neural network will be referred to as NN.

Hereinafter, the input sound mask processing learning device 300 will be described with reference to FIGS. 13 to 14. FIG. 13 is a block diagram showing the configuration of the input sound mask processing learning device 300. FIG. 14 is a flowchart showing the operation of the input sound mask processing learning device 300. As shown in FIG. 13, the input sound mask processing learning device 300 includes an input vector generation unit 305, a posterior probability distribution generation unit 310, a mask generation unit 320, a mask processing unit 330, a reward coefficient calculation unit 360, and parameters. It includes an optimization unit 370, a convergence condition determination unit 380, and a recording unit 390. The recording unit 390 is a component that appropriately records information necessary for the processing of the input sound mask processing learning apparatus 300.

The input sound mask processing learning device 300 is connected to the input sound learning data recording unit 930. The input sound learning data recording unit 930 records, as learning data, input sounds that have been collected in advance and are to be masked.

Various parameters used in each component of the input sound mask processing learning device 300 (for example, parameters used for learning NN) may be input from the outside similarly to the input sound, or may be input to each component in advance. It may be set.

It is also assumed that the mask processing is independent for each input sound and does not affect the processing of other input sounds, and the mask design for each input sound is performed independently of that of other input sounds. To do.

The operation of the input sound mask processing learning device 300 will be described with reference to FIG. The input vector generation unit 305 generates an input vector x _τ (τε{1,..., Τ}, Τ is an integer of 1 or more determined by the input sound) from the input sound to the NN (S305). Posterior probability distribution generating unit 310, using the parameters theta _Micromax the NN, the input vector _{x τ (τ∈ {1, ...} , Τ}) generated in S305 from an output of the NN, the input vector x _tau input Then, a posterior probability distribution p(G _τ |x _τ , Θ _Μ )( _τ ε{1,..., Τ}), which is the probability that the mask G _τ is generated, is generated (S310). Here, the posterior probability distribution p(G _τ |x _τ , Θ _Μ ) is expressed as in Expression (15).

Incidentally, NN is the parameter theta _Micromax used in the first process of the posterior probability distribution generating unit 310, for example, recorded in the recording unit 390, it is assumed to be given in advance.

The mask generation unit 320 uses the posterior probability distribution p(G _τ |x _τ , Θ _Μ ) ( _τ ε{1,..., Τ}) generated in S310 to use the mask G _τ (for masking the input vector x _τ ). τ ∈ {1,..., Τ}) is generated (S320). Specifically, the mask G _τ is _calculated by equation (16).

The mask processing unit 330 uses the mask G _τ ( _τ ∈ {1,..., Τ}) generated in S320 to generate an output sound from the input vector x _τ ( _τ ∈ {1,..., Τ}) ( S330). Specifically, the processing according to the processing content of the mask G _τ generated in S320 is performed on the input vector x _τ , and the output sound is generated.

The posterior probability distribution generation unit 310, the mask generation unit 320, and the mask processing unit 330 are collectively referred to as an output sound generation unit 340. The output sound generation unit 340 is a component unit corresponding to the DNN-RL time domain output signal generation unit 124 of the first embodiment, and generates an output sound from the input vector x _τ ( _τ ∈ {1,..., Τ}). To do.

The reward coefficient calculation unit 360 calculates the reward coefficient of the mask G _τ (τε{1,..., Τ}) from the output sound generated in S330 (S360). Specifically, the reward coefficient R _ew is calculated by the following equation based on the assumption of the mask processing and the input sound (see equation (26′) and equation (20)).

R(G) is the evaluation value of the output sound generated in S330. Further, Π _τ p(G _τ |x _τ , Θ _Μ ) is the probability that the mask G _τ is generated when the input vector x _τ is input. Generation probability p(G _τ |x _τ , Θ _Μ ) ( Since it is a product of _τ ∈ {1,..., Τ}), it indicates the certainty factor that is the certainty of the mask G _τ ( _τ ∈ {1,..., Τ}) generated when the input sound is input.

The evaluation value R(G) may not be differentiable by the parameter Θ _Μ of NN.

The reward coefficient is calculated for I (I is an integer of 1 or more) input sounds. That is, the processing from S305 to S360 is repeated I times.

The parameter optimizing unit 370 updates the parameter Θ _M of the NN so as to maximize the value of the objective function T _ar of Expression (26) (S370).

However, the letters i and (i) are variables that represent the i-th episode and are indexes that represent the number of repetitions.

The objective function T _{ar in} Eq. (26) is a function of the parameter Θ _Μ defined by using the reward coefficient and the posterior probability distribution p(G _τ |x _τ , Θ _Μ )( _τ ∈ {1,…, Τ}) And, specifically, the expression expressed using the reward coefficient and the posterior probability distribution p(G _τ |x _τ ,Θ _Μ )( _τ ∈ {1,...,Τ}) (here, Is the product of Σ _τ lnp(G _τ |x _τ , Θ _Μ )).

As an expression expressed using the posterior probability distribution p(G _τ |x _τ , Θ _Μ )( _τ ∈ {1,…, Τ}), when the evaluation value R(G) of the output sound is a positive value Fluctuates so that the value increases, and when the output sound evaluation value R(G) is a negative value, it fluctuates so that the value decreases, and the value when the confidence is relatively low. The fluctuation of is smaller than the fluctuation of the value when the certainty factor is relatively high.

The convergence condition determination unit 380 determines a convergence condition set in advance as a learning end condition, ends the process if the convergence condition is satisfied, and repeats the processes of S305 to S370 if the convergence condition is not satisfied ( S380). The NN parameter Θ _M at the end of learning is output as the NN parameter Θ ^NN . As the convergence condition, for example, a condition that the number of times the processes of S305 to S370 are executed reaches a predetermined number can be adopted.

According to the invention of the present embodiment, by optimizing the parameters of the NN using various evaluation values including non-differentiable evaluation values, an NN for generating a mask for masking an input sound is obtained. You can learn.

<Fourth Embodiment>
In the calculation of the reward coefficient in the first embodiment, the calculation is performed based on the comparison reward that also uses the evaluation value of the DNN-MMSE time domain output signal by the time frequency mask processing obtained using the DNN-MMSE parameter Θ ^MMSE _ini .

Therefore, here, an embodiment in which the reward coefficient is calculated using the comparative reward will be described.

The input sound mask processing learning device 301 will be described below with reference to FIGS. FIG. 15 is a block diagram showing the configuration of the input sound mask processing learning device 301. FIG. 16 is a flowchart showing the operation of the input sound mask processing learning device 301. As can be seen from FIG. 15, the input sound mask processing learning apparatus 301 includes the input sound mask only in that the comparison output sound generation unit 350 is further included and that the reward coefficient calculation unit 361 is included instead of the reward coefficient calculation unit 360. Different from the processing learning device 300. Further, as can be seen from FIG. 16, the operation of the input sound masking process learning device 301 differs from the input sound masking process learning device 300 only in that S350 and S361 are added instead of S360.

The processes of S350 and S361 will be described below. The comparative output sound generation unit 350 generates a comparative output sound from the input vector x _τ (τε{1,..., Τ}) generated in S305 (S350). Specifically, first, for the input vector x _τ , using the equations corresponding to the equations (6) to (8) in the case of DNN (that is, the equation for calculating the output of the neural network). , NN produces a mask G _τ as the output y ^ _τ . The parameters of the NN used in the first process of the comparative output sound generation unit 350 are assumed to be given in advance, for example, recorded in the recording unit 390.

Next, a comparison output sound is generated from the input vector x _τ ( _τ ∈ {1,..., Τ}) using the mask G _τ ( _τ ∈ {1,..., Τ}). Specifically, the processing corresponding to the processing content of the generated mask G _τ is performed on the input vector x _τ , and the comparative output sound is generated.

The reward coefficient calculation unit 361 calculates the reward coefficient of the mask G _τ (τ∈{1,..., Τ}) generated in S320 from the output sound generated in S330 and the comparative output sound generated in S350 (S361). .. Specifically, the evaluation value of the output sound and the evaluation value of the comparative output sound are calculated, the comparative reward is calculated using Expression (36), and the reward coefficient is calculated using Expression (26″).

According to the invention of the present embodiment, by optimizing the parameters of the NN using various evaluation values including non-differentiable evaluation values, an NN for generating a mask for masking an input sound is obtained. You can learn.

(Modification)
In the third and fourth embodiments, the learning of the NN targeted for the processing by the mask (filter) for the input sound has been described, but more generally, the learning of the NN for the processing by the predetermined processing function for the input data. The following describes an example in which the framework described in <Technical background> is applied.

The input data processing function learning device 400 will be described below with reference to FIGS. 17 to 18. FIG. 17 is a block diagram showing the configuration of the input data processing function learning device 400. FIG. 18 is a flowchart showing the operation of the input data processing function learning device 400. As illustrated in FIG. 17, the input data processing function learning device 400 includes an input vector generation unit 405, a posterior probability distribution generation unit 410, a processing function generation unit 420, a processing function application unit 430, and a reward coefficient calculation unit 460. A parameter optimization unit 470, a convergence condition determination unit 480, and a recording unit 490 are included.

The input data processing function learning device 400 is connected to the input data recording unit 940. The input sound data recording unit 940 records input data to be processed by a predetermined processing function.

Various parameters used in each component of the input data processing function learning device 400 (for example, parameters used for learning NN) may be input from the outside like the input data, or may be input to each component in advance. It may be set.

Further, the processing by the processing function is independent for each input data, does not affect the processing of other input data, and the design of the processing function for each input data is performed independently of that of other input data. Suppose.

The operation of the input data processing function learning device 400 will be described with reference to FIG. The input vector generation unit 405 generates an input vector x _τ ( _τ ε {1,..., Τ}, Τ is an integer of 1 or more determined by the input data) from the input data to the NN (S405). The posterior probability distribution generation unit 410 determines the processing function G _τ when the input vector x _τ , which is the output of the NN, is input from the input vector x _τ ( _τ ∈ {1,..., Τ}) generated in S405. A posterior probability distribution p(G _τ |x _τ , Θ _Μ ) ( _τ ε{1,..., Τ}) that is the generated probability is generated (S410). Processing function generation unit 420, the posterior probability distribution p generated in _{S410 (G τ | x τ,} Θ Μ) (τ∈ {1, ..., Τ}) from use in the process of the input vector x _tau processing function G _tau (Τε{1,...,Τ}) is generated (S420). The processing function application unit 430 generates output data from the input vector x _τ (τ∈{1,..., Τ}) using the processing function G _τ (τ∈{1,..., Τ}) generated in S420. Yes (S430). The reward coefficient calculation unit 460 calculates the reward coefficient of the processing function G _τ (τε{1,..., Τ}) from the output data generated in S430 (S460). The reward coefficient is calculated for I (I is an integer of 1 or more) input data. That is, the processing from S405 to S460 is repeated I times. The parameter optimizing unit 470 updates the NN parameter Θ _M so as to maximize the value of the objective function T _ar of Expression (26) (S470). The convergence condition determination unit 480 determines a convergence condition set in advance as a learning end condition, ends the process if the convergence condition is satisfied, and repeats the processes of S405 to S470 if the convergence condition is not satisfied ( S480). The NN parameter Θ _M at the end of learning is output as the NN parameter Θ ^NN .

That is, the processing of S405 to S480 may be the same as the processing of S305 to S380.

According to the invention of this embodiment, the NN for generating the processing function for processing the input data is optimized by optimizing the parameters of the NN using various evaluation values including the non-differentiable evaluation value. You can learn.

<Additional notes>
The device of the present invention is, for example, as a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating with the outside of the hardware entity. Connectable communication unit, CPU (Central Processing Unit, cache memory and registers may be provided), RAM or ROM that is memory, external storage device that is a hard disk, and their input unit, output unit, and communication unit , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged among external storage devices. Further, if necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. As a physical entity provided with such hardware resources, there is a general-purpose computer or the like.

The external storage device of the hardware entity stores a program necessary to realize the above-described functions and data necessary for the processing of this program (not limited to the external storage device, for example, the program is read). It may be stored in a ROM that is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in the RAM, the external storage device, or the like.

In the hardware entity, each program stored in an external storage device (or ROM, etc.) and data necessary for the processing of each program are read into the memory as necessary, and interpreted and executed/processed by the CPU as appropriate. .. As a result, the CPU realizes a predetermined function (each constituent element represented by the above,... Unit,... Means, etc.).

The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. Further, the processes described in the above-described embodiments are not only executed in time series in the order described, but may be executed in parallel or individually according to the processing capability of the device that executes the processes or as necessary. ..

As described above, when the processing functions of the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on the computer, the processing functions of the hardware entity are realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disc. Memory), CD-R (Recordable)/RW (ReWritable), etc. as a magneto-optical recording medium, MO (Magneto-Optical disc) etc., and semiconductor memory EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) etc. Can be used.

The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, the program may be stored in a storage device of a server computer and transferred from the server computer to another computer via a network to distribute the program.

A computer that executes such a program temporarily stores, for example, the program recorded on a portable recording medium or the program transferred from the server computer in its own storage device. Then, when executing the process, this computer reads the program stored in its own recording medium and executes the process according to the read program. As another execution form of this program, a computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to this computer. Each time, the processing according to the received program may be sequentially executed. Further, the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the execution instruction and the result acquisition without transferring the program from the server computer to the computer. May be It should be noted that the program in this embodiment includes information that is used for processing by an electronic computer and that is equivalent to the program (data that is not a direct command to a computer, but has the property of defining computer processing).

Further, in this embodiment, the hardware entity is configured by executing a predetermined program on the computer, but at least a part of these processing contents may be implemented by hardware.

Claims (10)

After modeling the generation probability that the mask G _τ ( _τ ∈ {1,…, Τ}) is generated when the input vector x _τ ( _τ ∈ {1,…, Τ}) based on the input sound is input N masks from the input vector x _N based on N input sounds (N is an integer between 1 and τ) based on the probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,…, Τ}) A mask generation unit that generates G _N ,
Using the mask G _N , from the N input sounds, a mask processing unit that generates N output sounds by masking the N input sounds,
A reward coefficient acquisition unit for obtaining the reward coefficient of the mask G _N for the N output sounds,
A mask G _N is generated when the input vector x _N based on the reward coefficient and the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) is input. An input sound mask process including an updating unit for updating the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) using the probability q(G _N |x _N ). A learning device,
The input sound mask processing learning device, wherein the reward coefficient is determined from an evaluation value of the output sound and a certainty factor that is the certainty of the mask G _N generated when the input sound is input. The input sound mask processing learning device according to claim 1,
The reward coefficient is a product of the evaluation value of the output sound and the certainty factor,
The update unit uses the product of the reward coefficient and the generation probability q(G _N |x _N ) to calculate the posterior probability distribution p(G _τ |x _τ )( _τ ∈ {1,...,Τ}. ) Is an input sound mask processing learning device. The input sound mask processing learning device according to claim 1 or 2, wherein
The generation probability q(G _N |x _N ) is
When the evaluation value of the output sound is a positive value, the value fluctuates so as to increase,
When the evaluation value of the output sound is a negative value, it fluctuates so that the value becomes smaller,
The input sound mask processing learning device, wherein the variation in the value when the certainty factor is relatively low is smaller than the variation in the value when the certainty factor is relatively high. The input sound mask processing learning device according to any one of claims 1 to 3,
The generated probability q(G _N |x _N ) is the sum of logarithms of the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,...,Τ}). Mask processing learning device. The input sound mask processing learning device according to claim 1,
The posterior probability distribution _{p (G τ | x τ)} (τ∈ {1, ..., Τ}) , using the parameters _{Θ Μ, p (G τ |} x τ, Θ Μ) (τ∈ {1, ..., Τ}),
The input sound mask processing learning device, wherein the evaluation value cannot be differentiated by a parameter Θ _Μ . The input sound mask processing learning device according to any one of claims 1 to 5,
further,
And a comparative output sound generation unit that generates N comparative output sounds from the N input sounds,
The input sound mask processing learning device, wherein the reward coefficient is determined from a difference between an evaluation value of the output sound and an evaluation value of the comparative output sound and the certainty factor. We modeled the generation probability that the processing function G _τ ( _τ ∈ {1,…, Τ}) is generated when the input vector x _τ ( _τ ∈ {1,…, Τ}) based on the input data is input. Based on the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,…, Τ}), input vector x _N based on N input data (N is an integer between 1 and τ) A processing function generator that generates a processing function G _N ,
Using the processing function G _N , from the N input data, a processing function application unit that generates N output data obtained by processing the N input data by a processing function,
A reward coefficient acquisition unit for obtaining a reward coefficient of the processing function G _N for the N output data,
A processing function G _N is generated when the input vector x _N based on the reward coefficient and the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) is input. Input data processing including an updating unit for updating the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) using the generation probability q(G _N |x _N ). A function learning device,
The input data processing function learning device is characterized in that the reward coefficient is determined from an evaluation value of the output data and a certainty factor that is the certainty of the processing function G _N generated when the input data is input. The input sound mask processing learning device generates a mask G _τ ( _τ ∈ {1,..., Τ}) when the input vector x _τ ( _τ ∈ {1,..., Τ}) based on the input sound is input. Based on the posterior probability distribution p(G _τ |x _τ )( _τ ∈ {1,…, Τ}) that models the generation probability, the input based on N input sounds (N is an integer between 1 and τ inclusive) A mask generation step for generating _N masks G _N from the vector x _N ,
The input sound mask processing learning device, using the mask G _N , from the N input sounds, a mask processing step of generating N output sounds by masking the N input sounds,
The input sound mask processing learning device, for the N output sounds, a reward coefficient acquisition step of obtaining a reward coefficient of the mask G _N ,
When the input sound mask processing learning device receives the reward coefficient and the input vector x _N based on the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) Updating the posterior probability distribution p(G _τ |x _τ )( _τ ∈ {1,...,Τ}) with the generation probability q(G _N |x _N ) that the mask G _N is generated in An input sound mask processing learning method including an updating step,
The input sound mask processing learning method, wherein the reward coefficient is determined from an evaluation value of the output sound and a certainty factor that is the certainty of the mask G _N generated when the input sound is input. The input data processing function learning device generates a processing function G _τ ( _τ ∈ {1,..., Τ}) when an input vector x _τ ( _τ ∈ {1,..., Τ}) based on the input data is input. Based on the posterior probability distribution p(G _τ |x _τ )( _τ ∈ {1,…, Τ}) that models the generated probability, based on N input data (N is an integer from 1 to τ) A processing function generation step for generating _N processing functions G _N from the input vector x _N ,
The input data processing function learning device, using the processing function G _N , a processing function applying step of generating N output data obtained by processing the N input data by a processing function from the N input data. When,
The input data processing function learning device, to said N output data, and rewards coefficient acquisition step of obtaining a compensation coefficient of the processing function G _N,
When the input data processing function learning device receives the reward coefficient and the input vector x _N based on the posterior probability distribution p(G _τ |x _τ ) ( _τ ∈ {1,..., Τ}) The posterior probability distribution p(G _τ |x _τ )( _τ ∈ {1,…, Τ}) is updated by using the generation probability q(G _N |x _N ) that the processing function G _N is generated in And an input data processing function learning method including
The input data processing function learning method is characterized in that the reward coefficient is determined from an evaluation value of the output data and a certainty factor that is the certainty of the processing function G _N generated when the input data is input. A program for causing a computer to function as the input sound mask processing learning device according to any one of claims 1 to 6 or the input data processing function learning device according to claim 7.

Images