JP2020122855A - Estimation device, method thereof and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an estimation device or the like for restoring a consistent phase spectrum from only an amplitude spectrum by utilizing the statistical property of a signal to be restored. An estimation device (i) converts a complex spectrogram whose phase and amplitude conflict with each other into a time waveform, and converts the converted time waveform into a complex spectrogram whose phase and amplitude do not conflict, and (ii) amplitude. Is converted to the magnitude of the amplitude spectrogram A of the desired acoustic signal, and (iii) the phase spectrogram is brought closer to the desired acoustic signal based on the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal. It has an estimation unit that estimates a phase spectrogram that brings the amplitude spectrogram A closer to a desired acoustic signal by associating with and. [Selection diagram] Fig. 1

Description

The present invention relates to an estimation device that estimates and restores a phase spectrum from only an amplitude spectrum, its method, and a program.

The STFT (short-time Fourier transform) spectrum is a complex number, and in order to recover the time signal from the STFT spectrogram, both (1) amplitude spectrogram and (2) phase spectrogram are necessary. However, since the phase spectrum is difficult to handle, it is possible to estimate or control only the amplitude spectrum in speech synthesis or speech enhancement, and substitute the minimum phase or the observed phase for the phase spectrum and convert it back into a time signal. Many. Since the amplitude spectrogram and the phase spectrogram are not independent variables, if one is controlled, the other must be the corresponding variable. Therefore, in speech synthesis or speech enhancement, the quality of the output sound may deteriorate due to the contradiction between the amplitude and the phase.

Non-Patent Document 1 is known as a technique for estimating a phase spectrogram that is consistent with the amplitude spectrogram. The technique of Non-Patent Document 1 (called a Griffin-Lim algorithm) is a technique of estimating a consistent phase spectrogram from the amplitude spectrogram A by repeating the following procedure.

Where X is a complex spectrogram with amplitude A, G and G ^† are short-time Fourier transform (STFT) and inverse STFT,

|•| represents the absolute value calculation for each element. This method is equivalent to solving the following optimization problem.

Where ||・|| ² _Fro represents the Frobenius norm. B is a set of spectrograms whose amplitude is A. As described above, since the phase spectrum is substituted with the minimum phase or the observed phase, when the complex spectrogram X is subjected to STFT and inverse STFT of equation (1), it does not return to the original complex spectrogram X. Therefore, the amplitude is fixed to the given amplitude spectrogram A by the equation (2), and the phase is obtained by the equation (3) so as to obtain a correct short-time Fourier transform expression.

D. Griffin and J. Lim, "Signal estimation from modied shorttime Fourier transform", IEEE Trans. Acoust., Speech, Signal Process., vol. 32, no. 2, pp. 236-243, Apr. 1984.

However, the method of Non-Patent Document 1 is adaptable to all acoustic signals, but requires a huge number of repetitions. This is because no assumption is made about the statistical properties of the signal to be restored (hereinafter also referred to as the desired acoustic signal) within the optimization framework.

It is an object of the present invention to provide an estimating apparatus, a method, and a program for restoring a consistent phase spectrum from only an amplitude spectrum by using the statistical property of a signal to be restored.

In order to solve the above problems, according to one aspect of the present invention, the estimation apparatus converts (i) a complex spectrogram in which the phase and the amplitude are inconsistent into a time waveform, and the converted time waveform has a phase and an amplitude. The process of converting into a complex spectrogram that does not contradict, (ii) the process of converting the amplitude into the magnitude of the amplitude spectrogram A of the desired acoustic signal, and (iii) the statistical analysis of the learning acoustic signal corresponding to the desired acoustic signal. An estimation unit that estimates the phase spectrogram that brings the amplitude spectrogram A closer to the desired acoustic signal by associating with the processing that brings the phase spectrogram closer to the desired acoustic signal based on the property.

In order to solve the above-mentioned problems, according to another aspect of the present invention, the estimation apparatus adds the phase of the complex spectrogram X to the amplitude spectrogram A of the desired acoustic signal, and the phase addition for obtaining the signal Y after the addition. And a transform unit that transforms the signal Y into a time waveform by the inverse short-time Fourier transform, and transforms the transformed time waveform into a signal Z in the frequency domain by the short-time Fourier transform corresponding to the inverse short-time Fourier transform, and a complex Using the spectrogram X and the signal Y and the signal Z, based on the statistical properties of the acoustic signal for learning corresponding to the desired acoustic signal, a phase changing unit that brings the phase of the complex spectrogram X closer to the phase of the desired acoustic signal. ,including.

Advantageous Effects of Invention According to the present invention, it is possible to restore a consistent phase spectrum from only an amplitude spectrum with a smaller amount of calculation than that in the conventional technique by using the statistical property of a signal to be restored.

The functional block diagram of the estimation apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the estimation apparatus which concerns on 1st embodiment. The functional block diagram of the estimation part which concerns on 1st embodiment. The functional block diagram of the learning device which concerns on 1st embodiment. The figure which shows the example of the processing flow of the learning apparatus which concerns on 1st embodiment.

Hereinafter, embodiments of the present invention will be described. In the drawings used in the following description, components having the same function and steps for performing the same process are denoted by the same reference numerals, and duplicate description will be omitted. In the following description, the symbol "^" or the like used in the text should be written directly above the character immediately after it, but due to the limitation of the text notation, it is written immediately before the character. In the formula, these symbols are written in their original positions. Unless otherwise specified, the processing performed for each element of a vector or matrix shall be applied to all the elements of the vector or matrix.

<Points of the first embodiment>
In this embodiment, deep learning is incorporated into the method of Non-Patent Document 1. Note that there is a system such as Reference 1 for phase restoration using deep learning.
(Reference 1) K. Oyamada, H. Kameoka, K. Tanaka T. Kaneko, N. Hojo, and H. Ando, "Generative adversarial network-based approach to signal reconstruction from magnitude spectrograms", in Eur. Signal Process. Conf. (EUSIPCO), Sept. 2018.

The difference between these methods and the present embodiment is that the reference document 1 restores the phase by end-to-end if the large-scale neural network is used, whereas the present embodiment repeats the non-patent document 1. By using DNN (Deep Neural Network) as a part of optimization, the number of parameters required for learning is reduced.

Further, since the number of repetitions is directly connected to the stacking (deepening) of the neural network, unlike the conventional neural network, the network shapes do not need to match during learning and testing. It is also characterized by having scalability in terms of the trade-off between processing time and restoration accuracy, according to the requirements of computer power and accuracy during practical use.

As described above, in this embodiment, deep learning is incorporated in the Griffin-Lim algorithm. For example, using the DNN trained with the training data, the statistical properties of the signal to be restored are incorporated into the Griffin-Lim algorithm. FIG. 1 is a functional block diagram of the estimation device 100 according to the first embodiment, and FIG. 2 shows an example of the processing flow. The estimation device 100 includes M estimation units 110-m (m=0, 1, 2,..., M-1, and M is any integer of 1 or more). FIG. 3 shows a functional block diagram of the estimation unit 110-m. The estimation unit 110-m includes a phase assignment unit 111 corresponding to the equation (2) and a conversion unit 112 corresponding to the equation (1), and further statistically evaluates a learning acoustic signal corresponding to a desired acoustic signal. It includes a phase changing unit 113 that brings the phase of the complex spectrogram X closer to the phase of the desired acoustic signal based on the property.

With the configurations of FIGS. 1 and 3, by performing the processing by the DNN after the Griffin-Lim algorithm is repeated once, the consistent phase estimation in consideration of the statistical property of the signal to be restored is realized. This is equivalent to stacking the internal DNN for the number of repetitions (M). In other words, the DNN scale during processing can be changed by controlling the number of repetitions (M) of this processing block. For example, when the number of DNN layers in the DNN unit 113-1 is 3, when M=1, 2, 3,..., As a whole, the DNN functions as a DNN composed of 3, 6, 9,. Reducing the number of iterations is equivalent to using a shallow DNN, which reduces processing performance but enables high-speed computation. On the other hand, increasing the number of repetitions is equivalent to using a deep DNN, and the processing speed becomes slower, but a high quality output sound can be obtained.

The condition of the DNN used here is based on the statistical properties of the signal to be restored (it may be learned in some way from the learning data of the signal to be restored), and the phase of the output sound of the Griffin-Lim algorithm is the signal to be restored. Any process may be used as long as it is close to. As an example, the following residual learning is shown as an embodiment.
Y ^[m] =P _B (X ^[m] ) (4)
Z ^[m] =P _C (Y ^[m] )(5)
X[m+1]=E(X ^[m] ) (6)
=Z ^[m] -F _θ (X ^[m] ,Y ^[m] ,Z ^[m] ) (7)
Where F _θ is a DNN implemented somehow. In other words, the DNN learned based on the statistical properties of the signal to be restored removes (subtracts) the distortion and estimation error caused by the Griffin-Lim algorithm. Here, the DNN does not directly estimate the signal to be restored, but estimates the component that is not the signal to be restored. The DNN can be learned by, for example, minimizing the following objective function.

Here, X ^* is a true complex spectrogram, ~X=X ^* +N, N is complex Gaussian noise, ~Y=P _B (~X), ~Z=P _C (~Y). However, since the Griffin-Lim algorithm is a process for restoring only the phase spectrum, the amplitude of ~Y is made to match the amplitude of X ^* .

This embodiment includes a DNN learning step and a phase spectrum estimation step. First, the learning stage will be described.
<Learning device according to the first embodiment>
FIG. 4 is a functional block diagram of the learning device 200 of this embodiment, and FIG. 5 shows an example of its processing flow.

The learning apparatus 200 includes learning data of a signal to be restored (clean acoustic signal X ^(L)*, which is represented by a complex spectrogram ⁾ , an amplitude spectrogram A ^(L) corresponding to the clean acoustic signal X ^(L)* , and noise. Input N and parameters required for various optimizations and output a trained DNN.

The learning device 200 includes a noise adding unit 209, a phase adding unit 211, a converting unit 212, a DNN unit 213, a subtracting unit 214, and a parameter updating unit 215.

For example, the learning device 200 initializes the parameter θ of the DNN used in the DNN unit 213 by using a random number in an initialization unit (not shown) (S208).

<Noise addition unit 209>
Noise addition section 209 receives as input a clean audio signal X ^{(L) *} and the noise N, by adding the noise N to clean the audio signal ^{X (L) * (S209)} , the complex spectrogram ~ X ⁽⁼ X ^{(L) *} +N) is calculated and output.

<Phase imparting unit 211>
The phase adding unit 211 receives the complex spectrogram ~X and the amplitude spectrogram A ^(L) corresponding to the clean acoustic signal X ^(L)* as input, and the complex spectrogram ~ ^(L) is added to the amplitude spectrogram A ^(L) as shown in the following equation. The phase of X is applied (S211), and the signal ~Y=P _B (~X) after application is determined and output.

In addition,

Corresponds to the process of extracting the phase of the complex spectrogram to X, and Eq. (12) corresponds to the process of adding the extracted phase of the complex spectrogram to X to the amplitude spectrogram A ^(L) . Note that in the equation (12), each element of the complex spectrogram ~X is multiplied by each element of the amplitude spectrogram A ^(L) , and the product is divided by the amplitude spectrogram |~X| of the complex spectrogram ~X. Therefore, it may be said that the amplitude of the complex spectrogram ~X is converted into the magnitude of the amplitude spectrogram A ^(L) .

<Conversion unit 212>
Conversion unit 212 inputs the signals ~ Y, the following equation to convert the signals ~ Y inverse short time Fourier transform G ^† by the time waveform, the corresponding converted time waveform inverse short time Fourier transform G ^† A short-time Fourier transform G is used to convert the signal in the frequency domain to ~Z=P _c (~Y) (S212) and output.

<DNN unit 213>
The DNN unit 213 receives the initial value of the parameter θ or the parameter θ updated by the parameter updating unit 215 to be described later, the complex spectrogram ~X, the signal ~Y, and the signal ~Z as input, and the DNN uses the Griffin-Lim. The distortion or estimation error generated by the algorithm is estimated (S213) and the estimated value F _θ (~X,~Y,~Z) is output.

<Subtraction unit 214>
The subtraction unit 214 receives the signals ~Z and the clean acoustic signal X ^(L)* as input, calculates a difference (S214), and outputs the calculated difference (complex spectrogram ~ZX ^(L)* ).

<Parameter updating unit 215>
The parameter updating unit 215 inputs the difference (complex spectrogram ~ZX ^(L)* ) and the estimated value F _θ (~X,~Y,~Z), and using these values,

The parameter θ of the DNN is updated so that (S215-1). The learning method may be a stochastic steepest descent method or the like, and the learning rate may be set to about 10 ^-5 . Further, the parameter updating unit 215 determines whether or not a predetermined condition is satisfied (S215-2), and when the predetermined condition is satisfied, outputs the DNN at that time point as a learned DNN. If the predetermined condition is not satisfied, the updated parameter θ is output to the DNN unit 213, and the new clean acoustic signal X ^(L)* , the new noise N, and the updated parameter θ are used in S209. ~S215-1 is repeated. It should be noted that learning is repeated a certain number of times (for example, 100,000 times) as the predetermined condition. Etc. can be used.

Through the above processing, the learning stage of DNN is realized. Next, the phase spectrum estimation step will be described.
<Estimation device 100>
As described above, FIG. 1 shows a functional block diagram of the estimation device 100 of this embodiment, and FIG. 2 shows an example of the processing flow thereof.

The estimation apparatus 100 receives the amplitude spectrogram A and the complex spectrogram X ^{[0] in} which the phase and the amplitude are inconsistent, and obtains and outputs the complex spectrogram Y ^[M] having the phase spectrogram that is not inconsistent with the amplitude spectrogram A. Here, the amplitude of the complex spectrogram X ^[0] is the amplitude spectrogram A.

The estimation device 100 includes M estimation units 110-m and a phase assignment unit 120 (see FIG. 1 ).

<Estimation unit 110-m>
The estimation unit 110-m receives the amplitude spectrogram A of the desired acoustic signal and the complex spectrogram X ^{[m] in} which the phase and the amplitude are inconsistent, and obtains a complex spectrogram X ^[m+1] having the estimated phase spectrogram. ,Output. For example, the estimation unit 110-m performs (i) a process of converting a complex spectrogram in which the phase and the amplitude are inconsistent into a time waveform, and a process of converting the converted time waveform into a complex spectrogram in which the phase and the amplitude are not inconsistent, and (ii) The phase spectrogram is brought closer to the desired acoustic signal based on the process of converting the amplitude into the magnitude of the amplitude spectrogram A of the desired acoustic signal and (iii) the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal. By associating with the processing, the phase spectrogram that brings the amplitude spectrogram A closer to the desired acoustic signal is estimated (S110).

FIG. 3 shows a functional block diagram of the estimation unit 110-m. The estimating unit 110-m includes a phase adding unit 111, a converting unit 112, and a phase changing unit 113, and the phase changing unit 113 includes a DNN unit 113-1 and a subtracting unit 113-2.

The DNN learned by the learning device 200 is set in the DNN unit 113-1 of the phase changing unit 113 of each estimation unit 110-m. As described above, since the number of iterations is directly linked to the stacking (deepening) of the neural network, unlike conventional neural networks, there is no need for the network shapes to match during learning and testing. Instead, you only have to learn one DNN. In addition, at the time of estimation, the number of iterations (M) can be controlled according to the requirements of computer power and accuracy, and scalability can be achieved with respect to the trade-off between processing time and restoration accuracy. For example, M=5 may be executed.

<Phase imparting unit 111>
The phase providing unit 111 receives the amplitude spectrogram A of the desired acoustic signal and the complex spectrogram X ^{[m] in} which the phase and the amplitude are inconsistent, and as shown in the following equation, the complex spectrogram X ^{[m] is} added to the amplitude spectrogram A. Is added (S111), and the signal Y ^[m] =P _B (X ^[m] ) after the addition is obtained and output.

In addition,

Corresponds to the process of extracting the phase of the complex spectrogram X ^[m] , and Eq. (21) corresponds to the process of adding the extracted phase of the complex spectrogram X ^[m] to the amplitude spectrogram A. In equation (21), each element of the complex spectrogram X ^[m] is multiplied by each element of the amplitude spectrogram A, and the product is divided by the amplitude spectrogram |X ^[m] | of the complex spectrogram X ^[m]. Therefore, it can be said that the amplitude of the complex spectrogram X ^[m] is converted into the magnitude of the amplitude spectrogram A.

<Conversion unit 112>
Conversion unit 112 inputs the signal Y ^[m], the following equation, the signal Y is converted into time waveforms by inverse short time Fourier transform G ^† a ^[m], briefly transformed time waveform inverse Fourier transform G The signal is converted into a frequency domain signal Z ^[m] =P _c (Y ^[m] ) by the short-time Fourier transform G corresponding to ^† (S112) and output.

This process corresponds to a process of converting the complex spectrogram Y ^[m] in which the phase and the amplitude are inconsistent into a time waveform, and the converted time waveform into a complex spectrogram Z ^[m] in which the phase and the amplitude are inconsistent.

<Phase changing unit 113>
The phase changing unit 113 uses the complex spectrogram X ^[m] , the signal Y ^[m], and the signal Z ^[m] based on the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal. The phase of X ^[m] is brought close to the phase of the desired acoustic signal (S113), and the approximated signal X ^[m+1] is output. For example, the phase changing unit 113 realizes this processing by the following DNN unit 113-1 and subtracting unit 113-2.

<DNN unit 113-1>
The DNN unit 113-1 receives the complex spectrogram X ^[m] , the signal Y ^[m], and the signal Z ^[m] as input, and uses the DNN based on the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal. , The distortion or estimation error (Z ^[m] -X ^[m] ) generated by the Griffin-Lim algorithm is estimated (S113-1), and the estimated value F _θ (X ^[m] ,Y ^[m] ,Z ^{[m ]} ) is output. The estimated value F _θ (X ^[m] ,Y ^[m] ,Z ^[m] ) is a complex spectrogram. For example, F _θ (X ^[m] ,Y ^[m] ,Z ^[m] ), the phase spectrogram can be obtained.

Therefore, it can be said that the process for obtaining the complex spectrogram F _θ (X ^[m] , Y ^[m] , Z ^[m] ) and the process for obtaining the phase spectrogram are equivalent.

<Subtraction unit 113-2>
The subtraction unit 113-2 receives the signal Z ^[m] and the estimated value F _θ (X ^[m] , Y ^[m] , Z ^[m] ) as input, calculates a difference (S113-2), and calculates the calculated difference. (Complex spectrogram X ^[m+1] =Z ^[m] -F _θ (X ^[m] ,Y ^[m] ,Z ^[m] )) is output. This subtraction corresponds to the process of removing the distortion or estimation error caused by the Griffin-Lim algorithm, and the phase spectrogram of the signal Z ^[m] (which may be called the corresponding complex spectrogram X ^[m] ) is desired. This is equivalent to the processing to bring the sound signal into

The estimation unit 110-m approximates the amplitude spectrogram A to the desired acoustic signal as a whole, and this is equivalent to the process of estimating the phase spectrogram that approximates the amplitude spectrogram A to the desired acoustic signal.

The above-described processes S111 to S113-2 are repeated M times by the number of estimation units 110-m, and the estimation unit 110-(M-1) obtains and outputs a complex spectrogram X ^[M] .

<Phase imparting unit 120>
Phase deposition unit 120 inputs the complex spectrogram X ^[M], as shown in the following equation, the phase of the complex spectrogram X ^[M] is given to the amplitude spectrogram A (S120), the signal after applying Y ^[M] =P _B (X ^[M] ) is output.

By this processing, the amplitude of the complex spectrogram X ^[M] is converted into the magnitude of the amplitude spectrogram A again.

<Effect>
With the above configuration, it is possible to restore a consistent phase spectrum from only the amplitude spectrum with a smaller amount of calculation than the conventional technique by using the statistical property of the signal to be restored.

<Modification>
In the present embodiment, the complex spectrogram X ^{[0] in} which the phase and the amplitude are contradictory is given as an input, but only the amplitude spectrogram A is input, and an appropriate phase spectrogram (initial value) is randomly generated for the amplitude spectrogram A. Alternatively, the initial value may be a complex spectrogram X ^[0] .

In the present embodiment, the noise addition unit 209 is provided in order to construct a DNN that is resistant to noise. However, without providing the noise addition unit 209, the clean acoustic signal X ^(L)* is directly converted into the complex spectrogram ~X(= X ^(L)* ) may be used.

In the present embodiment, an example of residual learning is shown, but a process of bringing the phase of the output signal of the Griffin-Lim algorithm closer to the signal to be restored may be included based on the statistical property of the signal to be restored.

<Other modifications>
The present invention is not limited to the above embodiments and modifications. For example, the above-described various processes may be executed not only in time series according to the description but also in parallel or individually according to the processing capability of the device that executes the process or the need. Other changes can be made as appropriate without departing from the spirit of the present invention.

<Hardware configuration>
The learning device 200 and the estimation device 100 are configured by loading a special program into a known or dedicated computer having, for example, a central processing unit (CPU) and a main memory (RAM: Random Access Memory). It is a special device. The learning device 200 and the estimation device 100 execute each process under the control of the central processing unit, for example. The data input to the learning device 200 and the estimation device 100 and the data obtained by each process are stored in, for example, the main storage device, and the data stored in the main storage device are read to the central processing unit as necessary. Issued and used for other processing. At least a part of each processing unit of the learning device 200 and the estimation device 100 may be configured by hardware such as an integrated circuit. Each storage unit included in the learning device 200 and the estimation device 100 can be configured by, for example, a main storage device such as a RAM (Random Access Memory) or middleware such as a relational database or a key-value store. However, each storage unit does not necessarily have to be provided inside the learning device 200 and the estimation device 100, and is configured by an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory (Flash Memory). The learning device 200 and the estimation device 100 may be provided outside the device.

<Program and recording medium>
Further, various processing functions in each device described in the above-described embodiment and modification may be realized by a computer. In that case, the processing content of the function that each device should have is described by the program. By executing this program on a computer, various processing functions of the above-mentioned devices are realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like.

The distribution of this program is performed by selling, transferring, or lending a portable recording medium such as a DVD or a CD-ROM in which the program is recorded. Further, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

A computer that executes such a program first stores, for example, the program recorded in a portable recording medium or the program transferred from the server computer in its own storage unit. Then, when executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. Further, as another embodiment of this program, the computer may directly read the program from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer to this computer, the processing according to the received program may be sequentially executed. Further, the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by executing the execution instruction and the result acquisition without transferring the program from the server computer to the computer. May be The program includes information used for processing by an electronic computer and equivalent to the program (data that is not a direct command to a computer but has the property of defining processing of a computer).

Further, although each device is configured by executing a predetermined program on a computer, at least a part of the processing contents may be realized by hardware.

Claims (7)

(i) A process for converting a complex spectrogram in which the phase and amplitude are inconsistent into a time waveform, and converting the converted time waveform into a complex spectrogram in which the phase and amplitude are inconsistent, and (ii) the amplitude spectrogram of the desired acoustic signal. By associating the process of converting to the magnitude of A and (iii) the process of bringing the phase spectrogram closer to the desired acoustic signal based on the statistical property of the learning acoustic signal corresponding to the desired acoustic signal. , Having an estimator for estimating a phase spectrogram that brings the amplitude spectrogram A closer to the desired acoustic signal,
Estimator. A phase imparting unit that imparts the phase of the complex spectrogram X to the amplitude spectrogram A of the desired acoustic signal and obtains the signal Y after imparting,
The signal Y is converted into a time waveform by an inverse short-time Fourier transform, and the converted time waveform is converted into a signal Z in the frequency domain by a short-time Fourier transform corresponding to the inverse short-time Fourier transform,
Using the complex spectrogram X, the signal Y and the signal Z, based on the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal, the phase of the complex spectrogram X is the phase of the desired acoustic signal. And a phase changing unit that brings the
Estimator. The acoustic signal restoration device according to claim 2,
The statistical properties of the acoustic signal for learning are expressed by a deep neural network,
The deep neural network is
A complex spectrogram X ^(L)* obtained from the acoustic signal for learning and its amplitude spectrogram A ^(L) are used for learning,
Inputting the complex spectrogram X, the signal Y, and the signal Z, and outputting an estimated value of the residual difference between the signal Z and the complex spectrogram X,
Estimator. (i) A process for converting a complex spectrogram in which the phase and amplitude are inconsistent into a time waveform, and converting the converted time waveform into a complex spectrogram in which the phase and amplitude are inconsistent, and (ii) the amplitude spectrogram of the desired acoustic signal. By associating the process of converting to the magnitude of A and (iii) the process of bringing the phase spectrogram closer to the desired acoustic signal based on the statistical property of the learning acoustic signal corresponding to the desired acoustic signal. , An estimation step of estimating a phase spectrogram approximating the amplitude spectrogram A to the desired acoustic signal,
Estimation method. A phase adding step of adding the phase of the complex spectrogram X to the amplitude spectrogram A of the desired acoustic signal and obtaining the signal Y after the addition,
A conversion step of converting the signal Y into a time waveform by an inverse short-time Fourier transform, and converting the converted time waveform into a signal Z in the frequency domain by a short-time Fourier transform corresponding to the inverse short-time Fourier transform,
Using the complex spectrogram X, the signal Y and the signal Z, based on the statistical properties of the learning acoustic signal corresponding to the desired acoustic signal, the phase of the complex spectrogram X is the phase of the desired acoustic signal. And a phase changing step of
Estimation method. The acoustic signal restoration method according to claim 5, wherein
The statistical properties of the acoustic signal for learning are expressed by a deep neural network,
The deep neural network is
A complex spectrogram X ^(L)* obtained from the acoustic signal for learning and its amplitude spectrogram A ^(L) are used for learning,
Inputting the complex spectrogram X, the signal Y, and the signal Z, and outputting an estimated value of the residual difference between the signal Z and the complex spectrogram X,
Estimation method. A program for causing a computer to function as the estimation device according to any one of claims 1 to 3.

Images