KR102556098B1 - Method and apparatus of audio signal encoding using weighted error function based on psychoacoustics, and audio signal decoding using weighted error function based on psychoacoustics - Google Patents

Abstract

According to an embodiment, by using a perceptually weighted error function using human hearing characteristics, improved audio signal quality is provided with the same model complexity, or the same level of audio signal is provided with a low model complexity. The quality of the audio signal can be provided.
According to an embodiment, in a neural network applied to an audio signal encoding method using an audio signal encoding apparatus, generating a masking threshold for a first audio signal before being learned; calculating a weight matrix to be applied to each frequency component of the first audio signal based on the masking threshold; generating a weighted error function by correcting a preset error function using the weighting matrix; and generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal.

Description

Audio signal encoding method and apparatus using psychoacoustic based weighted error function, and audio signal decoding method and apparatus }

The following embodiments relate to a method and apparatus for encoding an audio signal using a weighted error function based on psychoacoustics, and a method and apparatus for decoding an audio signal, and more specifically, a method and apparatus for decoding an audio signal using a weighted error function considering human hearing characteristics. It relates to an audio signal encoding method and apparatus applying a parameter, and an audio signal decoding method and apparatus.

Recently, voice and audio codecs applied to various purposes and applications are being developed by standardization organizations such as ITU-T, MPEG, and 3GPP. Most audio codecs are based on psychoacoustic models using various auditory characteristics of humans. In addition, voice codecs are mainly based on voice generation models, but at the same time, human cognitive characteristics are also utilized to improve subjective quality.

As such, conventional speech and audio codecs use a method based on human hearing characteristics to effectively control quantization noise generated in the coding step.

According to an embodiment, an improved audio signal quality may be provided with the same complexity of the model by using a perceptually weighted error function using human hearing characteristics.

According to an embodiment, it is possible to provide the same level of audio signal quality with a low model complexity by using a perceptually weighted error function using human hearing characteristics.

According to one aspect, in a neural network applied to an audio signal encoding method using an audio signal encoding apparatus, generating a masking threshold for a first audio signal before being learned; calculating a weight matrix to be applied to each frequency component of the first audio signal based on the masking threshold; generating a weighted error function by correcting a preset error function using the weighting matrix; and generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal.

The weighting matrix includes weights to be applied to frequency components of the first audio signal, and the weights are inversely proportional to a masking threshold for the first audio signal and proportional to the size of each frequency component of the first audio signal. It may be a neural network configured to

The neural network may further include performing perceptual quality evaluation by comparing the generated second audio signal with the first audio signal.

The perceptual quality evaluation is an objective evaluation of PESQ (Perceptual Evaluation of Speech Quality), POLQA (Perceptual Objective Listening Quality Assessment), PEAQ (Perceptual Evaluation of Audio Quality), MOS (Mean Opinion Score), MUSHRA (Multiple Stimuli with Hidden It may be a neural network including subjective evaluation of Reference and Anchor).

It may be a neural network that determines whether or not the topology included in the model can be adjusted based on the perceptual quality evaluation.

When determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy the preset quality requirements, parameters are re-learned using a model with increased complexity, and the result of the perceptual quality evaluation If t satisfies preset quality requirements, it may be a neural network that relearns parameters using a model whose complexity is reduced within the quality requirements.

According to one aspect, an audio signal encoding method performed by an audio signal encoding apparatus using a neural network, comprising: receiving an input audio signal; generating a dimensionally reduced latent vector of the input audio signal based on parameters of the hidden layer, wherein the neural network includes one or more hidden layers; It may be an audio signal encoding method comprising the step of encoding the generated latent vector and outputting a bitstream.

The generating of the dimensionally reduced latent vector of the input audio signal based on the parameters of the hidden layer learned using the neural network may include adjusting the topology of the model including the number of hidden layers and nodes. audio that generates the latent vector based on the learned parameter when it is impossible or not necessary, or generates the latent vector based on the relearned parameter by applying the adjusted topology when the topology of the model can be adjusted It may be a signal encoding method.

Coding of the latent vector may be a method of encoding an audio signal in which the bitstream is binarized in order to transmit the bitstream through a channel.

According to one aspect, in a neural network applied to an audio signal decoding method using an audio signal decoding apparatus, generating a masking threshold for a first audio signal before being learned; calculating a weight matrix to be applied to each frequency component of the first audio signal based on the masking threshold; generating a weighted error function by correcting a preset error function using the weighting matrix; and generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal.

The weighting matrix includes weights to be applied to frequency components of the first audio signal, and the weights are inversely proportional to a masking threshold for the first audio signal and proportional to the size of each frequency component of the first audio signal. It may be a neural network configured to

The neural network may further include performing perceptual quality evaluation by comparing the generated second audio signal with the first audio signal.

The perceptual quality evaluation is an objective evaluation of PESQ (Perceptual Evaluation of Speech Quality), POLQA (Perceptual Objective Listening Quality Assessment), PEAQ (Perceptual Evaluation of Audio Quality), MOS (Mean Opinion Score), MUSHRA (Multiple Stimuli with Hidden It may be a neural network including subjective evaluation of Reference and Anchor).

It may be a neural network that determines whether or not the topology included in the model can be adjusted based on the perceptual quality evaluation.

When determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy the preset quality requirements, parameters are re-learned using a model with increased complexity, and the result of the perceptual quality evaluation [0041] If [[tau]([tau]) satisfies preset quality requirements, it may be a neural network that relearns parameters using a model whose complexity is reduced within the quality requirements.

According to one aspect, in an audio signal decoding method performed by an audio signal decoding apparatus to which a neural network is applied, a latent vector generated by applying a parameter learned through the neural network to an input audio signal is an encoded bit. receiving a stream; restoring the latent vector from the received bitstream; The neural network may include one or more hidden layers, and the audio signal decoding method may include decoding an output audio signal from the reconstructed latent vector using the hidden layer to which the learned parameter is applied.

The generated latent vector is generated based on the learned parameter when topology adjustment including the number of hidden layers and the number of nodes belonging to the hidden layer is not possible or necessary, and the topology If it is possible to adjust the latent vector based on the parameter relearned by applying the adjusted topology, the latent vector may be an audio signal decoding method.

The encoding of the latent vector may be a method of decoding an audio signal in which binarization is performed to transmit the bitstream through a channel.

According to one aspect, in an audio signal encoding apparatus to which a neural network is applied, the audio signal encoding apparatus includes a processor and a memory including one or more instructions executable by the processor, and the one or more instructions are executed by the processor. , an input audio signal is received, the neural network includes one or more hidden layers, and a dimensionally reduced latent vector of the input audio signal is generated based on parameters of the hidden layer learned using the neural network. and outputting a bitstream by encoding the generated latent vector.

According to one aspect, in an audio signal decoding apparatus to which a neural network is applied, the audio signal decoding apparatus includes a processor and a memory including one or more instructions executable by the processor, and the one or more instructions are executed by the processor. , a bitstream in which a latent vector generated by applying the parameter learned through the neural network to an input audio signal is quantized, the latent vector is restored from the received bitstream, and the neural network may include one or more hidden layers, and may be an audio signal decoding apparatus that decodes an output audio signal from the reconstructed latent vector using the hidden layer to which the learned parameter is applied.

According to an embodiment, an improved audio signal quality may be provided with the same complexity of the model by using a perceptually weighted error function using human hearing characteristics.

According to an embodiment, it is possible to provide the same level of audio signal quality with a low model complexity by using a perceptually weighted error function using human hearing characteristics.

1 is a diagram showing the structure of an autoencoder including three hidden layers according to an embodiment.
2 may represent a simultaneous masking effect according to an embodiment.
3 is a diagram illustrating audible and non-audible regions in consideration of a masking effect, according to an exemplary embodiment.
4 is a diagram illustrating a learning process of a neural network using a weighted error function according to an embodiment.
5 is a diagram illustrating an audio signal encoding apparatus, a channel, and an audio signal decoding apparatus according to an exemplary embodiment.
6 is a diagram illustrating a neural network learning process applied to an audio signal encoding method using an audio signal encoding apparatus according to an exemplary embodiment.
7 is a diagram illustrating an audio signal encoding method performed by an audio signal encoding apparatus to which a neural network is applied, according to an exemplary embodiment.
8 is a diagram illustrating a neural network learning process applied to an audio signal decoding method using an audio signal decoding apparatus according to an exemplary embodiment.
9 is a diagram illustrating an audio signal decoding method performed by an audio signal decoding apparatus to which a neural network is applied, according to an exemplary embodiment.
10 is a diagram illustrating an audio signal encoding apparatus to which a neural network is applied, according to an exemplary embodiment.
11 is a diagram illustrating an audio signal decoding apparatus to which a neural network is applied according to an exemplary embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be modified and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing the structure of an autoencoder including three hidden layers according to an embodiment.

According to an embodiment, audio signal encoding and decoding may be performed by applying a neural network. In this case, the neural network may include various models of deep learning/machine learning. Specifically, the neural network may include an autoencoding method as a deep learning model.

Neural networks can solve optimization problems to minimize an error function or cost function. Here, optimization is performed by an iterative learning algorithm, and a parameter minimizing an error can be found through optimization.

For example, input data to which a neural network is applied may predict output data as shown in Equation 1 below. Here, N input data of D dimension , N output data of R dimension can mean, parameter may represent a model of a neural network applied to input data.

At this time, the predicted output data with target data The difference in may represent an error, and the error function may be defined as in Equation 2 below. Here, target data may be N target data of R dimension.

here, class may represent the n-th column vector of target data and output data. Therefore, as shown in Equation 3 below, the total error can be calculated for N pieces of data.

Thus, the parameters can be adjusted to minimize the total error, and parameters that minimize the total error can be found by an iterative learning algorithm. There may exist various models of neural networks that perform iterative learning algorithms. Among them, the autoencoding method can be effective in learning latent patterns from input data.

Hereinafter, a neural network to which an autoencoding method is applied according to an embodiment will be described. However, it is not limited to the autoencoding method, and models to which other neural networks are applied may also be included.

According to an embodiment, the autoencoder may have the same size as the target data Y and the input data X, so that dimensionality reduction may be effective. When an autoencoder includes L hidden layers, a fully-connected deep autoencoder can be recursively defined as in Equation 4 below.

here, class are respectively Weighting and bias for the th layer may be indicated. At this time, the parameter Is can represent

1 is a diagram illustrating an autoencoder including three hidden layers according to an embodiment. Here, the bottom layer 110 represents an input layer into which data is input, the top layer 120 represents an output layer into which data is output, and the middle layer represents a hidden layer 130 can represent In addition, the node is included in the input layer/output layer/hidden layer, and among them, the node 140 may represent a biased node.

According to an embodiment, an autoencoder may be a compression system including an encoder including an input layer and hidden layers 1 and 2 and a decoder including a hidden layer 3 and an output layer. In this case, hidden layer 2 may be a code layer for compressing input data.

The encoding part of Autoencoder, which is a compression system, If , the code layer is dimensionally reduced. can create In addition, using the error function set as in Equation 5 below for the target data Y and the input data X, the decoding unit of the autoencoder The input data can be restored from

For an effective code layer, the number of hidden layers and the number of nodes included in the hidden layer may be increased. At this time, a trade-off may be made between the complexity of the autoencoder and the quality of the output data. Therefore, when the complexity of the autoencoder is high, problems of battery consumption and memory capacity may occur.

According to another embodiment, the autoencoder may be a denoising autoencoder that generates a clean signal from which noise is removed using a noisy signal. Noise may include additive noise, reverberation, and band-pass filtering.

A denoising autoencoder that generates a clean signal from a noisy signal can be expressed as Equation 6 below. In this case, Y may be the amplitude spectrum of the original signal, and X may be the amplitude spectrum of the modified signal. where X is the transformation function by can be expressed as In Equation 6, is an approximation of the inverse function of the transformation function, can represent

At this time, for example, if the original signal is deformed due to the additional noise, as shown in Equation 7 below, the denoising autoencoder is an ideal mask for removing the additional noise rather than directly estimating the original signal. It can be effective to learn to estimate .

Here, the ideal mask can be used to remove additional noise of the transformed signal using the Hadamard product as shown in Equation 8 below.

At this time, may be an estimated ideal ratio mask, and may be expressed as in Equation 9 below. Here, Q may be a magnitude spectrum for the additive noise, and when the original signal is transformed by the additive noise, the transformation function may be defined as in Equation 10.

The denoising Autoencoder is a function using a structure similar to Equation 4. can learn However, the transformation function Since is very complex, a large number of hidden layers and nodes may be required. Thus, a trade-off between model complexity and performance may occur.

Therefore, in order to solve the trade-off, an autoencoding method based on an error function using human hearing characteristics will be described in detail below. By applying an error function using hearing characteristics, model complexity can be reduced or autoencoder performance can be improved.

2 may represent a simultaneous masking effect according to an embodiment.

Graph 210 shows the audible sound pressure level in decibels (db) versus the frequency of an audio signal in a quiet environment. For example, the lowest db is shown in a frequency band of 4 kHz, and the threshold of the audible music level may increase in a low or high frequency band. More specifically, most people cannot perceive a large tonal signal of about 30 db at 30 Hz, but can perceive a relatively small tonal signal of about 10 db at 1 kHz.

Graph 230 may represent a graph modified by a tonal signal present at 1 kHz. That is, the tonal signal can cause the graph 210 to rise at 1 kHz. Accordingly, a signal having a smaller magnitude than the graph 230 cannot be recognized by humans.

A masker 220 may represent a tonal signal at 1 kHz, and a maskee may represent a signal masked by the masker. For example, the maski may include a signal having a smaller size than the graph 230 .

3 is a diagram illustrating audible and non-audible regions in consideration of a masking effect, according to an exemplary embodiment.

FIG. 3 shows the overlapping of the graph of FIG. 2 and the spectrum of an input audio signal. Here, the audible region 340 may indicate that the spectrum 360 of the input audio signal has a spectrum greater than a masking threshold curve at a corresponding frequency. Also, the inaudible region 350 may indicate that the spectrum 360 of the input audio signal has a spectrum smaller than a masking threshold curve at a corresponding frequency.

For example, since the graph 310 is larger than the graph 360 at 30 Hz, it may be a non-audible area 350, and at 10 kHz, the graph 310 may be larger than the graph 360, so it may be a non-audible area 350. , and since the graph 360 is smaller than the graph 310 at 4 kHz, it may be an audible region 340.

According to an embodiment, an error function may be modified to consider an audible region more than a non-audible region according to characteristics of an input audio signal in training of an autoencoder. That is, when the magnitude of a specific frequency component of the input audio signal is smaller than the corresponding masking threshold, humans may not be relatively sensitive to errors. Alternatively, when the magnitude of a specific frequency component of the input audio signal is greater than the corresponding masking threshold, humans may be relatively more sensitive to errors.

Therefore, for the training of the autoencoder considering human hearing characteristics, the corrected error function can be expressed as Equation 11 below. In this case, the corrected error function may represent a weighted error function obtained by correcting a preset error function.

Here, H represents a weighting matrix, and the weighting matrix may include weights applied to each frequency component of the input audio signal. For example, of the second sample If the th coefficient has a large masking threshold, the corresponding weight may be relatively small. For another example, of the second sample If the th coefficient has a small masking threshold, the corresponding weight can be relatively large.

4 is a diagram illustrating a learning process of a neural network using a weighted error function according to an embodiment.

According to an embodiment, a frequency spectrum of the first audio signal may be obtained for an analysis frame having a preset length through the time-frequency analysis 401 . In this case, the frequency spectrum may be obtained by applying a filter bank or a modified discrete cosine transform (MDCT) to the first audio signal, or may be obtained by applying other methods.

Here, the first audio signal may represent a training audio signal for parameter learning. In addition, the analysis frame having a preset length may include an analysis frame having a length of 10 ms to 50 ms of the first audio signal, and may also include analysis frames having other lengths.

Therefore, through the time-frequency analysis 401, the frequency spectrum Y of the first audio signal or the frequency spectrum of the first audio signal modified by noise can be created.

According to an embodiment, a masking threshold for the first audio signal may be calculated through psychoacoustic analysis 402 . That is, the masking threshold may be calculated by analyzing the acoustic characteristics of the first audio signal.

The psychoacoustic analysis method may include MPEG PAM-I or MPEG PAM-II, and may include psychoacoustic analysis by other methods. For example, temporal masking can be used as well as concurrent masking effects.

According to an embodiment, a weight matrix to be applied to each frequency component of the first audio signal may be calculated (403) using the masking threshold calculated through the psychoacoustic analysis (402).

In this case, the weighting matrix may include weights to be applied to the frequency components of the first audio signal, and the weights may be set in inverse proportion to a masking threshold for the first audio signal and proportional to the size of each frequency component of the first audio signal. That is, the weight may be determined by a signal-to-mask ratio (SMR), and the SMR may represent a ratio between the magnitude of each frequency component of the first audio signal and the magnitude of the masking threshold.

For example, each frequency component of the first audio signal the weights of the weighting matrix for may be set to be inversely proportional to the corresponding masking threshold and proportional to the size of the frequency component.

According to an embodiment, when the masking threshold is expressed in decibels, each weight in the weighting matrix may be expressed in a relationship as shown in Equation 12 below. However, according to an embodiment, linear scale conversion or log scale conversion between the masking threshold and weights may be included, and other scale conversions may also be included.

According to one embodiment, a preset error function may generate a weighted error function via the error weighting function 404. Here, the weighted error function expressed as in Equation 11 may be used for learning model parameters by applying weights.

In this case, the model may include a neural network, and may include, for example, an autoencoder. Also, the model may include a topology, and the topology of the model may include an input layer, one or more hidden layers, an output layer, and nodes included in each layer.

According to one embodiment, the parameters of the model Learning 405 may be learned using a weighted error function. For example, learning may be performed on the topology of an initial model, and an audio spectrum Z predicted using the topology of a model on which learning is completed may be output.

In this case, the predicted audio spectrum Z is the frequency spectrum Y of the first audio signal or the frequency spectrum of the first audio signal modified by noise. It can be generated by applying the parameters of the model learned for

According to one embodiment, the perceptual quality assessment 406 compares the predicted audio spectrum to the first audio signal. Alternatively, quality evaluation may be performed by comparing with the frequency spectrum Y of the first audio signal.

At this time, the perceptual quality evaluation is based on the objective evaluation of PESQ (Perceptual Evaluation of Speech Quality), POLQA (Perceptual Objective Listening Quality Assessment), PEAQ (Perceptual Evaluation of Audio Quality), MOS (Mean Opinion Score), MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) subjective assessments can be used. Perceptual quality assessment is not limited thereto and may include other quality assessments.

Perceptual quality assessment 406 may represent a quality assessment for the currently learned model. Based on the quality evaluation, it may be determined whether or not model requirements such as preset quality and complexity of the model are satisfied. Also, based on the quality evaluation, it may be determined whether the model topology, such as model complexity, can be adjusted. Here, the complexity of the model may have a positive correlation with the number of hidden layers and nodes.

According to an embodiment, when it is determined that the topology of the model is not adjustable or does not need to be adjusted (407), parameters of the currently learned model may be stored and learning may be terminated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when it is determined that the topology of the model can be adjusted (407), the topology of the model may be updated (408), and the above-described process for learning may be repeated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the above-described process for learning may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the above-described process for learning may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

According to one embodiment, signal processing may be performed on the input audio signal based on the parameters of the learned model. In this case, signal processing may include compression/noise removal/encoding/decoding, but is not limited thereto.

5 is a diagram illustrating an audio signal encoding apparatus, a channel, and an audio signal decoding apparatus according to an exemplary embodiment.

According to an embodiment, the audio signal encoding apparatus 510 to which a neural network is applied may receive an input audio signal. In this case, the audio signal encoding apparatus 510 may apply a model learned through a neural network. Here, the neural network may include an autoencoder.

The audio signal encoding apparatus 510 may include an autoencoder encoding unit 511 and a quantization unit 512. Here, the autoencoder encoding unit 511 may include layers from the input layer to the code layer, in which case the code layer It may indicate a second hidden layer.

The audio signal encoding apparatus 510 may generate a dimensionally reduced latent vector of the received input audio signal based on the parameters of the hidden layer learned using the neural network. At this time, the generated latent vector may be quantized or coded by the quantization unit 512 and output as a bitstream. Here, the quantization unit 512 may include a process of binarizing the bitstream in order to transmit it through a transmission channel.

The output bitstream may be transmitted to the audio signal decoding apparatus 520 through the transmission channel 530 .

The audio signal decoding apparatus 520 may include an autoencoder decoding unit 521 and an inverse quantization unit 522. Here, the autoencoder decoding unit 521 Layers from the th hidden layer to the output layer may be included.

The audio signal decoding apparatus 520 may inversely quantize or inverse-code the bitstream transmitted through the transmission channel 530 in the inverse quantization unit 522 to restore the latent vector. Using the restored latent vector, the autoencoder decoding unit 521 can decode the output audio signal or the autoencoder decoding unit 521 can calculate the output audio signal. Here, the inverse quantization unit 522 may include a process of binarizing the bitstream.

Parameters of the autoencoder encoding unit 511 and the autoencoder decoding unit 521 may be parameters learned from FIG. 4 . For details on the learned parameters, refer to FIG. 4 .

According to an embodiment, an improved audio signal quality may be provided with the same complexity of the model by using a perceptually weighted error function using human hearing characteristics. Alternatively, the same level of audio signal quality may be provided with a low model complexity. Therefore, it can be used in an audio codec for compressing and restoring an audio signal.

6 is a diagram illustrating a neural network learning process applied to an audio signal encoding method using an audio signal encoding apparatus according to an exemplary embodiment.

In step 601, the audio signal encoding apparatus may generate a masking threshold for the first audio signal before being learned. Here, the first audio signal may represent a training audio signal for learning the neural network.

The masking threshold may be calculated by analyzing the acoustic characteristics of the first audio signal. The psychoacoustic analysis method may include MPEG PAM-I or MPEG PAM-II, and may include psychoacoustic analysis by other methods. For example, temporal masking can be used as well as concurrent masking effects.

In step 602, the audio signal encoding apparatus may calculate a weighting matrix to be applied to each frequency component of the first audio signal based on the generated masking threshold. In this case, the weighting matrix may include weights to be applied to the frequency components of the first audio signal, and the weights may be set in inverse proportion to a masking threshold for the first audio signal and proportional to the size of each frequency component of the first audio signal. That is, the weight may be determined by a signal-to-mask ratio (SMR), and the SMR may represent a ratio between the magnitude of each frequency component of the first audio signal and the magnitude of the masking threshold.

For example, each frequency component of the first audio signal the weights of the weighting matrix for may be set to be inversely proportional to the corresponding masking threshold and proportional to the size of the frequency component.

In step 603, the audio signal encoding apparatus may generate a weighted error function obtained by correcting a preset error function using a weighting matrix. Here, the weighted error function may be used to learn model parameters by applying weights.

In this case, the model may include a neural network, and for example, the neural network may include an autoencoder. Also, the model may include a topology, and the topology of the model may include an input layer, one or more hidden layers, an output layer, and nodes included in each layer.

In step 604, the audio signal encoding apparatus may generate a second audio signal by applying the learned parameter to the first audio signal using the weighted error function. Here, the parameters of the model may be learned using a weighted error function. For example, learning may be performed on the topology of an initial model, and an audio signal predicted using the topology of a model whose learning has been completed by an iterative learning algorithm may be output. Here, the predicted audio signal may represent the second audio signal.

In this case, the frequency spectrum of the second audio signal may be generated by applying the learned model parameter to the frequency spectrum of the first audio signal or the frequency spectrum of the first audio signal modified by noise.

By comparing the second audio signal with the first audio signal, a perceptual quality evaluation may be performed. At this time, the perceptual quality evaluation is based on the objective evaluation of PESQ (Perceptual Evaluation of Speech Quality), POLQA (Perceptual Objective Listening Quality Assessment), PEAQ (Perceptual Evaluation of Audio Quality), MOS (Mean Opinion Score), MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) subjective assessments can be used. Perceptual quality assessment is not limited thereto and may include other quality assessments.

The perceptual quality evaluation may indicate a quality evaluation of the currently learned model. Based on the quality evaluation, it may be determined whether or not model requirements such as preset quality and complexity of the model are satisfied. Also, based on the quality evaluation, it may be determined whether the model topology, such as model complexity, can be adjusted. Here, the complexity of the model may have a positive correlation with the number of hidden layers and nodes.

According to an embodiment, when it is determined that the topology of the model is not adjustable or does not need to be adjusted, parameters of the currently learned model may be stored and learning may be terminated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when it is determined that the topology of the model can be adjusted, the topology of the model can be updated, and the above-described process for learning can be repeated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the above-described process for learning may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the above-described process for learning may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

According to an embodiment, signal processing may be performed on the first audio signal based on the parameters of the learned model. In this case, signal processing may include compression/noise removal/encoding/decoding, but is not limited thereto.

7 is a diagram illustrating an audio signal encoding method performed by an audio signal encoding apparatus to which a neural network is applied, according to an exemplary embodiment.

In step 701, the audio signal encoding apparatus may receive an input audio signal. In this case, the audio signal encoding apparatus may apply a model learned through a neural network. Here, the neural network may include an autoencoder.

In step 702, the audio signal encoding apparatus may generate a dimensionally reduced latent vector of the input audio signal based on the parameters of the hidden layer learned using the neural network. Here, the process of learning the parameters of the hidden layer has been described in detail above.

According to an embodiment, latent vectors may be generated based on learned parameters when it is impossible or not necessary to adjust the topology of a model including the number of hidden layers and nodes. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when the topology of a model including the number of hidden layers and the number of nodes can be adjusted, the latent vector may be generated based on the relearned parameter by applying the adjusted topology. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the process for learning parameters may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the process for learning parameters may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

In step 703, the audio signal encoding apparatus may output a bitstream by encoding the generated latent vector. Here, the latent vector may be output as a bitstream after being quantized or coded to be transmitted through a transmission channel.

8 is a diagram illustrating a neural network learning process applied to an audio signal decoding method using an audio signal decoding apparatus according to an exemplary embodiment.

In step 801, the audio signal decoding apparatus may generate a masking threshold for the first audio signal before being learned. Here, the first audio signal may represent a training audio signal for learning the neural network.

The masking threshold may be calculated by analyzing the acoustic characteristics of the first audio signal. The psychoacoustic analysis method may include MPEG PAM-I or MPEG PAM-II, and may include psychoacoustic analysis by other methods. For example, temporal masking can be used as well as concurrent masking effects.

In step 802, the audio signal decoding apparatus may calculate a weighting matrix to be applied to each frequency component of the first audio signal based on the generated masking threshold. In this case, the weighting matrix may include weights to be applied to the frequency components of the first audio signal, and the weights may be set in inverse proportion to a masking threshold for the first audio signal and proportional to the size of each frequency component of the first audio signal. That is, the weight may be determined by a signal-to-mask ratio (SMR), and the SMR may represent a ratio between the magnitude of each frequency component of the first audio signal and the magnitude of the masking threshold.

For example, each frequency component of the first audio signal the weights of the weighting matrix for may be set to be inversely proportional to the corresponding masking threshold and proportional to the size of the frequency component.

In step 803, the audio signal decoding apparatus may generate a weighted error function by correcting a preset error function using a weighting matrix. Here, the weighted error function may be used to learn model parameters by applying weights.

In this case, the model may include a neural network, and for example, the neural network may include an autoencoder. Also, the model may include a topology, and the topology of the model may include an input layer, one or more hidden layers, an output layer, and nodes included in each layer.

In step 804, the audio signal decoding apparatus may generate a second audio signal by applying the learned parameter to the first audio signal using the weighted error function. Here, the parameters of the model may be learned using a weighted error function. For example, learning may be performed on the topology of an initial model, and an audio signal predicted using the topology of a model whose learning has been completed by an iterative learning algorithm may be output. Here, the predicted audio signal may represent the second audio signal.

In this case, the frequency spectrum of the second audio signal may be generated by applying the learned model parameter to the frequency spectrum of the first audio signal or the frequency spectrum of the first audio signal modified by noise.

By comparing the second audio signal with the first audio signal, a perceptual quality evaluation may be performed. At this time, the perceptual quality evaluation is based on the objective evaluation of PESQ (Perceptual Evaluation of Speech Quality), POLQA (Perceptual Objective Listening Quality Assessment), PEAQ (Perceptual Evaluation of Audio Quality), MOS (Mean Opinion Score), MUSHRA (Multiple Stimuli with Hidden Reference and Anchor) subjective assessments can be used. Perceptual quality assessment is not limited thereto and may include other quality assessments.

The perceptual quality evaluation may indicate a quality evaluation of the currently learned model. Based on the quality evaluation, it may be determined whether or not model requirements such as preset quality and complexity of the model are satisfied. Also, based on the quality evaluation, it may be determined whether the model topology, such as model complexity, can be adjusted. Here, the complexity of the model may have a positive correlation with the number of hidden layers and nodes.

According to an embodiment, when it is determined that the topology of the model is not adjustable or does not need to be adjusted, parameters of the currently learned model may be stored and learning may be terminated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when it is determined that the topology of the model can be adjusted, the topology of the model can be updated, and the above-described process for learning can be repeated. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the above-described process for learning may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the above-described process for learning may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

According to an embodiment, signal processing such as compression/noise removal/encoding/decoding may be performed on an input audio signal based on parameters of a learned model. At this time, the signal processing is not limited to this.

According to an embodiment, signal processing may be performed on the first audio signal based on the parameters of the learned model. In this case, signal processing may include compression/noise removal/encoding/decoding, but is not limited thereto.

9 is a diagram illustrating an audio signal decoding method performed by an audio signal decoding apparatus to which a neural network is applied, according to an exemplary embodiment.

In step 901, the audio signal decoding apparatus may receive a bitstream encoded with a latent vector generated by applying a parameter learned through a neural network to an input audio signal. In this case, the audio signal decoding apparatus may apply a model learned through a neural network. Here, the neural network may include an autoencoder.

In step 902, the audio signal decoding apparatus may restore a latent vector from the received bitstream. Here, the bitstream may be generated by quantizing or encoding a latent vector to be transmitted through a transmission channel.

In step 903, the audio signal decoding apparatus may decode the output audio signal from the restored latent vector using the hidden layer to which the learned parameter is applied. Here, the process of learning the parameters of the hidden layer has been described in detail above.

According to an embodiment, latent vectors may be generated based on learned parameters when it is impossible or not necessary to adjust the topology of a model including the number of hidden layers and nodes. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when the topology of a model including the number of hidden layers and the number of nodes can be adjusted, the latent vector may be generated based on the relearned parameter by applying the adjusted topology. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the process for learning parameters may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the process for learning parameters may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

10 is a diagram illustrating an audio signal encoding apparatus to which a neural network is applied, according to an exemplary embodiment.

The audio signal encoding apparatus 1000 may include a processor 1010 and a memory 1020. The memory 1020 may include one or more instructions executable by a processor.

The processor 1010 may receive an input audio signal. In this case, the processor 1010 of the audio signal encoding apparatus 1000 may apply a model learned through a neural network. Here, the neural network may include an autoencoder.

The processor 1010 of the audio signal encoding apparatus 1000 may generate a dimensionally reduced latent vector of the input audio signal based on the parameters of the hidden layer learned using the neural network. Here, the process of learning the parameters of the hidden layer has been described in detail above.

According to an embodiment, latent vectors may be generated based on learned parameters when it is impossible or not necessary to adjust the topology of a model including the number of hidden layers and nodes. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when the topology of a model including the number of hidden layers and the number of nodes can be adjusted, the latent vector may be generated based on the relearned parameter by applying the adjusted topology. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the process for learning parameters may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the process for learning parameters may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

The processor 1010 of the audio signal encoding apparatus 1000 may encode the generated latent vector and output a bitstream. Here, the latent vector may be output as a bitstream after being quantized or coded to be transmitted through a transmission channel.

11 is a diagram illustrating an audio signal decoding apparatus to which a neural network is applied according to an exemplary embodiment.

The audio signal decoding apparatus 1100 may include a processor 1110 and a memory 1120. The memory 1120 may include one or more instructions executable by a processor.

The processor 1110 may receive a bitstream encoded with a latent vector generated by applying a parameter learned through a neural network to an input audio signal. In this case, the processor 1110 of the audio signal decoding apparatus 1100 may apply a model learned through a neural network. Here, the neural network may include an autoencoder.

The processor 1110 of the audio signal decoding apparatus 1100 may reconstruct latent vectors from the received bitstream. Here, the bitstream may be generated by quantizing or encoding a latent vector to be transmitted through a transmission channel.

The processor 1110 of the audio signal decoding apparatus 1100 may decode an output audio signal from the restored latent vector using the hidden layer to which the learned parameter is applied. Here, the process of learning the parameters of the hidden layer has been described in detail above.

According to an embodiment, latent vectors may be generated based on learned parameters when it is impossible or not necessary to adjust the topology of a model including the number of hidden layers and nodes. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

According to an embodiment, when the topology of a model including the number of hidden layers and the number of nodes can be adjusted, the latent vector may be generated based on the relearned parameters by applying the adjusted topology. At this time, the determination of whether the topology of the model can be adjusted may differ depending on the field to which the model is applied and the requirements for the model.

For example, if the quality requirements are not satisfied, the topology of the model may be adjusted to increase the complexity of the model, and the process for learning parameters may be repeated. Accordingly, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation does not satisfy a preset quality requirement, the parameters may be re-learned using a model with increased complexity. Alternatively, when the quality requirements are not satisfied but the complexity of the model is not increased any more, parameters of the currently learned model may be stored and learning may be terminated.

For another example, if the quality requirements are satisfied, the topology of the model may be adjusted so that the complexity of the model is reduced within the quality requirements, and the process for learning parameters may be repeated. Therefore, when determining whether the topology can be adjusted, if the result of the perceptual quality evaluation satisfies the preset quality requirements, the parameters may be re-learned using a model whose complexity is reduced within the quality requirements.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. or even if replaced or substituted by equivalents, appropriate results can be achieved.

110: input layer
120: output layer
130: hidden layer

Claims (20)

In the neural network learning method applied to the audio signal encoding method using an audio signal encoding apparatus,
generating a masking threshold for the first audio signal before being learned;
calculating a weight matrix to be applied to each frequency component of the first audio signal based on the masking threshold;
generating a weighted error function by correcting a preset error function using the weighting matrix;
generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal;
and include
The learning method,
Comparing the generated second audio signal with the first audio signal to perform perceptual quality evaluation;
The neural network,
A learning method for determining whether a topology included in a model can be adjusted based on the perceptual quality evaluation.
According to claim 1,
The weighting matrix includes weights to be applied to frequency components of the first audio signal;
The weight is
The learning method is set to be inversely proportional to the masking threshold for the first audio signal and proportional to the size of the frequency component of each of the first audio signals.
delete According to claim 1,
The perceptual quality evaluation,
Objective assessment of Perceptual Evaluation of Speech Quality (PESQ), Perceptual Objective Listening Quality Assessment (POLQA), Perceptual Evaluation of Audio Quality (PEAQ), or
A learning method that includes subjective assessment of Mean Opinion Score (MOS) and Multiple Stimuli with Hidden Reference and Anchor (MUSHRA).
delete According to claim 1,
When determining whether the topology can be adjusted,
If the result of the perceptual quality evaluation does not satisfy a preset quality requirement, relearning parameters using a model with increased complexity,
If the result of the perceptual quality evaluation satisfies a preset quality requirement, a learning method for re-learning parameters using a model with reduced complexity within the quality requirement.
An audio signal encoding method performed by an audio signal encoding apparatus to which a neural network is applied,
receiving an input audio signal;
generating a dimensionally reduced latent vector of the input audio signal based on parameters of the hidden layer, wherein the neural network includes one or more hidden layers;
Encoding the generated latent vector to output a bitstream
including,
Generating a dimensionally reduced latent vector of the input audio signal based on the parameters of the hidden layer learned using the neural network includes:
If it is impossible or not necessary to adjust the topology of the model including the number of hidden layers and the number of nodes, the latent vector is generated based on the learned parameters;
If the topology of the model can be adjusted, the audio signal encoding method generates the latent vector based on the relearned parameters by applying the adjusted topology.
delete According to claim 7,
Encoding of the latent vector,
An audio signal encoding method for binarizing the bitstream for transmission through a channel.
In the neural network learning method applied to the audio signal decoding method using an audio signal decoding apparatus,
generating a masking threshold for the first audio signal before being learned;
calculating a weight matrix to be applied to each frequency component of the first audio signal based on the masking threshold;
generating a weighted error function by correcting a preset error function using the weighting matrix;
generating a second audio signal by applying a parameter learned using the weighted error function to the first audio signal;
including,
The learning method,
Comparing the generated second audio signal with the first audio signal to perform perceptual quality evaluation;
The neural network,
A learning method for determining whether a topology included in a model can be adjusted based on the perceptual quality evaluation.
According to claim 10,
The weighting matrix includes weights to be applied to frequency components of the first audio signal;
The weight is
The learning method is set to be inversely proportional to the masking threshold for the first audio signal and proportional to the size of the frequency component of each of the first audio signals.
delete According to claim 10,
The perceptual quality evaluation,
Objective assessment of Perceptual Evaluation of Speech Quality (PESQ), Perceptual Objective Listening Quality Assessment (POLQA), Perceptual Evaluation of Audio Quality (PEAQ), or
A learning method that includes subjective assessment of Mean Opinion Score (MOS) and Multiple Stimuli with Hidden Reference and Anchor (MUSHRA).
delete According to claim 10,
When determining whether the topology can be adjusted,
If the result of the perceptual quality evaluation does not satisfy the preset quality requirements, relearning parameters using a model with increased complexity,
If the result of the perceptual quality evaluation satisfies the preset quality requirements, the learning method of re-learning parameters using a model whose complexity is reduced within the quality requirements.
An audio signal decoding method performed by an audio signal decoding apparatus to which a neural network is applied,
receiving a bitstream encoded with a latent vector generated by applying the parameter learned through the neural network to an input audio signal;
restoring the latent vector from the received bitstream;
The neural network includes one or more hidden layers, and decoding an output audio signal from the reconstructed latent vector using the hidden layer to which the learned parameter is applied.
including,
The latent vector,
When it is impossible or necessary to adjust the topology of a model including the number of hidden layers and the number of nodes belonging to the hidden layer, it is generated based on the learned parameters;
An audio signal decoding method that is generated based on a relearned parameter by applying the adjusted topology when the topology can be adjusted.
delete According to claim 16,
Encoding of the latent vector,
An audio signal decoding method for binarizing the bitstream for transmission through a channel.
In the audio signal encoding apparatus to which the neural network is applied,
The audio signal encoding apparatus includes a processor and a memory including one or more instructions executable by the processor,
When the one or more instructions are executed in the processor,
receive an input audio signal;
The neural network includes one or more hidden layers, and generates a dimensionally reduced latent vector of the input audio signal based on parameters of the hidden layer learned using the neural network;
Encoding the generated latent vector to output a bitstream;
The latent vector,
If it is impossible or not necessary to adjust the topology of the model including the number of hidden layers and nodes, it is generated based on the learned parameters;
If the topology can be adjusted, the audio signal encoding device is generated based on the relearned parameter by applying the adjusted topology.
In the audio signal decoding apparatus to which the neural network is applied,
The audio signal decoding apparatus includes a processor and a memory including one or more instructions executable by the processor,
When the one or more instructions are executed in the processor,
Receiving a bitstream in which a latent vector generated by applying a parameter learned through the neural network to an input audio signal is quantized;
restoring the latent vector from the received bitstream;
The neural network includes one or more hidden layers, and decodes an output audio signal from the reconstructed latent vector using the hidden layer to which the learned parameter is applied;
The latent vector,
If it is impossible or not necessary to adjust the topology of the model including the number of hidden layers and nodes, it is generated based on the learned parameters;
If the topology can be adjusted, the audio signal decoding apparatus is generated based on the relearned parameter by applying the adjusted topology.

Images