CN101361117B - Method and apparatus for processing a media signal - Google Patents

Abstract

An apparatus for processing a media signal and method thereof are disclosed, by which the media signal can be converted to a surround signal by using spatial information of the media signal. The present invention provides a method of processing a signal, comprising: extracting a downmix signal from a bit stream; generating decorrelated downmix signal through using decorrelator on the downmix signal; generating surround signal through using the rendering information for generating surround signal on the downmix signal and the decorrelated downmix signal.

Description

Method and device for processing media signals

technical field

The present invention relates to a device and method for processing media signals, in particular to a device and method for generating surround signals by using spatial information of media signals.

Background technique

In general, various types of devices and methods have been widely used to generate a multi-channel media signal by using spatial information of the multi-channel media signal and a downmix signal, wherein the downmix signal It is generated by downmixing a multi-channel media signal to a mono or stereo signal.

However, the methods and apparatus described above are not usable in environments that are not suitable for generating multi-channel signals. For example, they are not available for devices that can only generate stereo signals. In other words, there is no existing method or apparatus for generating a surround signal having multi-channel characteristics in an environment where the multi-channel signal cannot be generated by using the spatial information of the multi-channel signal.

Therefore, since there is not any existing method or apparatus for generating surround signals in a device capable of generating only mono or stereo signals, it is difficult to efficiently process media signals.

invention disclosure

technical problem

Accordingly, the present invention is directed to an apparatus and method for processing a media signal that substantially obviate one or more problems due to limitations and disadvantages of the related art.

An object of the present invention is to provide an apparatus for processing a signal and method thereof, by which a media signal can be converted into a surround signal by using spatial information of the media signal.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Technical solutions

To achieve these and other advantages and in accordance with the object of the present invention, a method of processing a signal according to the present invention comprises generating a signal corresponding to each of the plurality of sources by using spatial information indicative of features between the plurality of sources generating sub-rendering information by applying filter information giving a surround effect to the source-mapping information by source; generating rendering information for generating a surround signal by integrating at least one of the sub-rendering information; and A surround signal is generated by applying this rendering information to a downmix signal generated by downmixing the multiple sources.

To further achieve these and other advantages and in accordance with the object of the present invention, an apparatus for processing a signal comprises: a source mapping unit which generates a signal corresponding to each of the plurality of sources by using spatial information indicative of features between the plurality of sources Source mapping information of the source; a sub-rendering information generation unit that generates sub-rendering information by applying filter information with a surround effect to the source mapping information by source; an integration unit that generates sub-rendering information by integrating at least one of the sub-rendering information generating rendering information for generating the surround signal; and a rendering unit generating the surround signal by applying the rendering information to the downmix signal generated by downmixing the plurality of sources.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

Beneficial effect

The signal processing apparatus and method according to the present invention enable a decoder receiving a bit stream including a down-mix signal generated by down-mixing a multi-channel signal and spatial information of the multi-channel signal to A signal with a surround effect is generated in an environment where it is not possible to restore the multi-channel signal.

Brief description of the drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

In the attached picture:

1 is a block diagram of an audio signal encoding device and an audio signal decoding device according to an embodiment of the present invention;

Fig. 2 is a structural diagram of a bit stream of an audio signal according to an embodiment of the present invention;

3 is a detailed block diagram of a spatial information conversion unit according to an embodiment of the present invention;

4 and 5 are block diagrams of channel configurations for a source mapping process according to one embodiment of the invention;

6 and 7 are detailed block diagrams of a rendering unit for a stereo channel downmix signal according to an embodiment of the present invention;

8 and 9 are detailed block diagrams of a rendering unit for a mono channel downmix signal according to an embodiment of the present invention;

10 and 11 are block diagrams of a smoothing unit and an expanding unit according to an embodiment of the present invention;

12 is a graph for explaining a first smoothing method according to an embodiment of the present invention;

13 is a graph for explaining a second smoothing method according to an embodiment of the present invention;

14 is a graph for explaining a third smoothing method according to an embodiment of the present invention;

15 is a graph for explaining a fourth smoothing method according to an embodiment of the present invention;

16 is a graph for explaining a fifth smoothing method according to an embodiment of the present invention;

FIG. 17 is a diagram for explaining prototype filter information corresponding to each channel;

18 is a block diagram of a first method for generating rendering filter information in a spatial information conversion unit according to an embodiment of the present invention;

19 is a block diagram of a second method of generating rendering filter information in a spatial information conversion unit according to an embodiment of the present invention;

20 is a block diagram of a third method of generating rendering filter information in a spatial information conversion unit according to an embodiment of the present invention;

21 is a diagram for explaining a method of generating surround signals in a rendering unit according to an embodiment of the present invention;

Figure 22 is a diagram of a first interpolation method according to one embodiment of the present invention;

Figure 23 is a diagram of a second interpolation method according to one embodiment of the present invention;

FIG. 24 is a diagram of a block switching method according to one embodiment of the present invention;

Fig. 25 is a block diagram of applying the position of the window length determined by the window length determination unit according to one embodiment of the present invention;

26 is a diagram of filters of various lengths used in processing audio signals according to one embodiment of the present invention;

27 is a diagram of a method for separately processing an audio signal by using a plurality of sub-filters according to one embodiment of the present invention;

28 is a block diagram of a method for rendering split rendering information generated by a plurality of sub-filters to a mono channel downmix signal according to an embodiment of the present invention;

29 is a block diagram of a method for rendering split rendering information generated by a plurality of sub-filters to a stereo downmix signal according to an embodiment of the present invention;

FIG. 30 is a block diagram of a first domain conversion method of a channel downmix signal according to an embodiment of the present invention; and

Fig. 31 is a block diagram of a second domain conversion method for a downmix signal according to an embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Reference will now be made in detail to the preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings.

FIG. 1 is a block diagram of an audio signal encoding device and an audio signal decoding device according to an embodiment of the present invention.

Referring to FIG. 1 , the encoding device 10 includes a downmix unit 100 , a spatial information generation unit 200 , a downmix signal encoding unit 300 , a spatial information encoding unit 400 , and a multiplexing unit 500 .

If a multi-source (X1, X2, ..., Xn) audio signal is input to the downmix unit 100, the downmix unit 100 downmixes the input signal into a downmix tone signal. In this case, the downmix signal includes mono, stereo and multi-source audio signals.

Sources include channels and are conveniently denoted as channels in the description below. In this specification, a mono or stereo downmix signal is used as a reference. However, the invention is not limited to mono or stereo downmix signals.

The encoding device 10 can optionally use an arbitrary downmix signal provided directly from the external environment.

The spatial information generation unit 200 generates spatial information from a multi-channel audio signal. This spatial information can be generated during downmixing. The generated downmix signal and spatial information are respectively encoded by the downmix signal encoding unit 300 and the spatial information encoding unit 400 , and then transmitted to the multiplexing unit 500 .

In the present invention, 'spatial information' refers to information required by a decoding device to generate a multi-channel signal from downmixing a downmix signal encoded by generated by the device by downmixing the multi-channel signal and transmitted to the decoding device. Spatial information includes spatial parameters. Spatial parameters include CLD (Channel Level Difference) indicating the energy difference between channels, ICC (Inter-Channel Coherence) indicating the correlation between channels, used when generating three channels from two channels CPC (Channel Prediction Coefficient) etc.

In the present invention, 'downmix signal encoding unit' or 'downmix signal decoding unit' refers to a codec that encodes or decodes audio signals instead of spatial information. In this specification, a downmix audio signal is taken as an example of an audio signal instead of spatial information. Also, the downmix signal encoding or decoding unit may include MP3, AC-3, DTS, or AAC. Furthermore, the downmix signal encoding or decoding unit may include future codecs as well as codecs that have been developed before.

The multiplexing unit 500 generates a bit stream by multiplexing the downmix signal with spatial information, and then transmits the generated bit stream to the decoding device 20 . Also, the structure of this bitstream will be explained later in FIG. 2 .

The decoding device 20 includes a demultiplexing unit 600 , a downmix signal decoding unit 700 , a spatial information decoding unit 800 , a rendering unit 900 , and a spatial information conversion unit 1000 .

The demultiplexing unit 600 receives a bitstream and then separates the encoded downmix signal and the encoded spatial information from the bitstream. Then, the down-mix signal decoding unit 700 decodes the encoded down-mix signal, and the spatial information decoding unit 800 decodes the encoded spatial information.

The spatial information conversion unit 1000 generates rendering information applicable to the downmix signal using the decoded spatial information and filter information. In this case, the rendering information is applied to the downmix signal to generate a surround signal.

For example, surround signals are generated as follows. First, the process of generating a downmix signal from a multi-channel audio signal by the encoding device 10 may include several steps using an OTT (one to two) box or a TTT (three to three) box. In this case, spatial information can be generated from each of these steps. The spatial information is transmitted to decoding means 20 . The decoding device 20 then generates the surround signal by converting the spatial information and then rendering the converted spatial information with the downmix signal. Instead of generating a multi-channel signal by downmixing a downmix signal, the present invention instead involves a rendering method comprising the steps of extracting the spatial information, and perform rendering by using the extracted spatial information. For example, HRTF (Head Related Transfer Function) filtering is usable in this rendering method.

In this case, spatial information is a value that also applies to mixed domains. Therefore, rendering can be classified into the following types according to domains.

The first type is to perform rendering on the mix domain by passing the downmix signal through a mix filter bank. In this case, domain conversion of the spatial information is unnecessary.

The second type is to perform rendering on the temporal domain. In this case, the second type exploits the fact that HRTF filters are modeled as FIR (finite inverse response) filters or IIR (infinite inverse response) filters in the time domain. Therefore, a process of converting spatial information into time-domain filter coefficients is required.

A third type is to perform rendering on a different frequency domain. For example, this rendering is performed on the DFT (Discrete Fourier Transform) domain. In this case, a process of transforming the spatial information into the corresponding domain is necessary. In particular, the third type enables fast operations by replacing filtering in the time domain with operations in the frequency domain.

In the present invention, filter information is information on a filter required for processing an audio signal, and includes filter coefficients provided to a specific filter. An example explaining the filter information is as follows. First, the prototype filter information is original filter information of a specific filter, and can be expressed as GL_L or the like. The converted filter information indicates filter coefficients after the prototype filter information has been converted, and may be expressed as GL_L or the like. The sub-rendering information refers to the filter information obtained by spatializing the prototype filter information to generate the surround signal, and may be expressed as FL_L1 and the like. Rendering information refers to filter information required to perform rendering, and may be expressed as HL_L or the like. The interpolated/smoothed rendering information refers to filter information obtained from interpolating/smoothing this rendering information, and may be expressed as HL-L or the like. In this specification, the above filter information is referred to. However, the present invention is not limited by the name of the filter information. Specifically, take HRTF as an example of filter information. However, the present invention is not limited to HRTFs.

The rendering unit 900 receives the decoded downmix signal and rendering information, and generates surround information using the decoded downmix signal and rendering information. A surround signal may be a signal that provides a surround effect to an audio system that is only capable of generating a stereo signal. The present invention can be applied to various systems other than an audio system capable of generating only stereo signals.

Fig. 2 is a structural diagram of a bit stream of an audio signal according to an embodiment of the present invention, wherein the bit stream includes encoded downmix signals and encoded spatial information.

Referring to FIG. 2, a 1-frame audio payload includes a downmix signal field and an ancillary data field. Encoded spatial information may be stored in this auxiliary data field. For example, if the audio payload is 48˜128 kbps (kilobits per second), the spatial information may have a range of 5˜32 kbps. However, there is no limit to the range of audio payload and spatial information.

FIG. 3 is a detailed block diagram of a spatial information conversion unit according to an embodiment of the present invention.

Referring to FIG. 3 , the spatial information converting unit 1000 includes a source mapping unit 1010 , a sub-rendering information generating unit 1020 , an integrating unit 1030 , a processing unit 1040 , and a domain converting unit 1050 .

The source mapping unit 101 generates source mapping information corresponding to each source of an audio signal by performing source mapping using spatial information. In this case, the source map information refers to per-source information generated by using spatial information or the like so as to correspond to each source of the audio signal. A source includes channels, and in this case, source map information corresponding to each channel is generated. The source map information may be represented as coefficients. And, the source mapping process will be explained in detail later with reference to FIGS. 4 and 5 .

The sub-rendering information generating unit 1020 generates sub-rendering information corresponding to each source by using source map information and filter information. For example, if the rendering unit 900 is an HRTF filter. Then the sub-rendering information generation unit 1020 can generate sub-rendering information by using HRTF filter information.

The integrating unit 1030 generates rendering information by integrating the sub-rendering information so as to correspond to each source of the downmix signal. Rendering information generated by using spatial information and filter information refers to information that generates a surround signal by being applied to a downmix signal. And, the rendering information includes a filter coefficient type. Integration can be omitted to reduce the computational load of the rendering process. Subsequently, the rendering information is transmitted to the processing unit 1042 .

The processing unit 1042 includes an interpolation unit 1041 and/or a smoothing unit 1042 . The rendering information is interpolated by an interpolation unit 1041 and/or smoothed by a smoothing unit 1042 .

The domain conversion unit 1050 converts the domain of the rendering information to the domain of the downmix signal used by the rendering unit 900 . And, the domain converting unit 1050 may be provided to one of various positions including the position shown in FIG. 3 . Therefore, if the rendering information is generated on the same domain as the rendering unit 900, the domain converting unit 1050 may be omitted. The domain-converted rendering information is then transmitted to the rendering unit 900 .

The spatial information conversion unit 1000 may include a filter information conversion unit 1060 . In FIG. 3 , a filter information converting unit 1060 is provided within the spatial information converting unit 100 . Alternatively, the filter information converting unit 1060 may be provided outside the spatial information converting unit 100 . The filter information conversion unit 1060 is converted to be suitable for generating sub-rendering information or rendering information from random filter information such as HRTF or the like. The conversion process of the filter information may include the following steps.

First, the step of matching domains to be applicable is included. This domain matching step is required if the domain of the filter information does not match the domain where rendering is performed. For example, a step of converting time-domain HRTF to DFT, QMF or hybrid domain for generating rendering information is required.

Second, a coefficient reduction step may be included. In this case, it is easy to save the domain-transformed HRTF and apply the domain-converted HRTF to spatial information. For example, if the prototype filter coefficients have a response with a long number of taps (length), the corresponding coefficients must be stored in a storage space corresponding to responses whose corresponding lengths add up to 10 in the case of 5.1 channels. This increases the load on the memory and the amount of computation. In order to prevent this problem, a method of compacting stored filter coefficients while maintaining filter characteristics during domain conversion may be employed. For example, HRTF responses can be transformed into a few parameter values. In this case, the parameter generation process and parameter values may differ according to the domain of application.

The downmix signal passes through the domain conversion unit 1110 and/or the decorrelation unit 1200 before being rendered with the rendering information. In case the domain of the rendering information is different from the domain of the downmix signal, the domain conversion unit 1110 converts the domain of the downmix signal to match the two domains.

The decorrelation unit 1200 is applied to the domain converted downmix signal. This can be relatively computationally expensive compared to applying a decorrelator to the rendered information. However, it prevents distortion during the generation of rendering information. If the amount of computation is allowable, the decorrelating unit 1200 may include a plurality of decorrelators different in characteristics from each other. If the downmix signal is a stereo signal, the decorrelation unit 1200 may not be used. In Figure 3, a domain-converted mono downmix signal is used in the rendering process—that is, the case of a mono downmix signal in the frequency, mixing, QMF, or DFT domains In , a decorrelator is used on the corresponding domain. Also, the present invention also includes a decorrelator used in the time domain. In this case, the monaural downmix signal preceding the domain conversion section 1100 is directly input to the decorrelation section 1200 . A first-order or higher-order IIR filter (or FIR filter) can be used as a decorrelator.

Subsequently, the rendering unit 900 generates a surround signal using the downmix signal, the decorrelated downmix signal, and the rendering information. If the down-mix signal is a stereo signal, the decorrelated down-mix signal may not be used. Details of the rendering process will be described later with reference to FIGS. 6 to 9 .

The surround signal is converted to the time domain by the domain inverse conversion unit 1300 and then output. If so, the user can hear the sound with multi-channel effects through stereo headphones or the like.

4 and 5 are block diagrams of channel configurations for a source mapping process according to one embodiment of the present invention. The source mapping process is a process of generating source mapping information corresponding to each source of an audio signal by utilizing spatial information. As mentioned in the above description, a source includes channels, and source mapping information may be generated so as to correspond to the channels shown in FIGS. 4 and 5 . Source map information is generated in a type suitable for the rendering pass.

For example, if the downmix signal is a mono signal, it is possible to generate source mapping information using spatial information such as CLD1-CLD5, ICC1-ICC5, and the like.

Source mapping information can be expressed as D_L(=D _L ), D_R(=D _R ), D_C(=D _C ), D_LFE(=D _LFE ), D_Ls(=D _Ls ), D_R(=D _Rs ), etc. value. In this case, the process of generating the source map information may vary according to the tree structure corresponding to the spatial information, the range of the spatial information to be used, and the like. In this specification, the downmix signal is, for example, a mono signal, which does not limit the present invention.

Right and left channel outputs output from the rendering unit 900 may be expressed as a mathematical operation 1 .

Mathematical Calculus 1

Lo=L*GL_L'+C*GC_L'+R*GR_L'+Ls*GLs_L'+Rs*GRs_L'

Ro=L*GL_R'+C*GC_R'+R*GR_R'+Ls*GLs_R'+Rs*GRs_R'

In this case, the operator '*' indicates a product on the DFT domain and can be replaced by a QMF or convolution on the time domain.

The present invention includes a method of generating L, C, R, Ls and Rs from source map information using spatial information or from source map information using spatial information and filter information. For example, source map information may be generated using only CLD of spatial information or using CLD and ICC of spatial information. The method of generating source map information using only CLD is explained below.

In the case where this tree structure has the structure shown in FIG. 4 , the first method of obtaining source map information using only CLD can be expressed as Mathematical Calculations 2 .

Mathematical Calculus 2

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; LFE</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}$

In this case,

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; OTT</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; =</span>\sqrt{\frac{{\mathrm{<span\; class="notranslate">\; 10</span>}}^{\frac{{\mathrm{<span\; class="notranslate">\; CLD</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}}}{\mathrm{<span\; class="notranslate">\; 10</span>}}}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 10</span>}}^{\frac{{\mathrm{<span\; class="notranslate">\; CLD</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}}}{\mathrm{<span\; class="notranslate">\; 10</span>}}}}}$

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OT</span>}{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; =</span>\sqrt{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 10</span>}}^{\frac{{\mathrm{<span\; class="notranslate">\; CLD</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}}}{\mathrm{<span\; class="notranslate">\; 10</span>}}}}}<span\; class="notranslate">\; ,</span>$

, and 'm' indicates a mono downmix signal.

In the case where this tree structure has the structure shown in FIG. 5 , the second method of obtaining source map information using only CLD can be expressed as Mathematical Calculations 3 .

Mathematical Calculus 3

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}\\ {\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; LFE</span>}}\end{array}\right.m<span\; class="notranslate">\; ]</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}$

If the source map information is generated using only CLD, the three-dimensional effect may be reduced. So source map information can be generated using ICC and/or decorrelators. And, the multi-channel information generated by using the decorrelator output signal dx(m) can be expressed as a mathematical operation 4 .

Mathematical Calculus 4

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\end{array}\right]$

In this case, 'A', 'B', and 'C' are values expressible by using CLD and ICC. 'd ₀ ' to 'd ₃ ' indicate decorrelators. And, 'm' indicates a monaural downmix signal. However, this method cannot be used to generate source map information such as D_L, D_R, etc.

Therefore, a first method of generating source mapping information using CLD, ICC and/or decorrelators on the downmix signal considers dx(m) (x=0, 1, 2) as independent inputs. In this case, 'dx' can be used in the process of generating sub-rendering filter information according to Mathematical Calculus 5.

Mathematical Calculus 5

FL_L_M=d_L_M*GL_L' (mono input → left output)

FL_R_M=d_L_M*GL_R' (mono input → right output)

FL_L_Dx＝d_L_Dx*GL_L'(Dx output→left output)

FL_R_Dx＝d_L_Dx*GL_R'(Dx output→right output)

And, the rendering information may be generated according to the mathematical calculation 6 using the result of the mathematical calculation 5 .

Mathematical Calculus 6

HM_L＝FL_L_M+FR_L_M+FC_L_M+FLS_L_M+FRS_L_M+FLFE_L_M

HM_R＝FL_R_M+FR_R_M+FC_R_M+FLS_R_M+FRS_R_M+FLFE_R_M

HDx_L＝FL_L_Dx+FR_L_Dx+FC_L_Dx+FLS_L_Dx+FRS_L_Dx+FLFE_L_Dx

HDx_R＝FL_R_Dx+FR_R_Dx+FC_R_Dx+FLS_R_Dx+FRS_R_Dx+FLFE_R_Dx

Details of the rendering information generation process are explained later. The first method of generating source map information using CLD, ICC and/or decorrelator processes the dx output value, 'dx(m)', as a separate input, which may increase the computational load.

A second method of generating source mapping information using CLD, ICC and/or decorrelators employs a decorrelator applied in the frequency domain. In this case, the source map information can be expressed as a mathematical operation7.

Mathematical Calculus 7

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 3</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}$

In this case, by applying the decorrelator on the frequency domain, the same source mapping information as before applying the decorrelator, such as D_L, D_R, etc., can be generated. So, it can be implemented in a simple manner.

A third method of generating source map information using CLD, ICC, and/or a decorrelator employs a decorrelator with an all-pass characteristic like the decorrelator of the second method. In this case, the all-pass characteristic means that the magnitude is fixed and only the phase changes. Also, the present invention can employ a decorrelator having an all-pass characteristic like the decorrelator of the first method.

A fourth method of generating source mapping information using CLD, ICC, and/or decorrelators replaces the ' d0' to 'd3' to perform decorrelation. In this case, the source map information can be expressed as a mathematical operation 8 .

Mathematical Calculus 8

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="A\; C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">A</figure-callout></span><figure-callout\; id="1"\; label="A\; C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}\\ {\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="LS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LS</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="ls\; A\; RS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span><figure-callout\; id="1"\; label="ls\; A\; RS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">\; </figure-callout>}}& \\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{}{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}_{}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}$

In this case 'k' is the energy value of the decorrelated signal determined from the CLD and ICC values. And 'd_L', 'd_R', 'd_C', 'd_Ls' and 'd_Rs' indicate decorrelators applied to channels, respectively.

The fifth method of generating source mapping information using CLD, ICC and/or decorrelator is by configuring 'd_L' and 'd_R' to be symmetrical to each other in the fourth method and configuring 'd_Ls' and 'd_Rs' in the fourth method The configurations are symmetrical to each other to maximize the decorrelation effect. Specifically, assuming that d_R=f(d_L) and d_Rs=f(d_Ls), only 'd_L', 'd_C' and 'd_Ls' need to be designed.

A sixth method of generating source map information using CLD, ICC and/or decorrelator is to configure 'd_L' and 'd_Ls' to have correlation in the fifth method. Also, 'd_L' and 'd_C' may be configured to have a correlation.

A seventh method for generating source map information using CLD, ICC and/or decorrelators is to use the decorrelators in the third method as a cascaded or nested structure of all-pass filters. The seventh method takes advantage of the fact that the all-pass characteristic is maintained even if an all-pass filter is used as a series or nested structure. In the case of using all-pass filters as series or nested structures, more diverse kinds of phase responses can be obtained. Therefore, the decorrelation effect can be maximized.

An eighth method of generating source mapping information using CLD, ICC, and/or a decorrelator is to use a related art decorrelator together with a frequency domain decorrelator of the second method. In this case, the multi-channel signal can be expressed as a mathematical operation9.

Mathematical Calculus 9

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="A\; C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">A</figure-callout></span><figure-callout\; id="1"\; label="A\; C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; </span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}\\ {\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="LS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LS</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="ls\; A\; RS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span><figure-callout\; id="1"\; label="ls\; A\; RS"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">\; </figure-callout>}}& \\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{}{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}_{}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}\end{array}\right]\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&CenterDot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>\\ {\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; new</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; \&Center\; Dot;</span>\end{array}\right]$

In this case, the filter coefficient generation process uses the same process explained in the first method - except 'A' is changed to 'A+Kd'.

A ninth method of generating source mapping information using CLD, ICC and/or a decorrelator is by applying a frequency domain decorrelator to the output of a related art decorrelator in the case of a related art decorrelator The value after further decorrelation. Therefore, source map information can be generated with a small amount of computation by overcoming the limitation of the frequency domain decorrelator.

A tenth method of generating source map information using CLD, ICC and/or decorrelators is expressed as mathematical calculation 10 .

Mathematical Calculus 10

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 4</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="c"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="c"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">c</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; OTT</span>}\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; m</span>}\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; LS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; ls</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; RS</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; K</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; Rs.</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\end{array}\right]$

In this case 'di_(m)' (i=L, R, C, Ls, Rs) is the decorrelator output value applied to channel i. Also, the output value can be processed in time domain, frequency domain, QMF domain, mixed domain, etc. If the output value was processed on a different domain than the one currently being processed, it may be transformed by a domain transformation. The same 'd can be used for d_L, d_R, d_C, d_Ls and d_Rs. In this case, the mathematical calculation 10 can be expressed in a very simple manner.

If Math10 is applied to Math1, Math1 can be expressed as Math11.

Mathematical Calculus 11

Lo=HM_L*m+HMD_L*d(m)

Ro＝HM_R*R+HMD_R*d(m)

In this case, the rendering information HM_L is a value resulting from combining the spatial information and the filter information to generate the surround signal Lo with the input m. And the rendering information HM_R is a value obtained from combining the spatial information and the filter information to generate the surround signal Ro with the input m. In addition, 'd(m)' is the decorrelator output value generated by converting the decorrelator output value on an arbitrary domain to the value on the current domain, or the decorrelator output value generated by processing on the current domain output value. The rendering information HMD_L is a value indicating the degree to which the decorrelator output value d(m) is added to 'Lo' at the time of rendering d(m), and is also a value obtained by combining spatial information and filter information. The rendering information HMD_R is a value indicating the degree to which the decorrelator output value d(m) is added to 'Ro' when d(m) is rendered.

Thus, in order to perform a rendering process on a mono down-mix signal, the present invention proposes a method for rendering the down-mix signal by combining the spatial information to the down-mix signal and the decorrelated down-mix signal. A method for generating surround signals from rendering information generated with filter information (eg, HRTF filter coefficients). This rendering process can be performed domain-independently. If 'd(m)' is expressed as 'd*m' (product operator) performed on the frequency domain, mathematical operation 11 can be expressed as mathematical operation 12 .

Mathematical Calculus 12

Lo=HM_L*m+HMD_L*d*m=HMoverall_L*m

Ro=HM_R*m+HMD_R*d*m=HMoveralf_R*m

Thus, in the case of performing a rendering process on a downmix signal in the frequency domain, it is possible to make the operation amount is minimized.

6 and 7 are detailed block diagrams of a rendering unit for a stereo downmix signal according to an embodiment of the present invention.

Referring to FIG. 6, the rendering unit 900 includes a rendering unit-A 910 and a rendering unit-B 920.

If the downmix signal is a stereo signal, the spatial information converting unit 1000 generates rendering information for left and right channels of the downmix signal. The rendering unit-A 910 generates the surround signal by rendering rendering information for the left channel of the down-mix signal to the left channel of the down-mix signal. And, the rendering unit-B 920 generates the surround signal by rendering the rendering information for the right channel of the down-mix signal to the right channel of the down-mix signal. The names of the channels are merely exemplary and do not limit the invention.

The rendering information may include rendering information delivered to the same channel and rendering information delivered to another channel.

For example, the spatial information conversion unit 1000 can generate rendering information HL_L and HL_R input to a rendering unit for the left channel of the downmix signal, wherein the rendering information HL_L is delivered to the left output corresponding to the same channel, and The rendering information HL_R is delivered to the right output corresponding to the other channel. And, the spatial information conversion unit 1000 can generate rendering information HR_R and HR_L input to a rendering unit for a right channel of a downmix signal, wherein the rendering information HR_R is delivered to a right output corresponding to the same channel, and The rendering information HR_L is delivered to the left output corresponding to the other channel.

Referring to FIG. 7, the rendering unit 900 includes a rendering unit-1A 911, a rendering unit-2A 912, a rendering unit-1B 921, and a rendering unit-2B 922.

The rendering unit 900 receives the stereo downmix signal and the rendering information from the spatial information converting unit 1000 . Then, the rendering unit 900 generates a surround signal by rendering the rendering information to the stereo downmix signal.

Specifically, the rendering unit-1A 911 performs rendering by using the rendering information HL_L delivered to the same channel among the rendering information for the left channel of the downmix signal. The rendering unit-2A 912 performs rendering by using the rendering information HL_R delivered to the other channel among the rendering information for the left channel of the downmix signal. The rendering unit-1B 921 performs rendering using the rendering information HR_R delivered to the same channel among the rendering information for the right channel of the downmix signal. And the rendering unit-2B 922 performs rendering by using the rendering information HR_L delivered to the other channel among the rendering information for the right channel of the downmix signal.

In the following description, rendering information delivered to another channel is named 'cross rendering information'. The cross-rendering information HL_R or HR_L is applied to the same channel and then added to the other channel by an adder. In this case, cross rendering information HL_R and/or HR_L may be 0. If the cross rendering information HL_R and/or HR_L is 0, it means that there is no contribution to the corresponding path.

An example of the surround signal generation method shown in FIG. 6 or 7 is explained as follows.

First, if the downmix signal is a stereo signal, the downmix signal is defined as 'x', the source map information generated by using spatial information is defined as 'D', the prototype is defined as 'G' The filter information, the multi-channel signal defined as 'p' and the surround signal defined as 'y' may be represented by a matrix as shown in Mathematics 13.

Mathematical Calculus 13

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Li</span>}\\ \mathrm{<span\; class="notranslate">\; Ri</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="L"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="ls"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="ls"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="R"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="C"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="LFE"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="LFE"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}\mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Lo</span>}\\ \mathrm{<span\; class="notranslate">\; Ro</span>}\end{array}\right]$

In this case, if the above values are on the frequency domain, they can be expanded as follows.

First, as shown in Mathematical Calculations 14, a multi-channel signal p can be expressed as a product between source map information D generated by using spatial information and a downmix signal x.

Mathematical Calculus 14

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; x</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; ls</span>}\\ \mathrm{<span\; class="notranslate">\; R</span>}\\ \mathrm{<span\; class="notranslate">\; Rs.</span>}\\ \mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; LFE</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="L"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="ls"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="ls"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="R"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="C"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="LFE"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="LFE"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Li</span>}\\ \mathrm{<span\; class="notranslate">\; Ri</span>}\end{array}\right]$

The surround signal y can be generated by rendering the prototype filter information G to the multi-channel signal p as shown in Mathematical Calculations 15 .

Mathematical Calculus 15

y=G·p

In this case, if the mathematical operation 14 is substituted into p, it can be generated as the mathematical operation 16 .

Mathematical Calculus 16

y = GDx

In this case, if the rendering information H is defined as H=GD, the surround signal y and the downmix signal x may have a relationship of mathematical operation 17 .

Mathematical Calculus 17

$h=\left[\begin{array}{cc}\mathrm{HL}\_L& \mathrm{HR}\_L\\ \mathrm{HL}\_R& \mathrm{HR}\_R\end{array}\right],$ y=Hx

Therefore, after generating the rendering information H by processing the product of the filter information and the source map information, the downmix signal x is multiplied by the rendering information H to generate the surround signal y.

According to the definition of the rendering information H, the rendering information H can be expressed as a mathematical operation 18 .

Mathematical Calculus 18

H=GD

$\left[\begin{array}{cccccc}\mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}\end{array}\right]\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="L"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="ls"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="ls"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">ls</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="R"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="R"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">R</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; Rs.</span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="C"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="C"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">C</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="LFE"\; filenames="S2007800015359E00062.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="LFE"\; filenames="S2007800015359E00031.png,S2007800015359E00101.png"\; state="\{\{state\}\}">LFE</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\end{array}\right]$

8 and 9 are detailed block diagrams of a rendering unit for a mono-channel downmix signal according to an embodiment of the present invention.

Referring to FIG. 8, the rendering unit 900 includes a rendering unit-A 930 and a rendering unit-B 940.

If the channel downmix signal is a mono signal, the spatial information conversion unit 1000 generates rendering information HM_L and HM_R, wherein the rendering information HM_L is used when rendering the mono signal to the left channel, and the rendering information HM_R is Used when rendering this mono signal to the right channel.

The rendering unit-A 930 applies the rendering information HM_L to the mono downmix signal to generate the left channel surround signal. The rendering unit-B 940 applies the rendering information HM_R to the mono downmix signal to generate the right channel surround signal.

The rendering unit 900 in the figure does not use a decorrelator. However, if the rendering unit-A 930 and the rendering unit-B 940 perform rendering by using the rendering information Hmoverall_R and Hmoverall_L defined in Mathematical Calculations 12, respectively, outputs of the applied correlators can be obtained, respectively.

Meanwhile, in the case of attempting to obtain an output of a stereo signal instead of a surround signal after completion of rendering performed on a down-mix signal of mono, the following two methods are possible.

The first is to use the values for stereo output instead of rendering information for surround effects. In this case, a stereo signal can be obtained by only modifying the rendering information in the structure shown in FIG. 3 .

The second method is that in a decoding process for generating a multi-channel signal using a downmix signal and spatial information, a stereo signal may be obtained by performing the decoding process only up to a corresponding step for obtaining a specific channel number.

Referring to FIG. 9 , a rendering unit 900 corresponds to a case in which a decorrelated signal is represented as one, ie, a mathematical operation 11 . The rendering unit 900 includes a rendering unit-1A 931, a rendering unit-2A 932, a rendering unit-1B 941, and a rendering unit-2B 942. Rendering unit 900 is similar to the rendering unit for the stereo downmix signal—except that rendering unit 900 includes rendering units 941 and 942 for the decorrelated signal.

In the case of a stereo downmix signal, one of the two channels may be considered to be a decorrelated signal. Therefore, without employing an additional decorrelator, it is possible to perform a rendering process by using the previously defined four kinds of rendering information HL_L, HL_R, etc. Specifically, the rendering unit-1A 931 generates a signal to be delivered to the same channel by applying the rendering information HM_L to the downmix signal of the mono channel. The rendering unit-2A 932 generates a signal to be delivered to another channel by applying the rendering information HM_R to the mono-channel downmix signal. The rendering unit-1B 941 generates a signal to be delivered to the same channel by applying the rendering information HMD_R to the decorrelated signal. And the rendering unit-2B 942 generates a signal to be delivered to another channel by applying the rendering information HMD_L to this decorrelated signal.

If the downmix signal is a mono signal, the downmix signal is defined as x, the source channel information is defined as D, the prototype filter information is defined as G, and the multichannel is defined as p The signal, and the surround signal defined as y, can be represented by the matrices shown in Mathematical Calculation 19.

Mathematical Calculus 19

x=[Mi], $p=\left[\begin{array}{c}L\\ \mathrm{ls}\\ R\\ \mathrm{Rs.}\\ C\\ \mathrm{LFE}\end{array}\right],$ $\mathrm{D.}=\left[\begin{array}{c}\mathrm{D.}\_L\\ \mathrm{D.}\_\mathrm{ls}\\ \mathrm{D.}\_R\\ \mathrm{D.}\_\mathrm{Rs.}\\ \mathrm{D.}\_C\\ \mathrm{D.}\_\mathrm{LFE}\end{array}\right],$

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}\mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; L</span>}\\ \mathrm{<span\; class="notranslate">\; GL</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GR</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GRs</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}& \mathrm{<span\; class="notranslate">\; GLFE</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; R</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ $\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; Lo</span>}\\ \mathrm{<span\; class="notranslate">\; Ro</span>}\end{array}\right]$

In this case, the relationship between these matrices is similar to that in the case where the downmix signal is a stereo signal. Therefore its details are omitted.

Meanwhile, source map information described with reference to FIGS. 4 and 5 and rendering information generated by using this source map information have different values per frequency band, parameter band, and/or transmission slot. In this case, distortion may occur during rendering if the values of source map information and/or rendering information have a considerable difference between adjacent bands or between boundary slots. In order to prevent this distortion, a smoothing process in frequency domain and/or time domain is required. In addition to frequency domain smoothing and/or time domain smoothing, other smoothing methods suitable for rendering may also be used. Also, a value obtained by multiplying source map information or rendering information by a certain gain may be used.

10 and 11 are block diagrams of a smoothing unit and an expanding unit according to one embodiment of the present invention.

As shown in FIGS. 10 and 11 , the smoothing method according to the present invention can be applied to rendering information and/or source map information. However, this smoothing method can also be applied to other types of information. In the following description, smoothing in the frequency domain is described. However, in addition to smoothing in the frequency domain, the present invention also includes smoothing in the time domain.

Referring to FIGS. 10 and 11 , the smoothing unit 1042 can perform smoothing on rendering information and/or source map information. A detailed example of where smoothing occurs will be described later with reference to FIGS. 18 to 20 .

The smoothing unit 1042 may be configured to work in conjunction with an extension unit 1043 in which rendering information and/or source map information may be extended into a wider range than the parameter band - eg into a filter band. In particular, the source map information may be expanded to a frequency resolution (eg, filter band) corresponding to the filter information to be multiplied by this filter information (eg, HRTF filter coefficients). Smoothing according to the invention is performed before or together with the expansion. Smoothing used with extensions can take one of the methods shown in Figures 12 to 16.

FIG. 12 is a graph for explaining a first smoothing method according to an embodiment of the present invention.

Referring to FIG. 12, the first smoothing method employs values having the same size as spatial information in each parameter band. In this case, a smoothing effect can be achieved by using a suitable smoothing function.

FIG. 13 is a graph for explaining a second smoothing method according to an embodiment of the present invention.

Referring to FIG. 13 , the second smoothing method is to obtain a smoothing effect by connecting representative positions of the parameter bands. The representative position is the exact center of each of the parameter bands, a center position proportional to a logarithmic scale, a Bark scale, or the like. The lowest frequency value, or the location determined in advance by different methods.

FIG. 14 is a graph for explaining a third smoothing method according to an embodiment of the present invention.

Referring to FIG. 14 , the third smoothing method is to perform smoothing in the form of a curve or a straight line that smoothly connects the boundaries of parameters. In this case, the third smoothing method uses a preset boundary smoothing curve or low-pass filtering by a first-order or higher-order IIR filter or FIR filter.

FIG. 15 is a graph for explaining a fourth smoothing method according to an embodiment of the present invention.

Referring to FIG. 15 , the fourth smoothing method is to achieve a smoothing effect by adding a signal such as random noise to the spatial information profile. In this case, values that differ in channels or frequency bands can be used as random noise. In the case of adding random noise on the frequency domain, only the magnitude value can be added while keeping the phase value unchanged. In addition to the smoothing effect in the frequency domain, the fourth smoothing method can also achieve the decorrelation effect between channels.

FIG. 16 is a graph for explaining a fifth smoothing method according to one embodiment of the present invention.

Referring to FIG. 16 , the fifth smoothing method is to use a combination of the second to fourth smoothing methods. For example, after the representative positions of the representative parameter bands have been connected, random noise is added and then low-pass filtering is applied. This allows the sequence to be modified. The fifth smoothing method minimizes the discontinuities in the frequency domain and can enhance the decorrelation effect between channels.

In the first to fifth smoothing methods, the total power of spatial information values (for example, CLD values) on the corresponding frequency domain for each channel should be uniform as a constant. For this, power normalization should be performed after performing the smoothing method per channel. For example, if the channel downmix signal is a mono signal, the level values of corresponding channels should satisfy the relationship of mathematical calculation 20.

Mathematical Calculus 20

D_L(pb)+D_R(pb)+D_C(pb)+D_Ls(pb)+D_Rs(pb)+D_Lfe(pb)＝C

In this case, 'pb=0~total parameter frequency 1', and 'C' is an arbitrary constant.

Fig. 17 is a diagram for explaining prototype filter information per channel.

Referring to FIG. 17 , for rendering, the signal that has passed the GL_L filter for the left channel source is sent to the left output, while the signal that has passed the GL_R filter is sent to the right output.

Subsequently, a left final output (eg Lo) and a right final output (eg Ro) are generated by summing all signals received from respective channels. Specifically, the rendered left/right channel output can be expressed as a mathematical operation 21 .

Mathematical Calculus 21

Lo＝L*GL_L+C*GC_L+R*GR_L+Ls*GLs_L+Rs*GRs_L

Ro＝L*GL_R+C*GC_R+R*GR_R+Ls*GLs_R+Rs*GRs_R

In the present invention, the rendered left/right channel output may be generated by utilizing L, R, C, Ls, and Rs generated by decoding the downmix signal into a multi-channel signal using spatial information. And, the present invention can generate rendered left/right channel output using rendering information generated by using spatial information and filter information without generating L, R, C, Ls, and Rs.

The process of generating rendering information using spatial information is explained with reference to FIGS. 18 to 20 as follows.

FIG. 18 is a block diagram of a first method of generating rendering information in the spatial information converting unit 900 according to an embodiment of the present invention.

Referring to FIG. 18 , as mentioned in the above description, the spatial information conversion unit 900 includes a source mapping unit 1010 , a sub-rendering information generation unit 1020 , an integration unit 1030 , a processing unit 1040 , and a domain conversion unit 1050 . The spatial information conversion unit 900 has the same configuration as that shown in FIG. 3 .

The sub-rendering information generation unit 1020 includes at least one or more sub-rendering information generation units (the first sub-rendering information generation unit to the Nth sub-rendering information generation unit).

The sub-rendering information generating unit 1020 generates sub-rendering information by using filter information and source map information.

For example, if the downmix signal is a mono signal, the first sub-rendering information generation unit can generate sub-rendering information corresponding to the left channel on the multi-channel. And, this sub-rendering information can be represented as a mathematical operation 22 using the source map information D_L and the converted filter information GL_L' and GL_R'

Mathematical Calculus 22

FL_L=D_L*GL_L'

(mono input → filter coefficients to left output channel)

FL_R=D_L*GL_R'

(mono input → filter coefficients to right output channel)

In this case, D_L is a value generated by utilizing spatial information in the source mapping unit 1010 . However, the process of generating D_L may follow a tree structure.

The second sub-rendering information generating unit may generate sub-rendering information FR_L and FR_R corresponding to the right channel on the multi-channel. And, the Nth sub-rendering information generation unit can generate sub-rendering information FRs_L and FRs_R corresponding to the right surround channel on the multi-channel.

If the downmix signal is a stereo signal, the first sub-rendering information generating unit may generate sub-rendering information corresponding to a left channel on a multi-channel. And, this sub-rendering information can be expressed as a mathematical operation 23 by utilizing the source map information D_L1 and D_L2.

Mathematical Calculus 23

FL_L1=D_L1*GL_L'

(left input → filter coefficients to left output channel)

FL_L2=D_L2*GL_L'

(right input → filter coefficients to left output channel)

FL_R1=D_L1*GL_R'

(left input → filter coefficients to right output channel)

PL_R2=D_L2*GL_R'

(right input → filter coefficients to right output channel)

In Math 23, for example, FL_R1 is explained as follows.

First, in FL_R1, 'L' indicates the position of the multi-channel, 'R' indicates the output channel of the surround signal, and '1' indicates the channel of the downmix signal. That is, FL_R1 indicates sub-rendering information used when generating the right output channel of the surround signal from the left channel of the down-mix signal.

Second, D_L1 and D_L2 are values generated by utilizing spatial information in the source mapping unit 1010 .

If the downmix signal is a stereo signal, a plurality of sub rendering information can be generated from at least one sub rendering information generating unit in the same manner as a case where the downmix signal is a monaural signal. The types of the sub-rendering information generated by the plurality of sub-rendering information generating units are exemplary, which does not limit the present invention.

The sub-rendering information generated by the sub-rendering information generation unit 1020 is transmitted to the rendering unit 900 via the integration unit 1030 , the processing unit 1040 , and the domain conversion unit 1050 .

The integration unit 1030 integrates sub-rendering information generated per channel into rendering information (for example, HL_L, HL_R, HR_L, HR_R) for a rendering process. The integration process in the integration unit 1030 in the case of a mono signal and in the case of a stereo signal is explained as follows.

First, if the downmix signal is a mono signal, the rendering information can be expressed as a mathematical operation 24 .

Mathematical Calculus 24

HM_L＝FL_L+FR_L+FC_L+FLs_L+FRs_L+FLFE_L

HM_R＝FL_R+FR_R+FC_R+FLs_R+FRs_R+FLFE_R

Second, if the downmix signal is a stereo signal, the rendering information can be expressed as a mathematical operation 25 .

Mathematical Calculus 25

HL_L＝FL_L1+FR_L1+FC_L1+FLs_L1+FRs_L1+FLFE_L1

HR_L＝FL_L2+FR_L2+FC_L2+FLs_L2+FRs_L2+FLFE_L2

HL_R＝FL_R1+FR_R1+FC_R1+FLs_R1+FRs_R1+FLFE_R1

HR_R＝FL_R2+FR_R2+FC_R2+FLs_R2+FRs_R2+FLFE_R2

Subsequently, the processing unit 1040 includes an interpolation unit 1041 and/or a smoothing unit 1042, and performs interpolation and/or smoothing for rendering information. Interpolation and/or smoothing can be performed in the time domain, frequency domain, or QMF domain. In this specification, the time domain is used as an example, which does not limit the present invention.

If the transmitted rendering information has a wide interval in the temporal domain, interpolation is performed to obtain non-existing rendering information between the rendering information. For example, assuming that rendering information exists in the n-th slot and the (n+k)-th slot respectively, it is possible to use the generated rendering information (for example, HL_L, HR_L, HL_R, HR_R) Perform linear interpolation.

The rendering information generated from the interpolation is explained with reference to the case where the downmix signal is a mono signal and the case where the downmix signal is a stereo signal.

If the downmix signal is a mono signal, the interpolated rendering information can be expressed as a mathematical operation 26 .

Mathematical Calculus 26

HM_L(n+j)=HM_L(n)*(1-a)+HM_L(n+k)*a

HM_R(n+j)＝HM_R(n)*(1-a)+HM_R(n+k)*a

If the downmix signal is a stereo signal, the interpolated rendering information can be expressed as a mathematical operation 27 .

Mathematical Calculus 27

HL_L(n+j)=HL_L(n)*(1-a)+HL_L(n+k)*a

HR_L(n+j)=HR_L(n)*(1-a)+HR_L(n+k)*a

HL_R(n+j)=HL_R(n)*(1-a)+HL_R(n+k)*a

HR_R(n+j)=HR_R(n)*(1-a)+HR_R(n+k)*a

In this case, 0<j<k. 'j' and 'k' are integers. And, 'a' is a real number corresponding to ''0<a<1' which will be expressed as a mathematical operation 28 .

Mathematical Calculus 28

a=j/k

If so, the value corresponding to the non-transmitted slot on the straight line connecting the values in these two slots can be obtained according to the mathematical operation 27 and the mathematical operation 28 . Details of the interpolation will be explained later with reference to FIGS. 22 and 23 .

In the case where the filter coefficient value changes abruptly between two adjacent slots on the time domain, the smoothing unit 1042 performs smoothing to prevent distortion problems due to the occurrence of discontinuous points. Smoothing in the temporal domain can be performed using the smoothing methods described with reference to FIGS. 12 to 16 . Smoothing can be performed together with expansion. Also, smoothing can vary depending on where it is applied. If the downmix signal is a mono signal, temporal smoothing can be expressed as a mathematical operation 29 .

Mathematical Calculus 29

HM_L(n)'=HM_L(n)*b+HM_L(n-1)'*(1-b)

HM_R(n)'=HM_R(n)*b+HM_R(n-1)'*(1-b)

That is, the smoothing can be performed by multiplying the rendering information HM_L(n-1) or HM_R(n-1) that has been smoothed in the previous time slot n-1 by (1-b), and multiplying the rendering information generated in the current time slot HM_L(n) or HM_R(n) is multiplied by b and the 1-pol IIR filter type is performed by adding the results of these two multiplications. In this case, 'b' is a constant of 0<b<1. If 'b' gets smaller, the smoothing effect gets bigger. If 'b' becomes larger, the smoothing effect becomes smaller. And, the rest of the filters can be applied in the same way.

Interpolation and smoothing can be expressed as one expression shown in math 30 by using math 29 for temporal smoothing.

Mathematical Calculus 30

HM_L(n+j)'=(HM_L(n)*(1-a)+HM_L(n+k)*a)*b+HM_L(n+j-1)'*(1-b)

HM_R(n+j)'=(HM_R(n)*(1-a)+HM_R(n+k)*a)*b+HM_R(n+j-1)'*(1-b)

If interpolation is performed by the interpolation unit 1041 and/or if smoothing is performed by the smoothing unit 1042, rendering information having an energy value different from that of the original rendering information may be obtained. To prevent this problem, energy normalization may additionally be performed.

Finally, the domain conversion unit 1050 performs domain conversion on the rendering information for the domain used to perform rendering. This domain conversion may not be performed if the domain used to perform the rendering is the same as that of the rendering information. Afterwards, the domain-converted rendering information is transmitted to the rendering unit 900 .

FIG. 19 is a block diagram of a second method of generating rendering information in a spatial information conversion unit according to an embodiment of the present invention.

The second method is similar to the first method in that the spatial information conversion unit 1000 includes a source mapping unit 1010, a sub-rendering information generation unit 1020, an integration unit 1030, a processing unit 1040, and a domain conversion unit 1050, and in that the sub-rendering information generation Unit 1020 includes at least one sub-rendering information generation unit.

Referring to FIG. 19 , the second method of generating rendering information differs from the first method in the location of the processing unit 1040 . Therefore, sub-rendering information (for example, FL_L and FL_R in the case of mono or FL_L1, FL_L2, FL_R1, FL_R2 in the case of stereo signals) generated per channel in the sub-rendering information generation unit 1020 can be channel ground to perform interpolation and/or smoothing.

Subsequently, the integration unit 1030 integrates the interpolated and/or smoothed sub-rendering information into rendering information.

The generated rendering information is transmitted to the rendering unit 900 via the domain converting unit 1050 .

FIG. 20 is a block diagram of a third method of generating rendering filter information in a spatial information conversion unit according to an embodiment of the present invention.

The third method is similar to the first or second method in that the spatial information conversion unit 1000 includes a source mapping unit 1010, a sub-rendering information generation unit 1020, an integration unit 1030, a processing unit 1040, and a domain conversion unit 1050, and in that the sub- The rendering information generating unit 1020 includes at least one sub-rendering information generating unit.

Referring to FIG. 20 , the third method of generating rendering information is different from the first or second method in that the processing unit 1040 is adjacent to the source mapping unit 1010 . Therefore, interpolation and/or smoothing may be performed per channel on source map information generated by using spatial information in the source map unit 1010 .

Subsequently, the sub-rendering information generation unit 1020 generates sub-rendering information by using the interpolated and/or smoothed source map information and filter information.

The sub-rendering information is integrated into rendering information in the integrating unit 1030 . And, the generated rendering information is transmitted to the rendering unit 900 via the domain conversion unit 1050 .

FIG. 21 is a diagram for explaining a method of generating surround signals in a rendering unit according to one embodiment of the present invention. Figure 21 shows the rendering process performed on the DFT domain. However, the rendering process can also be implemented on different domains in a similar manner. FIG. 21 shows a case where the input signal is a monaural downmix signal. However, FIG. 21 can be applied in a similar manner to other input channels including a stereo downmix signal and the like.

Referring to FIG. 21 , the monaural downmix signal in the time domain is preferably windowed with an overlapping interval OL in the domain conversion unit. Figure 21 shows the case where 50% overlap is used. However, the present invention includes the use of other overlapping situations.

A window function for performing windowing may employ a function having good frequency selectivity in the DFT domain by being seamlessly connected without discontinuity in the time domain. For example, a sine square window function can be used as this window function.

Then, using the rendering information converted in the domain conversion unit, the tap length of the rendering filter is performed on the mono down-mix signal with length OL*2 obtained from the windowing [precisely, is the zero-padding ZL of (tap length)-1]. Then perform a domain transformation into the DFT domain. Fig. 20 shows that the block-k channel downmix signal is domain transformed into the DFT domain.

The domain converted downmix signal is rendered by a rendering filter using the rendering information. The rendering process can be expressed as the product of the downmix signal and the rendering information. The rendered downmix signal undergoes IDFT (Inverse Discrete Fourier Transform) in the domain inverse transform unit and is then compared with the downmix signal previously performed with a delay of length OL (block k-1 in Fig. 20) overlap to create a surround signal.

Interpolation can be performed on every block that goes through this rendering process. The interpolation method is explained as follows.

Figure 22 is a diagram of a first interpolation method according to one embodiment of the present invention. Interpolation according to the invention can be performed at various positions. For example, interpolation may be performed at various positions in the spatial information conversion unit shown in FIGS. 18 to 20 , or may be performed in the rendering unit. Spatial information, source map information, filter information, etc. may be used as values to be interpolated. In this specification, spatial information is exemplarily used for description. However, the present invention is not limited to spatial information. Interpolation is performed before or along with the extension to the wider frequency band.

Referring to FIG. 22, the spatial information transmitted from the encoding device may be transmitted from a random position instead of being transmitted on every slot. One spatial frame can carry multiple sets of spatial information (eg, parameter sets n and n+1 in FIG. 22 ). In the case of low bit rates, one spatial frame can carry a single new set of spatial information. Therefore, the interpolation of the non-transmitted slots is performed using the values of adjacent transmitted spatial information sets. The spacing between windows used to perform rendering does not always match the time slot. So, as shown in Figure 22, find the interpolated value at the center of the rendering window (K-1, K, K+1, K+2, etc.) to use. Although FIG. 22 shows that linear interpolation is performed between time slots in which sets of spatial information exist, the present invention is not limited to this interpolation. For example, false interpolation over time slots where no spatial information set exists. Instead, previous or preset values may be used.

Figure 23 is a diagram of a second interpolation method according to one embodiment of the present invention.

Referring to FIG. 23 , the second interpolation method according to one embodiment of the present invention has a structure combining intervals using previous values, intervals using preset default values, and the like. For example, interpolation may be performed by using at least one of a method of maintaining a previous value, a method of adopting a preset default value, and a method of performing linear interpolation in an interval of one spatial frame. Distortion may occur in situations where there are at least two new sets of spatial information in a window. In the following description, block switching for preventing distortion is explained.

FIG. 24 is a diagram of a block switching method according to one embodiment of the present invention.

Referring to FIG. 24( a ), since the window length is greater than the slot length, there may be at least two spatial information sets in one window interval (eg, parameter sets n and n+1 in FIG. 24 ). In this case, each of the spatial information sets should be applied to a different time slot. However, distortion may occur if a value resulting from interpolating the at least two sets of spatial information is applied. That is, distortion due to insufficient temporal resolution according to the window length may occur.

To solve this problem, a switching method that changes the window size to match the slot resolution can be used. For example, as shown in Fig. 24(b), for an interval requiring high resolution, the window size can be switched to a window with a shorter size. In this case, at the beginning and end of the switched window, a connection window is used to prevent seams from appearing on the time domain of the switched window.

The window length may not be transmitted as separate additional information but instead be determined by using the spatial information in the decoding device. For example, the window length can be determined by using the interval of the time slot for updating the spatial information. That is, if the interval for updating spatial information is narrow, a window function with a short length is used. If the interval used to update the spatial information is wide, use a window function with a long length. In this case, by using a variable length window in the rendering, it is advantageous not to use the bits that transmit the window length information separately. Two types of window lengths are shown in Fig. 24(b). However, windows with various lengths may be used depending on the relationship between transmission frequency and spatial information. The determined window length information can be applied to various steps of generating the surround signal, which will be explained in the following description.

FIG. 25 is a block diagram of a location where a window length determined by a window length determination unit is applied according to one embodiment of the present invention.

Referring to FIG. 25, the window length decision unit 1400 may decide the window length by using spatial information. Information on the decided window length may be applied to the source mapping unit 1010 , the integration unit 1030 , the processing unit 1040 , the domain conversion units 1050 and 1100 , and the domain inverse conversion unit 1300 . FIG. 25 shows a case where a stereo downmix signal is used. However, the invention is not limited to stereo downmix signals. As mentioned in the above description, even if the window length is shortened, the length of zero padding determined according to the number of filter taps is not adjustable. Therefore, a solution to this problem is explained in the following description.

FIG. 26 is a diagram of filters of various lengths used in processing an audio signal according to one embodiment of the present invention. As mentioned in the above description, if the length of zero padding determined according to the number of filter taps is not adjusted, overlays amounting to a corresponding length substantially occur, resulting in insufficient temporal resolution. The solution to this problem is to shorten the length of zero padding by limiting the length of the filter taps. A method of shortening the zero-padding length can be achieved by truncating the tail of the response (eg, corresponding to the diffusion interval of the reverberation). In this case, the rendering process may be less accurate than if the tails of the filter response were not truncated. However, the values of the filter coefficients in the time domain are small, thereby mainly affecting the reverberation. Therefore, sound quality is not significantly affected by truncation.

Referring to Figure 26, there are four filters available. These four filters can be used in the DFT domain, which is not a limitation of the present invention.

Filter-N indicates a filter with a long filter length FL and a long zero-padding length 2*OL that is not limited by the number of filter taps. Filter-N2 indicates a filter having the same filter length FL having a zero-padding length 2*LO shorter than that of Filter-N1 by limiting the number of taps of the filter. Filter-N3 indicates a filter having a shorter filter length FL than Filter-N1 with a long zero-padding length 2*LO by not limiting the number of filter taps. And, Filter-N4 indicates a filter having a short zero-padding length 2*LO having a window length FL shorter than that of Filter-N1 by limiting the number of taps of the filter.

As mentioned in the above description, the above exemplary four filters can be used to solve the problem of temporal resolution. Also, for the tails of the filter response, different filter coefficients may be used for each domain.

FIG. 27 is a diagram of a method of separately processing an audio signal by using a plurality of sub-filters according to one embodiment of the present invention. One filter may be divided into sub-filters having mutually different filter coefficients. After an audio signal is processed by using sub-filters, a method of adding the processed results may be used. In the case of applying spatial information to the tail of the filter response with small energy, i.e. in the case of rendering by using a filter with long filter taps, the method provides for separately processing the audio signal in predetermined length units function. For example, since the tail of the filter does not change significantly for each HRTF corresponding to each channel, rendering may be performed by extracting coefficients common to a plurality of windows. In this specification, the case performed on the DFT domain is described. However, the invention is not limited to the DFT domain.

Referring to FIG. 27, after one filter FL is divided into a plurality of subsections, the subsections can be processed by a plurality of subfilters (filter-A and filter-B) having filter coefficients different from each other.

Subsequently, the Filter-A processed output and the Filter-B processed output are combined. For example, IDFT (Inverse Discrete Fourier Transform) is performed on each of the filter-A processed output and the filter-B processed output to generate a time-domain signal. And, add up the generated signals. In this case, the position where the output processed by Filter-B is added is delayed by FL by more time than the position where the output processed by Filter-A is added. In this way, a signal processed by multiple sub-filters brings about the same effect as if the signal were processed by a single filter.

Also, the present invention includes a method of rendering the Filter-B processed output directly to the downmix signal. In this case, this output can be rendered to the downmix signal by using coefficients extracted from the spatial information, partially using the spatial information, or without using the spatial information.

The method is characterized in that a filter with a long number of taps can be applied separately, and a filter tail with a small energy can be applied without conversion using spatial information. In this case, if no transformation utilizing spatial information is applied, then a different filter is not applied to each processed window. Therefore, it is not necessary to apply the same scheme as block switching. Figure 26 shows a filter divided into two regions. However, the present invention may divide the filter into multiple regions.

FIG. 28 is a block diagram of a method of rendering split rendering information generated by a plurality of sub-filters to a mono downmix signal according to an embodiment of the present invention. Figure 28 relates to a rendering factor. This method may be performed per render factor.

Referring to FIG. 28 , the filter-A information of FIG. 27 corresponds to the first split rendering information HM_L_A, and the filter-B information of FIG. 27 corresponds to the second split rendering information HM_L_B. Figure 28 shows an embodiment of splitting into two sub-filters. However, the invention is not limited to these two sub-filters. These two sub-filters may be obtained by using the rendering information HM_L generated in the spatial information generating unit 1000 via the splitting unit 1500 . Alternatively, the two sub-filters can be obtained using prototype HRTF information or information determined by user selection. The information determined according to the user's selection may include, for example, spatial information selected according to the user's preference. In this case, HM_L_A is rendering information based on the received spatial information, while HM_L_B may be rendering information for providing a 3D effect typically applied to the signal.

As mentioned in the above description, processing with multiple sub-filters is applicable not only to the DFT domain, but also to the time domain and QMF domain. Specifically, the coefficient values split by Filter-A and Filter-B are applied to the downmix signal by time domain or QMF domain rendering, and then added to generate the final signal.

The rendering unit 900 includes a first split rendering unit 950 and a second split rendering unit 960 . The first split rendering unit 950 performs a rendering process using HM_L_A, and the second split rendering unit 960 performs a rendering process using HM_L_B.

If Filter-A and Filter-B as shown in FIG. 27 are divisions of the same filter according to time, an appropriate delay can be considered to correspond to this time interval. FIG. 28 shows an example of a mono downmix signal. In the case of using a mono downmix signal and a decorrelator, the part corresponding to Filter-B is not applied to the decorrelator, but directly to the mono downmix Signal.

FIG. 29 is a block diagram of a method for rendering split rendering information generated using a plurality of sub-filters to a stereo downmix signal according to an embodiment of the present invention.

The split rendering process shown in FIG. 29 is similar to the process of FIG. 28 in that two subdivisions are obtained in the splitter 1500 by using the rendering information generated by the spatial information conversion unit 1000, the prototype HRTF filter information, or the user decision information. filter. The difference from FIG. 28 is that the split rendering process corresponding to Filter-B is commonly applied to the L/R signal.

Specifically, the splitter 1500 generates first split rendering information corresponding to filter-A information, second split rendering information, and third split rendering information corresponding to filter-B information. In this case, the third split rendering information may be generated by using filter information or spatial information commonly applicable to L/R signals.

Referring to FIG. 29 , the rendering unit 900 includes a first split rendering unit 970 , a second split rendering unit 980 , and a third split rendering unit 990 .

The generated third split rendering information is applied to the sum signal of the L/R signals in the third split rendering unit 990 to generate an output signal. This output signal is added to the L/R output signal which is independently rendered by Filter-A1 and Filter-A2 in the first and second split rendering units 970 and 980 respectively to generate the surround signal. In this case, the output signals of the third split rendering unit 990 may be summed after an appropriate delay. The expression of applying cross-rendering information from the L/R input to another channel is omitted in FIG. 29 for convenience of explanation.

Fig. 30 is a block diagram of a first domain conversion method of a downmix signal according to an embodiment of the present invention. The rendering process performed on the DFL domain has been described so far. As mentioned in the above description, besides the DFL domain, the rendering process can also be performed on other domains. However, Fig. 30 shows a rendering process performed on the DFT domain. The domain conversion unit 1100 includes a QMF filter and a DFL filter. The domain inverse transform unit 1300 includes an IDFT filter and an IQMF filter. Fig. 30 relates to a mono channel downmix signal, which does not constitute a limitation of the present invention.

Referring to FIG. 30 , the time-domain downmix signal with p samples is passed through a QMF filter to generate P subband samples. W additional samples are taken per frequency band. Zero padding is performed after windowing is performed on the reacquired samples. An M-point DFT (FFT) is then performed. In this case, DFT achieves processing through windowing of the type described above. A value that connects M/2 frequency-domain values per frequency band obtained by M-point DFT to P frequency bands can be regarded as an appropriate value of the frequency spectrum obtained by M/2*P-point DFT. So, multiplying this spectrum by the filter coefficients expressed on the M/2*P point DFT domain brings about the same effect as rendering on the DFT domain.

In this case, the signal that has passed through the QMF filter has leakage, eg, aliasing between adjacent frequency bands. Specifically, values corresponding to adjacent frequency bands bleed into the current frequency band, and values present in the current frequency band are shifted into adjacent frequency bands. In this case, if QMF integration is performed, the original signal can be restored due to QMF characteristics. However, if the filtering process is performed on the signal of the corresponding frequency band as is the case in the present invention, the signal is distorted due to this leakage. To minimize this problem, the procedure for recovering the original signal can be increased by passing the signal through a leakage-minimizing butterfly filter after QMF in the domain transformation unit 100 before performing DFT per band and inverse transforming in the domain The IDFT in unit 1300 is followed by the inverse process.

At the same time, in order to match the generation process of the rendering information generated in the spatial information conversion unit 1000 with the generation process of the channel downmix signal, instead of performing the M/2*P point DFT at the beginning, the DFT is performed on the signal of the QMF to obtain the prototype filter information. In this case there may be delay and data spreading caused by the QMF filter.

Fig. 31 is a block diagram of a second domain conversion method for a downmix signal according to an embodiment of the present invention. Figure 31 shows the rendering process performed on the QMF domain.

Referring to FIG. 31 , the domain conversion unit 1100 includes a QMF domain conversion unit, and the domain inverse conversion unit 1300 includes an IQMF domain conversion unit. The configuration shown in FIG. 31 is the same as the case of using only DFT—only the domain-dividing transform unit is a QMF filter. In the following description, QMF refers to QMF and hybrid QMF having the same bandwidth. The difference from the case of using only DFT is that the generation of rendering information is performed on the QMF domain, and this rendering process is expressed as a convolution rather than a product on the DFT domain, because this rendering process performed by the renderer-M 3012 is executed on the QMF domain.

Assuming that a QMF filter is provided with B frequency bands, the filter coefficients can be expressed as a set of filter coefficients having different characteristics (coefficients) for the B frequency bands. Sometimes, if the number of filter taps becomes first order (ie, multiplied by a constant), the rendering process with B spectra on the DFT domain is matched with the operational process. Mathematical calculation 31 represents a rendering process performed on one QMF band (b) for a path in which the rendering process is performed using the rendering information HM_L.

Mathematical Calculus 31

$\mathrm{<span\; class="notranslate">\; Lo</span>}<span\; class="notranslate">\; \_</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; H\; M</span>}<span\; class="notranslate">\; \_</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; filter</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; order</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; hm</span>}<span\; class="notranslate">\; \_</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In this case, k indicates timing in the QMF band, ie, slot unit. An advantage of the rendering process performed on the QMF domain is that if the transmitted spatial information is a value applicable to the QMF domain, the application of the corresponding data is most convenient and the distortion during application can be minimized. However, in the case of QMF domain conversion during conversion of prototype filter information (eg, prototype filter coefficients), the process of applying the converted values requires a considerable amount of computation. In this case, the amount of computation can be minimized by parameterizing HRTF coefficients during filter information conversion.

Industrial Applicability

Therefore, the signal processing method and apparatus of the present invention use spatial information provided by an encoder to generate surround signals by using HRTF filter information or according to user's filter information in a decoding apparatus capable of generating multi-channels. Also, the present invention is well suited for various decoders that can only reproduce stereo signals.

While the present invention has been described and illustrated with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and variations can be made therein without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover all such modifications and variations of this invention that come within the scope of the appended claims and their equivalents.

Claims (15)

1. the method for a processing signals, described method comprises:

From bit stream, extract down-mix audio signal;

Generate down-mix audio signal by decorrelator being applied to described down-mix audio signal through decorrelation; And

By will play up information be applied to described down-mix audio signal and described down-mix audio signal through decorrelation generate have surrounding effect around signal.

2. the method for claim 1 is characterized in that, described application of playing up information is carried out on one of time domain, frequency domain, QMF territory and hybrid domain.

3. the method for claim 1 is characterized in that, the described information of playing up is to be used for the filter information of described surrounding effect and the spatial information that extracts from described bit stream generates by use.

4. the method for claim 1 is characterized in that, comprises also that territory with described down-mix audio signal is converted to generate described territory around signal therein.

5. method as claimed in claim 4 is characterized in that, described territory of playing up information equals the territory of generation around signal.

6. the method for claim 1 is characterized in that, described decorrelator has all-pass characteristics.

7. the method for claim 1 is characterized in that, described down-mix audio signal is a monophonic signal.

8. the device of a processing signals, described device comprises:

Correlated elements, it generates down-mix audio signal through decorrelation by the down-mix audio signal that decorrelator is applied to extract from bit stream; And

Rendering unit, its by will play up information be applied to described down-mix audio signal and described down-mix audio signal through decorrelation generate have surrounding effect around signal.

9. device as claimed in claim 8 is characterized in that, described rendering unit generates described around signal on one of time domain, frequency domain, QMF territory and hybrid domain.

10. device as claimed in claim 8 is characterized in that, the described information of playing up is to be used for the filter information of described surrounding effect and the spatial information that extracts from described bit stream generates by use.

11. device as claimed in claim 8, it is characterized in that the described information of playing up comprises that being applied to down-mix audio signal sound channel and in the down-mix audio signal sound channel of decorrelation one first plays up information and be applied to described down-mix audio signal sound channel and described in the down-mix audio signal sound channel of decorrelation one transmits then second plays up information on another sound channel what transmit on the same sound channel then.

12. device as claimed in claim 8 is characterized in that, also comprises the territory converting unit, the territory of described down-mix audio signal is converted to generates described territory around signal therein.

13. device as claimed in claim 12 is characterized in that, described territory of playing up information equals the territory of generation around signal.

14. device as claimed in claim 8 is characterized in that, described decorrelator has all-pass characteristics.

15. device as claimed in claim 8 is characterized in that, described down-mix audio signal is a monophonic signal.

Images