KR100891669B1 - Apparatus for processing an medium signal and method thereof - Google Patents

Abstract

The present invention relates to a method and apparatus for processing a mixed signal. To this end, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Generating a second source signal similar to the first source signal when there is no first source signal specific to the mix signal; Generating additional information by using the second source signal, wherein the additional information indicates a relationship between a source signal included in the mix signal and a source signal to be remixed among the second source signals and the mix signal; It provides a signal processing method characterized in that.

Source signal, remix signal, additional information

Description

Method and apparatus for processing mixed signal {APPARATUS FOR PROCESSING AN MEDIUM SIGNAL AND METHOD THEREOF}

1 is a block diagram of an apparatus for encoding a first remix signal according to an embodiment of the present invention.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when using a stereo signal. FIG.

3 is a domain for processing a media signal according to an embodiment of the present invention.

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention.

FIG. 6 is a detailed view of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. FIG.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention.

8B is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention used in combination with a conventional decoding apparatus.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention.

10 is a flowchart illustrating a method of generating additional information for each source signal according to an embodiment of the present invention.

11 is a block diagram illustrating a decoding apparatus for generating additional information for each source signal according to an embodiment of the present invention.

12 is a block diagram illustrating a signal processing apparatus for replacing a specific source signal according to an embodiment of the present invention.

13 is a flow diagram illustrating a method for replacing a particular source signal in accordance with one embodiment of the present invention.

14 is a block diagram of an encoding apparatus according to an embodiment of the present invention.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention.

16 is a diagram illustrating an internal structure of a mix signal modification unit according to an exemplary embodiment of the present invention.

17 is a diagram illustrating a signal processing method using a source signal providing server according to an embodiment of the present invention.

18 is a flowchart illustrating a method of modifying a mix signal according to an embodiment of the present invention.

19 is a flowchart illustrating a method of modifying a mix signal according to an embodiment of the present invention.

The present invention relates to a method and apparatus for processing a mix signal. To date, stereo signals are most commonly generated as mix signals and most widely used by consumers. In recent years, multichannel signals have been increasingly used. However, there is a limitation that the mixed signal is processed in the channel signal unit, not in the source signal unit constituting the mix signal. Therefore, when processing the mixed signal in the channel signal unit, there is a problem that only a specific source signal constituting the mixed signal can not be processed independently. For example, while watching a movie, it is impossible to increase the volume of background music while keeping the volume of actors' voices constant. In addition, since the mix signal is processed on a channel basis, it is impossible to replace the source signal included in the mix signal with a specific source signal provided by the user.

In order to solve the above problems, the present invention provides a method for generating additional information for each source signal by using a signal similar to the source signal when the source signal included in the mixed signal does not exist separately. There is this.

Another object of the present invention is to provide a method and apparatus for generating additional information for remixing the mixed signal using the mixed signal in a decoding apparatus.

Another object of the present invention is to provide a method and apparatus for replacing a specific source signal included in a mixed signal with a source signal provided by a user.

In addition, an object of the present invention is to provide a signal processing method and apparatus that allows a user to modify a mix signal using a source signal.

In order to achieve the above object, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Generating a second source signal similar to the first source signal if a particular first source signal of the source signals is not independently present; Generating additional information by using the second source signal, wherein the additional information indicates a relationship between a source signal included in the mix signal and a source signal to be remixed among the second source signals and the mix signal; It provides a signal processing method characterized in that.

In addition, to achieve the above object, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Obtaining a user mix parameter; Generating additional information using the mix signal; Generating a remix signal using the mix signal, the user mix parameter, and the additional information, wherein the additional information includes a source signal to be remixed among the source signals included in the mix signal and the mixed signal. It provides a signal processing method characterized in that the relationship.

In addition, to achieve the above object, the present invention comprises the steps of obtaining a first mix signal comprising one or more first source signals; Acquiring a second source signal; And generating a second mix signal using the first mix signal and the second source signal.

In addition, to achieve the above object, the present invention provides a source signal providing method performed by a source signal providing server coupled to a user terminal through a communication network, the method comprising: obtaining selection information of a source signal from the user terminal; Generating additional information about the selected source signal according to the selection information; And transmitting the selected source signal and the additional information to the user terminal, wherein the additional information is information used to generate a remix signal using the source signal.

In addition, in order to achieve the above object, the present invention provides a mixed signal decoding unit for obtaining a mixed signal including at least one source signal; An additional information generator configured to generate additional information using a signal component representing a specific space included in the mixed signal; And a remix rendering unit configured to generate a remix signal using the mix signal, the mix parameter, and the additional information, wherein the additional information includes: a source signal to be remixed among the source signals included in the mix signal; It provides a signal processing apparatus characterized in that the relationship between.

In addition, in order to achieve the above object, the present invention includes a remix rendering unit for generating a first remix signal by adjusting a specific first source signal included in the mix signal; A source signal generator configured to generate a second source signal which is not included in the mix signal; And a recording unit configured to generate a second remix signal using the first remix signal and the second source signal.

In addition, in order to achieve the above object, the present invention provides a mixed signal decoding unit for obtaining a first mixed signal including at least one first source signal; A source signal decoding unit obtaining a second source signal not included in the first mix signal; And a mix signal modifying unit configured to generate a second mix signal using the first mix signal and the second source signal, wherein the second mix signal is selected by a user from the first source signal and the second source signal. Provided is a signal processing apparatus comprising only a source signal.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

The present invention provides an algorithm that can process a mixed signal in an object signal unit. When the mix signal is processed in units of source signals, a wide variety of effects can be produced. For example, while watching a movie, it is possible to increase only the volume of the background music while keeping the volume of the actors' voices constant. In the present invention, the source signal includes one or more sources (eg, pianos) constituting the mix signal. Processing in source signal units means that in processing a mix signal, characteristics (eg, localization, gain) associated with a particular source signal constituting the mix signal are " individually ". It can be modified. By “individually” it is meant that modifying the characteristics associated with a particular source does not affect the characteristics of other source signals, or only a small effect that is difficult to perceive. For convenience of explanation, hereinafter, a stereo signal will be described as an example, but the present invention is not limited to the stereo signal. In the present invention, a small amount of side information is transmitted as well as a typical mix signal format (for example, PCM, MP3, MPEG-AAC). The remix signal may be generated using the mixed signal and the additional information.

1 is a block diagram of a first remix signal encoder according to an embodiment of the present invention. The first remix signal encoder includes an additional information generator 103 and an additional information encoder 105.

Referring to FIG. 1, the additional information generator 103 generates additional information 104 using a typical mix signal 101 and a source signal 102 constituting the mixed signal. The mix signal 101 may be a mono, stereo, and multi-channel audio signal. The source signal 102 may be some or all of the source signals constituting the mix signal 101. The additional information 104 refers to information used to process the mixed signal in source signal units. The additional information 104 includes mix parameters for remixing the mix signal. The mix parameter may include an encoder mix parameter generated using a source signal in an encoder, and optionally, a blind mix parameter generated using only a mix signal. Examples of the mix parameter may include a gain value and a subband power for each source signal. A detailed definition and generation method for the additional information 104 is described in FIG. 2. The present invention also includes generating the additional information 104 using only the source signal 102 constituting the mix signal. The side information encoding unit 105 encodes the generated side information 104 to generate an encoded side information signal 106. The mix signal 101 and the side information signal 106 are transmitted to a decoding apparatus.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when the mixed signal is a stereo signal. As described above, the mixed signal used in the present invention may be a mono, stereo, and multi-channel audio signal, but for convenience, it will be described with reference to the stereo signal 201.

및

는 상기 스테레오 신호를 구성하는 소스 신호들의 합으로 표현될 수 있다. 여기서, n은 타임 인덱스를 의미한다. 따라서, 상기 스테레오 신호(201)는 아래의 [수학식 1]과 같이 표현될 수 있다.The stereo signal 201

And

May be expressed as a sum of source signals constituting the stereo signal. Here, n means time index. Therefore, the stereo signal 201 may be expressed as Equation 1 below.

은 소스 신호들을 나타낸다.

및

는 각각의 소스 신호에 대한 진폭 패닝(amplitude panning) 및 게인(gain)을 결정하는 값이다. 모든

들은 서로 독립적이다. 상기

는 모두 순수한 소스 신호이거나, 또는 순수한 소스 신호에 약간의 잔향(reverberation) 및 효과음 신호성분(sound effect signal components)을 포함할 수 있다. 예를 들면, 특정한 잔향 신호성분은 2개의 소스 신호, 즉, 왼쪽 채널로 믹스된 신호와 오른쪽 채널로 믹스된 신호로 표현될 수 있다.Here, I is the number of source signals included in the stereo signal,

Represents source signals.

And

Is a value that determines the amplitude panning and gain for each source signal. all

They are independent of each other. remind

Are all pure source signals, or may include some reverberation and sound effect signal components in the pure source signal. For example, the specific reverberation signal component may be represented by two source signals, that is, a signal mixed with the left channel and a signal mixed with the right channel.

It is an object of the present invention to modify a stereo signal comprising the source signal such that M (0 <= M <= I) source signals are remixed. The source signals may be remixed into a stereo signal with different gain factors. The remix signal may be expressed as Equation 2 below.

및

는 리믹스되는 M개의 소스 신호들에 대한 새로운 게인 팩터들이다. 상기

및

는 디코더 단에서 제공될 수 있다. 이 경우에, 부가정보생성부(206)는 스테레오 신호(201) 및 M개의 소스 신호(202)를 이용하여 부가 정보(207)를 생성할 수 있다. here,

And

Are new gain factors for the M source signals to be remixed. remind

And

May be provided at the decoder stage. In this case, the additional information generator 206 may generate the additional information 207 using the stereo signal 201 and the M source signals 202.

에 대한 접근 없이, [수학식 1]로 표현되는 통상적인 믹스 신호가 주어지는 경우에, [수학식 2]로 표현되는 리믹스 신호를 지각적으로 모방하는 것을 목적으로 한다.As described above, an object of the present invention is to remix the stereo signal in the unit of the source signal when given a conventional stereo signal and some additional information. As in the present invention, it is not possible to completely generate the remix signal represented by Equation 2 from the mixed signal represented by Equation 1 using a very small amount of additional information. Thus, the present invention provides the respective source signals

It is aimed to perceptually mimic the remix signal represented by [Equation 2], given an ordinary mix signal represented by [Equation 1] without access to.

및

로 표시된다. 여기서 k는 서브밴드 신호들의 시간 인덱스(time index)이다. 유사하게, M개의 소스 신호들(202)의 서브밴드 신호들(205)은

,

,...,

로 표시된다. 명료한 표현을 위해, 서브밴드(주파수) 인덱스를 사용하지 않았다.Referring to FIG. 2, a stereo signal 201 and M source signals 202 included in the stereo signal 201 are input to a first remix signal encoding apparatus. The stereo signal 201 may be delayed to some extent to be synchronized with the side information and used directly as an output signal. To generate the additional information, the stereo signal 201 and the source signals 202 are decomposed into subband-specific signals 204 and 205 in the time-frequency domain through the filter bank 203. That is, the stereo signal 201 and the source signal are processed in the time-frequency domain, which will be described later with reference to FIG. 3. The subband-specific signal 204 is similarly processed at the center frequency of each subband. At a particular frequency, the subband pair 204 of the stereo signal 201

And

Is displayed. Where k is the time index of the subband signals. Similarly, the subband signals 205 of the M source signals 202 may be

,

, ...,

Is displayed. For clarity, no subband (frequency) index is used.

를 생성한다. 또한, 상기 부가정보생성부(206)는 스테레오 신호(201)의 서브밴드 쌍(204)을 이용하여, 서브밴드별로 게인 팩터

및

를 생성한다. 상기 게인 팩터

및

는 외부에서 직접 주어질 수 있다. 상기 서브밴드별 숏-타임 서브밴드 파워 및 게인 팩터를 이용하여 서브밴드별 부가 정보(207)가 생성된다. 상기 부가정보생성부(206)는 상기 숏-타임 서브밴드 파워 및 게인 팩터들 이외에 상기 스테레오 신호에 관련된 다른 정보를 부가 정보(207)로 생성할 수 있다. 부가정보인코딩부(208)는 상기 서브밴드별 부가정보(207)를 이용하여 부호화된 부가 정보 신호(209)를 생성한다. Given the subband signals 205 of the source signals 202, the side information generator 206 may perform short-time subband power for each subband,

Create In addition, the additional information generator 206 uses a subband pair 204 of the stereo signal 201 to obtain a gain factor for each subband.

And

Create The gain factor

And

Can be given directly from the outside. Subband-specific additional information 207 is generated using the short-time subband power and gain factor for each subband. The additional information generator 206 may generate other information related to the stereo signal as additional information 207 in addition to the short-time subband power and gain factors. The additional information encoding unit 208 generates an additional information signal 209 encoded using the additional information 207 for each subband.

및

는 고정적이 될 것이다. 만일

및

가 시간 k에 따라 가변적이라면, 상기 게인 팩터들은 시 간의 함수로 생성될 것이다. 상기 게인 팩터들은 직접 양자화 및 부호화되지 않고, 먼저 양자화 및 부호화에 더 적합한 다른 값들로 전환될 수 있다. 또한,

는 스테레오 신호(201)의 서브밴드 파워에 상대적인 값으로 정규화될 수 있다. 이것은 스테레오 신호를 효율적으로 부호화하기 위해 통상적인 인코딩 장치가 이용되는 경우에, 본 발명을 상대적으로 변화에 강하도록 만들어준다. 예를 들면,

및

는 아래의 [수학식 3]으로 표현되는 게인 및 데시벨(dB) 단위의 레벨차로 전환되어 전송될 수 있다. For many stereo signals 201, the gain factor

And

Will be fixed. if

And

If is variable over time k, the gain factors will be generated as a function of time. The gain factors are not directly quantized and coded, but may first be converted to other values more suitable for quantization and coding. Also,

Can be normalized to a value relative to the subband power of the stereo signal 201. This makes the present invention relatively resistant to changes when conventional encoding devices are used to encode stereo signals efficiently. For example,

And

May be converted into a level difference in units of gain and decibel (dB) expressed by Equation 3 below, and then transmitted.

는 부가 정보로서 직접 부호화되는 것이 아니라, 아래의 [수학식 4]로 표현되는 스테레오 신호에 상대적으로 정의된 값으로 변환되어 전송될 수 있다. Also,

Is not directly encoded as additional information, but may be converted into a value defined relative to the stereo signal represented by Equation 4 below and transmitted.

는 아래의 [수학식 5]와 같이 계산될 수 있다. To produce short-time subband power, the present invention uses single-pole averaging. In other words,

May be calculated as shown in Equation 5 below.

Here, α∈ [0,1] determines the time-constant of the estimation window (estimation window) which decreases exponentially as shown in Equation 6 below.

는 서브밴드 샘플링 주파수를 나타낸다. 예를 들면, T=40 ms를 이용할 수 있다. 이하에서,

는 숏-타임 평균(short-time averaging)을 나타낸다. 만일

및

가 주어지지 않는다면, 상기

및

는 부가정보생성부(206)에서 생성될 필요가 있다.

이므로,

는 아래의 [수학식 7]과 같이 계산된다.here,

Denotes a subband sampling frequency. For example, T = 40 ms can be used. In the following,

Denotes short-time averaging. if

And

If is not given,

And

Needs to be generated in the additional information generation unit 206.

Because of,

Is calculated as shown in Equation 7 below.

는 아래의 [수학식 8]과 같이 계산된다.Similarly,

Is calculated as shown in Equation 8 below.

3 illustrates a domain for processing an audio signal according to an embodiment of the present invention. As mentioned above, the audio signal and the additional information are processed as subband-specific signals in the time-frequency domain as shown in FIG. Subband-specific signals in the time-frequency domain are perceptually derived. For example, a signal for each subband may be generated by using a short time fourier transform (STFT) having a sine wave analysis window and a sine analysis and synthesis window having a length of about 20 ms. In this case, the STFT coefficients may be grouped such that one group has a bandwidth that is about twice the equivalent rectangular bandwidth (ERB).

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention. The second remix signal encoding apparatus includes a downmixing unit 402, an additional information generating unit 403, and an additional information encoding unit 406.

Referring to FIG. 4, the downmixing unit 402 generates a sum signal 404 by adding a plurality of source signals 401. Unlike the first remix signal encoding apparatus, the second remix signal encoding apparatus transmits the sum signal 404 instead of the stereo signal. The additional information generator 403 generates the additional information 405 using the source signals 401. The additional information 405 includes subband power and gain factor corresponding to each source signal. In addition, the additional information 405 may include a parameter corresponding to a delay in the remix renderer. Similar to the first remix signal encoding apparatus, the additional information 405 may be converted into another value more suitable for quantization and encoding and transmitted. The additional information encoding unit 406 generates an encoded additional information signal 407 using the generated additional information 405. The generated sum signal 404 and the side information signal 407 are transmitted to the decoding apparatus. The present invention also includes an encoding apparatus that does not have a downmixing unit 402. In this case, the source signals 401 are not converted to the sum signal 404, and each source signal 401 is transmitted directly.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention. The first remix signal decoding apparatus includes a side information decoder 503 and a remix renderer 505.

Referring to FIG. 5, the mix signal 501 and the additional information signal 502 are input to the first remix signal decoding apparatus. The mix signal 501 may be a mono, stereo or multichannel audio signal. The additional information decoding unit 503 decodes the additional information signal 502 to generate additional information 504. The additional information 504 includes a gain factor and a subband power of source signals included in the transmitted audio signal 501. The remix renderer 505 may input a user-mix parameter 506 generated using control information directly provided by the user. The remix renderer 505 generates the remix signal 507 using the mix signal 501, the transmitted additional information 504, and the user mix parameter 506. A detailed description of the method of generating the remix signal will be described later with reference to FIG. 6. The remix signal 507 is generated as an Eq-channel mix signal having the same number of channels as the number of channels of the transmitted mix signal, or an upchannel mix signal having more channels than the number of channels of the mix signal. (Up-channel mix signal) can be generated.

FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. As described above, the transmitted mixed signal may be a mono, stereo, and multichannel audio signal, but for convenience, the stereo signal 601 will be described.

및

로 표현된다. 부가정보디코딩부(605)는 전송된 부가 정보 신호(602)를 복호화하여, 서브밴드별 부가 정보(606)를 생성한다. 또한, 리믹스 렌더링부(607)에 사용자가 제공하는 제어 정보를 이용하여 생성된 사용자 믹스 파라미터(608)가 입력될 수 있으며, 상기 사용자 믹스 파라미터(608)는 서브밴드별로 제공될 수 있다. 전술한 것처럼, 상기 부가 정보(606)는 리믹스 될 M개의 소스 신호에 대한 서브밴드별 게인 팩터(

및

) 및

로 표현되는 서브밴드 파워를 포함한다. 리믹스 렌더링부(607)는 서브밴드별로 생성된 스테레오 신호(604), 전송된 부가 정보(606) 및 사용자 믹스 파라미터(608)를 이용하여, 서브밴드별 리믹스 신호(609),

및

를 생성한다. 상기 리믹스 신호(609)를 생성하는 방법은 아래에서 더욱 상세하게 기술된다. 상기 리믹스 신호(609)는 역필터뱅크(610)를 통해 시간 도메인(time domain)의 스테레오 신호(611),

및

로 변환된다. Referring to FIG. 6, the stereo signal 601 is decomposed into a subband-specific signal 604 in the time-frequency domain through the filter bank 603. As shown in FIG. 6, the subband-specific signal 604 at a particular frequency

And

It is expressed as The additional information decoding unit 605 decodes the transmitted additional information signal 602 to generate additional information 606 for each subband. In addition, a user mix parameter 608 generated using control information provided by a user may be input to the remix renderer 607, and the user mix parameter 608 may be provided for each subband. As described above, the additional information 606 is a subband-specific gain factor for M source signals to be remixed.

And

) And

It includes the subband power represented by. The remix rendering unit 607 may use the stereo signal 604 generated for each subband, the additional information 606 transmitted, and the user mix parameter 608, and the remix signal 609 for each subband,

And

Create The method of generating the remix signal 609 is described in more detail below. The remix signal 609 is a stereo signal 611 in the time domain through the inverse filter bank 610,

And

Is converted to.

는 서브밴드별 소스 신호

로 교체된다. 즉, 서브밴드별 믹스 신호(604)는 아래의 [수학식 9]과 같이 표현될 수 있다. A method of generating the remix signal 609 generated by the remix renderer 607 is as follows. [Equation 1] and [Equation 2] are also valid for the subband signals 604 and 609. In this case, the source signal

Is the subband-specific source signal

Is replaced by. That is, the mix signal 604 for each subband may be expressed by Equation 9 below.

The subband remix signal 609 may be expressed by Equation 10 below.

및

가 주어지면, 아래의 [수학식 11]와 같이 서로 다른 게인들을 가지는 서브밴드별 리믹스 신호(609)가 상기 서브밴드별 믹스 신호(604)의 선형 조합으로 추정될 수 있다.To generate the remix signal 609, least squares estimation may be used. Per subband mix signal 604,

And

Given by Equation 11 below, the subband remix signal 609 having different gains may be estimated as a linear combination of the subband mix signals 604.

,

,

및

는 가중 팩터들(weighting factors)이다. 이때, 생성되는 추정 에러(estimation error)는 아래의 [수학식 12]과 같이 정의될 수 있다.here,

,

,

And

Is the weighting factors. In this case, the generated estimation error may be defined as shown in Equation 12 below.

,

,

및

는 평균제곱오차(mean square error),

및

가 최소가 되도록 서브밴드별로 생성될 수 있다. 이때, 추정 에러,

및

가

및

에 직교(orthogonal)될 때, 상기 평균제곱오차가 최소가 된다는 것을 이용할 수 있다. 생성되는

및

는 아래의 [수학식 13]과 같이 표현될 수 있다.The weighting factors,

,

,

And

Is the mean square error,

And

May be generated for each subband such that is minimized. In this case, the estimation error,

And

end

And

When orthogonal to, it can be used that the mean square error is minimized. Generated

And

May be expressed as Equation 13 below.

,

및

는 직접 생성될 수 있지만,

및

은 전송된 부가 정보(606)(예를 들면,

,

,

) 및 사용자가 제공하는 제어 정보(608)(예를 들면, 게인 팩터

및

)를 이용하여, 아래의 [수학식 14]와 같이 생성될 수 있다.here,

,

And

Can be generated directly,

And

Is transmitted additional information 606 (e.g.,

,

,

) And control information 608 provided by the user (e.g., gain factor

And

) Can be generated as shown in Equation 14 below.

및

가 아래의 [수학식 15]와 같이 생성될 수 있다. Similarly,

And

May be generated as shown in Equation 15 below.

및

는 아래의 [수학식 16]과 같이 표현될 수 있다. here,

And

May be expressed as Equation 16 below.

If the phases of the mix signal 604 are coherent or nearly synchronized with each other, a value expressed by Equation 17 below approaches 1.

In this case, the weights may be expressed as Equation 18 below.

및

)를 이용하여 각각의 소스 신호를 독립적으로 리믹스하여 생성된 리믹스 신호와 유사하게 들린다.The subband remix signal 609 generated as described above is converted into the remix signal 611 of the time-domain through the inverse filter bank 610 as described above. The remix signal 611 may generate a user mix parameter generated using control information provided by a user.

And

It sounds similar to the remix signal generated by independently remixing each source signal with

So far, the focus has been on remixing two-channel stereo signals. However, as described above, the present invention is not limited to stereo signals, but may be extended to remixing multichannel audio signals, for example, 5.1 channel audio signals. Those skilled in the art can remix multichannel audio signals, similar to the stereo signals described herein. In this case, Equation 11 may be written as Equation 19 below.

Optionally, one of the channels of the mix signal can be left without remixing. For example, for a 5.1 surround channel, it is possible to apply remixing only to the front channel without modifying the two back channels. In this case, a two or three channel remix algorithm is applied to the front channel.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. The second remix signal decoding apparatus includes an additional information decoding unit 703, a spatial information integration unit 705, and a remix rendering unit 707.

Referring to FIG. 7, the sum signal 701 of the source signals and the additional information signal 702 are input to the second remix signal decoding apparatus. The additional information decoding unit 703 decodes the additional information signal 702 to generate additional information 704. The additional information 704 includes a gain factor, a delay constant, a subband power, and the like. The additional information integrator 705 separates the sum signal 701 into a plurality of source signals 706 by using the additional information 704. The remix renderer 707 may generate the remix signal 709 using the source signals 706. In this case, the remix renderer 707 may generate the remix signal 709 using the mix parameter transmitted as additional information. In addition, the remix renderer 707 may selectively generate the remix signal 709 using the user mix parameter 708 generated by using control information provided by the user.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. The mixed signal 801 may be encoded by the conventional encoding apparatus 803 and converted into the encoded mixed signal 805. The mix signal 801 may be a channel-specific signal or a source signal. The conventional encoding device 803 includes not only conventional encoding devices such as AAC, MP3 encoder, etc., but also encoding devices to be developed in the future. The remix signal encoding apparatus 804 according to the present invention generates the additional information signal 806 using the mixed signal 801 and the source signal 802 included in the mixed signal. The multiplexer 807 generates the bitstream 808 using the encoded mix signal 805 and the side information signal 806. As described above, the additional information signal 806 may be inserted into an auxiliary data area in a conventional mix signal format to be compatible with conventional devices.

8B is a block diagram of a combination of a conventional decoding device and a remix signal decoding device according to an embodiment of the present invention. The demultiplexer 810 separates the encoded mix signal 811 and the side information signal 812 from the transmitted bitstream 809. The conventional decoding device 813 then decodes the encoded mix signal 811 to produce a mix signal 814 that can be used in the remix signal decoding device 815 according to the present invention. The conventional decoding device 813 includes not only a conventional encoding device such as an AAC, an MP3 decoder, etc. but also an encoding device to be developed in the future. The mix signal 814 may be a channel-specific signal or a source signal. The remix signal decoding apparatus 817 according to the present invention may convert the mix signal 814 into the remix signal 816 using at least one of the additional information signal 812 and the user mix parameter 817.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention. Referring to FIG. 9, the remix signal decoding apparatus includes a mixed signal decoder 901, a parameter generator 902, and a remix renderer 908. It may optionally include an effector (911). The parameter generator 902 may include a blind mix parameter generator 903, a user mix parameter generator 904, and a remix parameter generator 905. The remix parameter generation unit 905 may include an equimix parameter generation unit 906 and may optionally include an upmix parameter generation unit 907. In addition, the remix renderer 908 may include an equimix renderer 909 and optionally include an upmix renderer 910.

The mixed signal decoding unit 901 decodes the encoded mixed signal transmitted from the encoding end to generate a mixed signal. The parameter generator 902 receives additional information and user control information (or configuration information) transmitted from an encoding stage. The user control information is not transmitted at the encoder stage but may be generated at the decoder stage. The user mix parameter generator 904 generates a user mix parameter by using user control information. The additional information transmitted from the encoder stage may include an encoder mix parameter. In addition, the blind mix parameter generator 903 may generate a blind mix parameter using the mix signal. The encoder mix parameter and the blind mix parameter are alternatively input to the remix parameter generator 905.

The remix parameter generator 905 generates a remix parameter using additional information and a user mix parameter. The remix parameter may be generated to be applied to a channel of the remix signal. Equimix parameter generation unit 906 included in the remix parameter generation unit 905 generates a remix parameter used to generate a remix signal having the same channel number as the number of channels of the mix signal, the remix parameter generation unit 905 The upmix parameter generation unit 907, which may be included in the control unit, generates a remix parameter used to generate a remix signal having a larger number of channels than the number of channels of the mixed signal. The remix parameter is input to the remix renderer 908.

Equimix renderer 909 included in the remix renderer 908, using the remix parameter and the mix signal, having an equal number of channels equal to the number of channels of the mix signal (Eq-channel remix signal ) The upmix renderer 910, which may be included in the remix renderer 908, uses more channels than the number of channels of the mix signal by using the remix parameter and the mix signal generated by the upmix parameter generator 907. Generate an up-channel remix signal having a number. The upmix renderer 910 may generate an upchannel remix signal using the remix signal generated by the Equichannel renderer 909.

Accordingly, the decoding apparatus may output the mixed signal transmitted from the encoding stage as it is, output as an equal channel remix signal, or output as an up channel remix signal. Optionally, the remix renderer may give various effects to the remixed signal by using information provided from the effector 911.

10 is a flowchart illustrating a method of generating additional information according to an embodiment of the present invention. In order to generate additional information in the encoding apparatus, a separate source signal is required. However, in many mix signals, there are cases where a source signal included in the mix signal does not exist separately. In this case, additional information may be generated using a signal similar to the source signal included in the mix signal. The additional information includes parameters such as a gain value, subband power, and the like.

Referring to FIG. 10, when a mix signal is input (S1001), it is determined whether a separate source signal exists independently of the mix signal (S1002). If a separate source signal exists, additional information is generated using the source signal (S1004). If a separate source signal does not exist, it is determined whether a MIDI file for the mixed signal exists (S1003). If the MIDI file exists, additional information may be generated using the MIDI file (S1006). For example, a specific source signal (for example, a piano sound) may be generated using a MIDI file, and additional information may be generated using the specific source signal.

If no MIDI file exists, additional information can be generated using the following methods. The first method is to generate additional information by using an instrument using a range similar to a specific source signal (S1005). For example, in the case of a vocal, additional information may be generated by using an instrument using a range similar to the vocal. The second method is to generate additional information by using a sound component representing a specific space of the mixed signal (S1005). For example, in the case of a vocal, a sound component representing a center space of a mixed signal may be analyzed and considered as a vocal, and additional information may be generated using the sound component. The third method is to combine the first and second methods. That is, in addition to using a musical instrument using a sound range similar to a specific source signal, additional information is generated using a sound component representing a specific space in the mixed signal (S1005). Thereafter, the mix signal file and the additional information may be used to generate a mix signal file (S1007).

11 is a block diagram illustrating a decoding apparatus for generating additional information according to an embodiment of the present invention. The decoding apparatus includes a mixed signal decoder 1102, an additional information generator 1104, and a remix renderer 1106. The decoding device may generate additional information from the mix signal.

Referring to FIG. 11, the mix signal 1101 is input to the mix signal decoding unit 1102. The mix signal decoding unit 1102 reproduces the mix signal 1103 using the mix signal 1101. The additional information generator 1104 may generate the additional information 1105 using the mix signal 1103. Various methods may be used to generate the additional information 1105. In the first method, as described above, additional information may be generated by using a sound component representing a specific space among the mixed signals. The second method may generate additional information by using spatial decomposition of audio signals (SDAS). The third method can generate additional information by introducing a linear predictive coding (LPC) analysis method. For example, the LPC analysis method can be used to model a speech signal and generate additional information from the speech. The additional information 1105 generated using the above method is input to the remix renderer 1106. The remix renderer 1106 may generate the remix signal 1107 using the additional information 1105 and control information 1108 directly provided by the user.

12 is a block diagram illustrating a signal processing apparatus for replacing a specific source signal according to an embodiment of the present invention. The signal processing apparatus may include a remix renderer 1202, a controller 1203, a playback unit 1204, a source signal generator 1205, a memory 1208, a recording unit 1209, and a recording medium 1211. Can be. In FIG. 12, the playback unit 1204 is illustrated as a speaker, but the present invention is not limited to the illustrated embodiment.

Referring to FIG. 12, the mix signal 1201 is input to the remix renderer 1202. The remix renderer 1202 generates a remix signal using the additional information, the mix signal, and the user mix parameter. The user mix parameter is generated using control information obtained from a user. For example, the remix signal may be generated by lowering or muting a gain of a specific source signal (for example, a vocal signal) included in the mixed signal.

The controller 1203 may output the remix signal through the speaker 1204. At this time, during the process of outputting the remix signal, a new source 1206 may be input through the source signal generator 1205. The new source 1206 may be provided by a user or by using a separate device. Next, the source signal generated by the source signal generator 1205 may be stored in the memory 1208. The recording unit 1209 may generate a new remix signal 1210 using the remix signal and the new source signal stored in the memory 1208. For example, a remix signal may be generated by muting a piano signal included in a mix signal, and a new remix signal may be generated by inputting a piano signal generated by a user playing during a process of outputting the remix signal. In addition, the piano signal generated by the user may be stored in a memory and then synthesized with the remixed signal to be used to generate a new remixed signal. The new mix signal 1210 is stored in the recording medium 1211 and output to the outside 1212. The recording medium 1211 may be included in the signal processing apparatus according to the present invention or may exist independently.

13 is a flowchart illustrating a method of replacing a specific source signal according to an embodiment of the present invention. Referring to FIG. 13, first, a gain of a specific source signal (hereinafter, referred to as a “first source signal”) included in a mix signal is adjusted to generate a first remix signal (S1301). Then, while reproducing the first remixed signal, a new source signal (hereinafter referred to as a "second source signal") is generated, the second source signal is synthesized to the first remixed signal (S1302), and the second A remix signal can be generated. In this case, the second source signal may be stored in a separate memory and later synthesized with the first remix signal. Then, the second remix signal is stored in the recording medium (S13203).

14 is a block diagram of an encoding apparatus according to an embodiment of the present invention. Referring to FIG. 10, the encoding apparatus may include a first additional information generator 1403, a first additional information encoder 1404, a second additional information generator 1406, and a second additional information encoder 1407. Include. The encoding device transmits the mix signal 1401 to the decoding device. The encoding apparatus may directly transmit the mixed signal 1401 to the decoding apparatus, but when there are a plurality of the mixed signals 1401, the mixed signal 1001 may be downmixed into one or two downmix signals and transmitted. Although not shown in the figure, the mix signal 1401 may be quantized and encoded and transmitted for the purpose of the mix signal transmission efficiency.

)(1402)와 상기 믹스 신호에 포함되지 않는 소스 신호(

)(1405)로 구분될 수 있다. 예컨대, 믹스 신호(1401)에 포함되어 있는 소스 신호(1402)는 드럼과 베이스 신호이고, 믹스 신호(1401)에 포함되어 있지 않은 소스 신호(1405)는 보컬1, 보컬2, 보컬3 및 피아노와 바이올린 신호라고 가정한다. 이때, 제1 부가정보생성부(1403)는 믹스 신호(1401) 및 믹스 신호에 포함되어 있는 드럼 및 베이스 신호(1402)를 이용하여 드럼 및 베이스 신호를 조절하기 위한 제1 부가 정보를 생성한다. 이하, 믹스 신호(1401)에 포함되어 있는 소스 신호(1402)에 대한 부가 정보를 제1 부가 정보라 하고, 믹스 신호(1401)에 포함되어 있지 않은 소스 신호(1405)에 대한 부가 정보를 제2 부가 정보라 한다.The source signal is a signal included in the mix signal 1401 (

1402 and a source signal (not included in the mix signal)

1405). For example, the source signal 1402 included in the mix signal 1401 is a drum and bass signal, and the source signal 1405 not included in the mix signal 1401 is a vocal 1, vocal 2, vocal 3, and piano. Assume it is a violin signal. In this case, the first additional information generator 1403 generates the first additional information for adjusting the drum and the bass signals using the mix signal 1401 and the drum and bass signals 1402 included in the mix signal. Hereinafter, the additional information about the source signal 1402 included in the mix signal 1401 is called first additional information, and the additional information about the source signal 1405 not included in the mix signal 1401 may be referred to as second information. Referred to as additional information.

The second additional information generator 1406 generates the second additional information by using the vocals 1 to 3 and the piano and violin signals. The second additional information is information for adjusting the source signal 1405 not included in the mix signal 1401 for each source signal. The first additional information and the second additional information include a gain factor, power or delay constant for each subband, and the like. The first additional information encoder 1404 encodes the first additional information generated by the first additional information generator 1403 and transmits the encoded first additional information to the decoding apparatus. The second additional information encoder 1407 encodes the second additional information generated by the second additional information generator 1406 and transmits the encoded second additional information to the decoding apparatus. The source signal 1405 not included in the mix signal 1401 and the second additional information thereof may be generated outside of the encoding apparatus.

That is, the source signal 1405 not included in the mix signal 1401 and the second additional information thereof may be generated in a source signal providing server separate from the encoding apparatus and provided to the decoding apparatus, or directly by the user. May be generated. The decoding apparatus may generate a new mix signal using the source signal 1402 included in the mix signal 1401 and the source signal 1405 not included in the mix signal. In the above example, the user selects only the base signal from the source signal 1402 included in the mix signal 1401 according to taste, and the vocals 1 and the vocal 1 among the source signals 1405 not included in the mix signal 1401. You can select only piano sounds to create new mix signals that include bass signals, vocal 1, and piano sounds.

The user may receive each of the source signals 1405 not included in the mix signal 1401 from the source signal providing server and the decoding device. The user may generate a new mix signal by using each source signal 1405 separately transmitted.

In addition, in a control such as attenuating or boosting a specific source signal to a high level such as a karaoke mode, there may be a limitation in processing with only a small amount of additional information. For example, it is not possible to mute a vocal signal completely for karaoke functions, or lowering the vocal signal by more than 20 dB can affect other source signals, significantly damaging sound quality.

Accordingly, in order to maximize the degree of control over a specific source signal, a method of extracting more detailed information about the corresponding source signal and additionally including the information in the bit string may be transmitted. For example, the second additional information generator 1406 illustrated in FIG. 14 may generate an enhanced object parameter (EOP) (hereinafter, referred to as “extended additional information”) corresponding to additional additional information. The extended additional information may be a bit string encoding information of a time domain or a spectral domain for a specific source signal such as a residual signal, or encoding an envelope having a higher time or frequency resolution than the existing additional information. It may be a bit string.

When the extension additional information has a form of a residual signal, the extension additional information may be used as a signal for additionally transmitting a source signal that does not exist in the original mix signal. For example, if a movie sound track is transmitted as a residual signal instead of being included in a downmix signal in a specific language-specific dialogue track, it may be generated and included in the decoding step. In this case, it is possible to ensure higher sound quality than when removing and adding to the signal present in the mix signal. In addition, the source signal providing the extended additional information may provide basic additional information or may not provide basic additional information.

If the extended additional information exists in the bitstream, the extended additional information may be sent to the remix rendering unit. The remix renderer may obtain the desired result by applying the extended additional information to the mix signal. For example, if the extended side information is a vocal signal and the user issues a vocal signal removal command for karaoke, the remix rendering unit does not affect the other source signals by using the extended side information for the vocal signal included in the mix signal. Can be removed with minimal or minimal impact.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention. Referring to FIG. 11, the decoding apparatus includes a mix signal transform unit 1503, an additional information transform unit 1507, and a remixing unit 1511. The decoding apparatus extracts the mix signal 1501 received from the encoding apparatus or previously stored, and a source signal 1102 not included in the mix signal 1501. The mix signal modifying unit 1503 transforms the mix signal 1501 using the mix signal 1501 and the source signal 1502 not included in the mix signal.

That is, the mix signal modifying unit 1503 receives the source signal selection information 1508 from the user, and the user may select one of the source signal included in the mix signal 1501 and the source signal 1502 not included in the mix signal 1501. Generates a new mix signal 1504 containing only the selected source signal. The additional information transformation unit 1507 receives the source signal selection information 1508 from the user and generates additional information 1509 for adjusting the new mix signal 1104 for each source signal. Hereinafter, the additional information about the new mix signal 1504 is called third additional information 1509. The additional information transformation unit 1507 may include first additional information 1505 for adjusting the mix signal 1501 for each source signal and second additional information 1506 for adjusting a source signal 1502 not included in the mix signal. ) May generate third additional information 1509 for the new mix signal 1504.

In addition, the additional information transformation unit 1507 may directly generate the third additional information 1509 using the mix signal 1501 and the source signal 1502 not included in the mix signal 1501. The remixing unit 1511 receives control information 1510 from a user, receives a mixed signal 1504 from the mix signal modifying unit 1503, and receives a new mix signal 1504 from the additional information modifying unit 1507. Third additional information 1509 is received. The remixing unit 1511 generates the remixed mix signal 1512 using the control information 1510, the modified mix signal 1504, and the third additional information 1509.

Although not shown in FIG. 11, the decoding apparatus may include an icon processor that may extract an icon representing a source signal, modify the icon, or generate a new icon. The user can use icons representing each source signal when adjusting the source signal included in the mix signal. The icon may be an image representing an instrument or a singer's face of a source signal, or may include text describing an instrument name or the like. The user may adjust the source signal by using an icon for the source signal transmitted by the encoding apparatus. In addition, the user may modify the icon for the source signal transmitted by the encoding apparatus as desired. In addition, the user may access the server providing the source signal and receive an icon for the source signal to use the same. In addition, the user may directly generate an icon for the source signal and use it in conjunction with the source signal.

16 is a diagram illustrating an internal structure of a mix signal modifying unit according to an embodiment of the present invention. Referring to FIG. 16, the mix signal modifying unit 1603 includes a source signal extracting unit 1601, a controller 1602, and a signal modifying unit 1603. The source signal extractor 1601 extracts a source signal. In this case, the source signal includes a source signal included in the mix signal and a source signal not included in the mix signal. The controller 1602 receives the source signal selection information 1608 from the user. The controller 1602 extracts the source signal selected by the user from the source signal extractor 1601 and sends it to the signal modifier 1603. The signal modifier 1603 generates a new mix signal using the source signal selected by the user. That is, the signal modulator 1603 excludes a specific source signal among the source signals included in the mix signal from the new mix signal according to a user's selection, and selects a specific source signal among the source signals not included in the mix signal. Include in the mix signal to create a new mix signal that contains the source signal of your choice. The user can select the desired source signal according to taste and transform the original mix signal into a new mix signal.

17 is a diagram illustrating a signal processing method using a source signal providing server according to an embodiment of the present invention. Referring to FIG. 17, a user accesses a source signal providing server 1701 using a user terminal 1702. The user terminal 1702 includes a device that can access a server through a communication network such as a user PC, a mobile phone, a PDA, a PMP, and the like. The source signal providing server 1701 includes a source signal that is not included in the mix signal but can be reproduced with the mix signal. The source signal providing server 1301 includes a source signal that is synchronized to be played with the mix signal because it has the same time signature, speed, and the like as the mix signal. For example, when playing the November rain on a drum, bass, guitar, or piano, each source signal is played with the same pace and beat for the same song. In this case, if only the drum and bass signals among the source signals are included in the mix signal, the user may access the source signal providing server and download a guitar or piano signal playing a song called November rain.

The user can play the downloaded guitar or piano signal together with the original mix signal. The user accesses the source signal providing server 1701 and selects a desired song. The source signal providing server 1701 displays a list of source signals currently included for the song desired by the user. The user selects a desired source signal from among source signals displayed by the source signal providing server 1701. The source signal providing server 1701 extracts the source signal selected by the user and transmits the source signal to the user terminal 1702. In some cases, the source signal providing server 1701 may generate additional information about the source signal selected by the user and transmit the additional information to the user terminal 1702 together with the source signal. The user may generate a new mix signal from the source signal received from the source signal providing server 1701 and the source signal included in the original mix signal using the decoding apparatus.

The user may receive the source signal from the source signal providing server 1701, but may directly generate a source signal for a specific song. The user may generate a new mix signal by modifying the original mix signal using the source signal included in the mix signal and the generated source signal. The source signal providing server 1701 may include an icon for the source signal in addition to the source signal. The user may access the source signal providing server 1701 and download and use an icon for the source signal. In addition, the user may modify the icon for the source signal received from the source signal providing server 1701.

18 is a flowchart illustrating a method of modifying a mix signal according to an embodiment of the present invention. Referring to FIG. 18, the decoding apparatus extracts a mix signal received from an encoding apparatus or previously stored. The decoding apparatus extracts a source signal included in the mix signal (S1701). Hereinafter, a source signal included in the mix signal is called a first source signal, and a source signal not included in the mix signal is called a second source signal. The decoding apparatus extracts the second source signal received from the encoding apparatus or received from the source signal providing server 1701 or generated directly by the user (S1702). The decoding apparatus transforms the mix signal using the first source signal and the second source signal (S1703). That is, the decoding apparatus generates a new mix signal including only the source signal selected by the user among the first source signal and the second source signal.

19 is a flowchart illustrating a method of modifying a mix signal according to an embodiment of the present invention. Referring to FIG. 19, the decoding apparatus extracts a source signal (S1901). The source signal includes a first source signal included in the mix signal and a second source signal not included in the mix signal. The decoding apparatus receives a source signal from the user (S1502). The decoding apparatus generates a new mix signal using the source signal selected by the user (S1503). The decoding apparatus removes source signals that the user does not want from the source signals included in the original mix signal, and adds the source signal that the user wants from the source signals not included in the mix signal to the mix signal. Create a new mix signal containing only. The decoding apparatus generates third additional information about the new mix signal in order to adjust the new mix signal for each source signal (S1904).

The decoding apparatus may generate third additional information about the new mixed signal by using the first additional information about the mixed signal and the second additional information about the source signal not included in the mixed signal. Also, the decoding apparatus may generate third additional information without using the first and second additional information. That is, the decoding apparatus may generate a new mix signal and then directly generate third additional information about the new mix signal by using the generated mix signal. The decoding apparatus receives control information from the user (S1905). The decoding apparatus generates a remixed mix signal capable of controlling the new mix signal for each source signal using the control information received from the user and the third additional information (S1906).

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

As described above, in processing the mixed signal according to the present invention, additional information having a low bit rate may be used to remix specific source signals included in the mixed signal. The signal processing method according to the present invention provides an effect of generating a parameter for each source signal from a mix signal in which no separate source signal exists. In addition, the decoding apparatus provides an effect of generating a parameter for each source signal using the mixed signal. In addition, it provides an effect of generating a mixed signal synthesized by the source signal generated by the user.

Claims (24)

Obtaining a mix signal comprising one or more source signals; Generating a second source signal similar to the first source signal if a particular first source signal of the source signals is not independently present; Generating additional information by using the second source signal; The additional information indicates a relationship between the source signal included in the mix signal and the source signal to be remixed among the second source signal and the mix signal. The method of claim 1, And the second source signal is generated using a MIDI file. The method of claim 1, The second source signal is a signal processing method, characterized in that generated using the instrument sound similar to the first source signal Obtaining a mix signal comprising one or more source signals; Obtaining a user mix parameter; Generating additional information using the mix signal; And Generating a remix signal using the mix signal, the user mix parameter, and the additional information; The additional information indicates a relationship between the source signal to be remixed and the mix signal among the source signals included in the mix signal. The method of claim 4, wherein generating the additional information And generating the signal using a signal component representing a specific space of the mixed signal. The method of claim 5, wherein generating the additional information If the specific space is a center space, generating a vocal signal using the signal component representing the center space, and generating additional information using the vocal signal. The method of claim 4, wherein The mix parameter is generated using the control information obtained from a user. Obtaining a first mix signal comprising one or more first source signals; Obtaining a second source signal; And And generating a second mix signal by using the first mix signal and the second source signal. The method of claim 8, wherein generating the second mix signal comprises: Adjusting one or more source signals of the first source signals included in the first mix signal; And And generating the second mix signal using the adjusted first mix signal and the second source signal. The method of claim 9, The adjusted first mixed signal is generated by lowering the gain of the first source signal. The method of claim 10, The second source signal is a signal processing method, characterized in that the signal corresponding to the first source signal. The method of claim 8, And the second source signal is not included in the first mix signal. The method of claim 12, wherein the signal processing method is Obtaining source signal selection information from a user, And the second mixed signal includes only a source signal selected by the user from among the first source signal and the second source signal, according to the source signal selection information. The method of claim 13, wherein the signal processing method is Generating side information for remixing the second mixed signal; Obtaining a user mix parameter; And And generating a remix signal using the second mix signal, the additional information, and the user mix parameter. 15. The method of claim 14, wherein the signal processing method is Obtaining first additional information about the first mixed signal and second additional information about the second source signal; The additional information for remixing the second mixed signal is generated using the first additional information and the second additional information. The method of claim 14, wherein the signal processing method is Obtaining control information from a user, The user mix parameter is generated using the control information. The method of claim 16, The control information is a signal processing method, characterized in that the user is generated using an icon representing the first source signal or the second source signal. The method of claim 17, And the icon is an icon transmitted from an encoding device, modified from an icon transmitted from an encoding device, or an icon generated by the user. The method of claim 8, And the second source signal is the same as one or more of the first source signals included in the first mix signal. 20. The method of claim 19, wherein generating the second mix signal And generating a second mixed signal by removing the first source signal identical to the second source signal from the first mixed signal using the second source signal. In the source signal providing method performed in the source signal providing server coupled to the user terminal through a communication network, Obtaining selection information of a source signal from the user terminal; Generating additional information about the selected source signal according to the selection information; And Transmitting the selected source signal and the additional information to the user terminal, Wherein the additional information is information used to generate a remix signal using the source signal. A mix signal decoding unit obtaining a mix signal including one or more source signals; An additional information generator configured to generate additional information using a signal component representing a specific space included in the mixed signal; And Remix rendering unit for generating a remix signal using the mix signal, the mix parameter, and the additional information, And the additional information indicates a relationship between the source signal to be remixed and the mix signal among the source signals included in the mix signal. A mix signal decoding unit obtaining a first mix signal including one or more first source signals; A source signal decoding unit obtaining a second source signal; And And a mix signal modifying unit configured to generate a second mix signal using the first mix signal and the second source signal. The apparatus of claim 23, wherein the signal processing apparatus And an additional information transformation unit configured to generate additional information for remixing the second mixed signal by using the first additional information on the first mixed signal and the second additional information on the second source signal. Signal processing device.

Images