JP2006279555A - Signal regeneration apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a signal regeneration apparatus and a method, capable of positioning a clear virtual sound image at an optimal position in a regeneration sound field. <P>SOLUTION: In a signal regeneration apparatus 1, a sound image information extraction processor 24 separates music sounds of each sound source included in an audio data. Also, a sound source signal calculation processor 26 calculates a control parameter for shifting a sound source located near a listening point to a position in which a listener has a sensitive distance perception, and calculates a control parameter for relocating the sound image positioned remotely to a farther position. Further, by resynthesizing a particular sound source signal in a virtual sound source position calculated in a virtual sound source calculation processor 25, the presence of a virtual sound field is made remarkable. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a signal reproduction device and a signal reproduction method, and more particularly to a signal reproduction device and a signal reproduction method for localizing a sound image to a virtual sound source position by a multi-channel method and a sound field synthesis technique.

  With the development of information transmission forms such as large-capacity storage media, content distribution by download or stream over the network, and the evolution of playback device performance, video content and music content can be enjoyed at home with high image quality and high sound quality. became.

  In a playback device for video data, music data, etc., the sense of reality and sound quality are relatively easy for the user to judge. For example, when a user listens to orchestra music, it is preferable that the position of each instrument is clearly felt in the virtual sound field, and an image as if a real orchestra is playing in front of the eyes is recalled. Users tend to desire more realistic sound reproduction, and how the sound field when sound or music is recorded can be reproduced faithfully when the sound is reproduced. It becomes important because it will end up.

  For example, in 2-channel stereo, the balance of each signal channel of 2-channel stereo signal consisting of L signal and R signal is adjusted and output from two speakers so that the sound image of the reproduction sound field is localized at the optimum place as a virtual sound image. ing. However, in the case of 2-channel stereo, the virtual image has a drawback that the sound image is not clear and the localization position of the sound image changes when the listening position deviates from the center of the left and right speakers. Therefore, in order to ensure that a clearer sound image is localized at the center position of at least two speakers installed on the left and right sides, that is, the front position of the listener, and the sound image remains in the center even if the listening position is shifted left and right. There are a three-channel stereo system in which a center speaker is added to the center of left and right two-channel speakers, and a 5.1-channel stereo system in which a rear speaker is further added.

  On the other hand, a conventional two-channel stereo signal must be able to be reproduced by a multi-channel stereo device such as a three-channel or 5.1-channel device. For this purpose, a technique has been disclosed that can convert a 2-channel stereo signal into a multi-channel signal by generating a center channel signal or a rear L channel signal and a rear R channel signal from a two-channel stereo signal. (See Patent Document 1).

  In addition to this, as a technique for reproducing a more realistic sound field, a technique for reproducing a 5.1 channel stereo signal by synthesizing a point sound source or a plane wave using a sound case forming technique (non-synthesizing) Patent Document 1), and a technique for arbitrarily moving a sound image position by using a virtual sound source position control technique of sound case formation technology is disclosed (see Non-Patent Document 2). Furthermore, there is a general harmonic signal analysis technique for extracting a specific signal from a mixed acoustic signal (see Non-Patent Document 3).

Special table 2003-516069 gazette Spatial Sound in the Age of Fast Convolution Technologies, ICA2004, P.I-515 to P.I-518) MONITORING DISTANCE EFFECT WITH WAVE FIELD SYNTHESIS, Proc. Of the 6th Int. Conference on Digital Audio Effects (DAFX-03), London, UK, September 8-11, 2003) Separation of Acoustic Signals by General Harmonic Analysis Proceedings of the Acoustical Society of Japan P607-608 September 1996

  However, although the method of synthesizing the center channel signal from the two-channel stereo signal is simple, a clear sound image cannot be obtained. In addition, the general harmonic signal analysis technique described in Non-Patent Document 3 extracts a sound source signal included in a conventional two-channel stereo signal, and produces many independent signal channels for each sound source. is required. In addition, it may be necessary to repeat the process of verifying the signal processing results of the generated channel and changing the processing conditions again, so that the general harmonic signal analysis processing is performed in real time on the 2-channel stereo signal. It was difficult to re-synthesize the detection results to generate a multi-channel stereo signal and reproduce it in real time.

  Therefore, the present invention has been proposed in view of the above-described conventional situation, and an object of the present invention is to provide a signal reproducing apparatus and method capable of localizing a clear virtual sound image at an optimal position in a reproduced sound field. And

  In order to achieve the above-described object, a signal reproduction apparatus according to the present invention includes a multi-channel data acquisition unit that acquires multi-channel data in a signal reproduction apparatus that reproduces multi-channel data in which audio signals are multiplexed, Signal analysis means for extracting sound source information and sound image position information of a specific sound image from channel data, sound source signal calculation means for calculating a sound source signal from sound source information, and specifying after changing the sound image information of the extracted specific sound image Sound field generating means for performing sound field generation in which a sound source signal of a sound image is arranged at an arbitrary virtual sound source position is provided, and a clear virtual sound image is localized at an optimum position in the reproduced sound field.

  Here, the signal reproduction device according to the present invention includes virtual sound source position calculating means for calculating the virtual sound source position of the specific sound image according to the extracted sound source information and sound image position information of the specific sound image. The means places the sound source signal calculated by the sound source signal calculating means at the virtual sound source position calculated by the virtual sound source calculating means.

  In order to achieve the above-described object, a signal reproduction method according to the present invention includes a multichannel data acquisition step of acquiring multichannel data in a signal reproduction method of reproducing multichannel data in which audio signals are multiplexed. , A signal analysis step of extracting sound source information and sound image position information of a specific sound image from multi-channel data, a sound source signal calculation step of calculating a sound source signal from the sound source information, and changing the sound image information of the extracted specific sound image A sound case generation step for performing sound case formation in which a sound source signal of a specific sound image is arranged at an arbitrary virtual sound source position.

  According to the signal reproducing device and the signal reproducing method of the present invention, it is possible to generate a sound when a clear virtual sound image is localized at an optimum position in the reproduced sound field. Also, by separating and preparing sound source information and sound image position information of a specific sound image from multi-channel data in which audio signals are multiplexed, the processing load related to sound field analysis and sound case formation processing can be reduced.

  A signal reproducing apparatus shown as a specific example of the present invention is a multi-channel method, and for music data including a plurality of sound source information, a sound source signal extraction including a general harmonic signal analysis technique and a sound case forming technique are performed. It is a playback device that performs signal processing to extract a specific sound source signal from an audio signal and localize the sound image of this sound source signal to an optimal virtual sound source position.

  In the present invention, the data in which the audio signal is multiplexed includes, for example, a plurality of sound sources such as music data played by a plurality of musical instruments and music data including “performance” and “singing voice”. Audio data including audio data determined in linear PCM (Linear Pulse Code Modulation), Dolby Digital, DTS (Digital Theater Systems), DVD-Audio format, and DVD-Video format. Audio data that can be synthesized (multiplexed) with multiple parts of data created in advance for each channel (for each sound source), such as data that conforms to the Digital Interface) standard, is included in the present invention and specific examples of the present invention. I can't.

  Hereinafter, a signal reproducing apparatus 1 shown as a first specific example of the present invention will be described in detail with reference to the drawings. The signal reproduction device 1 is a reproduction device that reproduces content data including audio data input from a recording medium.

  A signal reproducing apparatus 1 shown in FIG. 1 separates the read data into compressed audio data, video data, caption data, other data, and the like, and an optical disk reproducing unit 11 that reads the recorded data by rotating the optical disk. And a signal separation circuit 12. Also, an audio decoder 13 for decoding the compressed audio data, an audio signal processing circuit 14 for reproducing the decoded audio data and generating a sound according to the position information of the virtual sound source, and decoding the compressed subtitle data A subtitle decoder 15 and a subtitle reproduction circuit 16 for reproducing the decoded subtitle. In addition, the video decoder 17 that decodes the compressed video data, the video signal reproduction circuit 18 that reproduces the decoded video data, the caption synthesis circuit 19 that synthesizes the caption in synchronization with the video signal, and the caption are synthesized. And a video signal output circuit 20 for outputting the video signal to the outside.

  The output from the audio decoder 13 and the input to the sound source signal calculation processing unit 26 and the audio signal reproduction processing unit 23 corresponding thereto, and the output from the synthesis processing unit 27 and the corresponding multi-channel amplifier 21 are output. For the input and output to the audio signal output circuit 22, signal lines corresponding to the number of channels are prepared.

  As audio data and video data, formats such as MPEG1, MPEG2, AVI, WMV, and WMA are applicable. In the example described below, a DVD (Digital Versatile Disc) is used as an optical disk. In this case, video data is MPEG2 standardized by the Moving Picture Experts Group (MPEG), and linear PCM, Dolby Digital, DTS, SDDS, etc. are applied as audio data.

  The audio signal processing circuit 14 includes an audio signal reproduction processing unit 23 that reproduces audio data decoded by the audio decoder 13, and a sound image information extraction process that performs processing of extracting sound image information from the reproduction signal reproduced by the optical disc reproduction unit 11. Unit 24, virtual sound source position calculation processing unit 25 for calculating a virtual sound source position, sound source signal calculation processing unit 26 for calculating a sound source signal from sound image information, and sound for synthesizing a sound field based on the sound source signal and the virtual sound source position The case completion processing unit 27 is provided.

  The audio signal reproduction processing unit 23 reproduces the audio data decoded by the audio decoder 13 and sends it to the multi-channel amplifier 21.

  The sound image information extraction processing unit 24 extracts sound image information and sound image position information from the data read by the optical disc reproduction unit 11, and the extracted sound image information and sound image position information are used as a virtual sound source position calculation processing unit 25 and a sound source signal calculation processing unit. 26. The sound source information includes musical instruments, voices, and the like that make up audio data. Further, the sound image information extraction processing unit 24 extracts frequency components correlated with temporal fluctuations of the main sound source from a speech waveform in which sounds from several types of sound sources are mixed by general harmonic signal analysis. For example, the sound image information extraction processing unit 24 can separate frequency components constituting the target female vocal signal by general harmonic signal analysis. For example, the frequency component of the sound source signal localized at the center position can be separated.

  The virtual sound source position calculation processing unit 25 calculates the sound source position where the sound field should be synthesized according to the sound image position information extracted by the sound image information extraction processing unit 24. Here, the sound source position to be generated in the sound case is the position of the virtual sound source in accordance with the accuracy of the subsequent sound case generation processing unit 27 and the speaker arrangement. For example, if the sound source signal is a sound source signal localized at the center position, the virtual sound source position calculation processing unit 25 calculates the virtual sound source distance from the listening point to the sound virtual source signal, the accuracy of the subsequent sound case generation processing unit 27, and the speaker arrangement. Calculate or change according to the calculation.

  The sound source signal calculation processing unit 26 performs sound image localization of a predetermined sound source at the sound source position calculated by the virtual sound source position calculation processing unit 25 to the audio data separated for each sound source by the sound image information extraction processing unit 24 by the general harmonic signal analysis. Gives control parameters. 2 to 5 show frequency waveforms separated from audio data.

  The sound source signal calculation processing unit 26 is a sound source signal that localizes a sound image at a specific position based on the analysis results in the sound image information extraction processing unit 24 and the virtual sound source position calculation processing unit 25 from audio data such as 2-channel audio data and multi-channel audio data. Different sound source signals are separated for each part. For example, the sound source signal calculation processing unit 26 calculates a female vocal sound source signal from the frequency components constituting the target female vocal signal separated by the general harmonic signal analysis in the sound image information extraction processing unit 24. For example, the frequency component of the sound source signal localized at the center position can be separated. The sound source signal calculation processing unit 26 sends the calculated sound source signal to the sound case generation processing unit 27.

  The sound source signal calculation processing unit 26 calculates a control parameter for changing the localization position of the separated predetermined frequency component in the spreading direction (horizontal direction) or the depth direction.

  As an example of changing the control parameter for changing the localization position of the sound source sound image in the spreading direction (horizontal direction), the sound source signal calculation processing unit 26, for example, a sound case generation processing unit described later with respect to the direction of the observation point and the virtual sound source If the accuracy of 27 can be generated only with an accuracy of about 10 °, the parameter that causes the change in the position of the sound image in units of 8 ° of the sound image direction is reconverted to 10 °. In addition, when the sound image can be arranged only within the range of 40 ° in the horizontal direction due to restrictions on the speaker arrangement, the parameter that causes the sound image position change of 50 ° is reconverted to 40 °.

  The same applies to the sound image position distance data, which is the distance between the observation point and the virtual sound source. In general, it is known that the accuracy of human distance perception decreases for a sound source farther than 3 meters. Therefore, the sound source signal calculation processing unit 26, as an example of changing the control parameter for changing the localization position of the sound source sound image in the depth direction, sets a parameter that causes a distance change of about 5 meters as a sound image at a sound source position of about 3 meters. The parameter is converted into a parameter to be localized, and the parameter that causes a distance change of about 8 meters in the sound image distance is reconverted into the sound source distance of 10 meters.

  The sound source signal calculation processing unit 26 can also create a new sound source signal based on the separated sound source. For example, a parameter for localizing the sound source signal of the specific sound source separated by the sound image information extraction processing unit 24 is calculated, or a control parameter for generating a sound source signal having a frequency slightly different from that of the specific sound source is calculated. . As a result, it is possible to generate a frequency component of another sound source that unisons a certain sound source, or another sound source that produces harmony with a certain sound source.

  The sound case formation processing unit 27 performs sound case formation in which the sound source signal calculated by the sound source signal calculation processing unit 26 is arranged at the virtual sound source position calculated by the virtual sound source position calculation processing unit 25. At this time, the sound case formation processing unit 27 performs sound case formation by rearranging the extracted sound source signal at the virtual sound source position by one or a combination of pitch change, timing change, and envelope generator. In addition, the sound case generator 27 performs normal intensity stereo reproduction of the intensity stereo signal from which the sound source signal having the sound image localized at the center position is removed, and rearranges the separated sound source signal at the center position. Do.

  When a planar array speaker is used as an example of an audio signal output means for the audio signal output circuit 22, the sound case processing unit 27 is a digital for driving the array speaker for outputting a sound wave extending concentrically from the central position. The filter coefficient is calculated, and the convolution operation is performed on the target vocal signal.

  2 to 5 show frequency components separated by the sound image information extraction processing unit 24. 2 to 5, the horizontal axis represents time, and the vertical axis represents the signal level. Since the signal level of the frequency component f1 shown in FIG. 2 and the signal level of the frequency component f2 shown in FIG. 3 have a strong correlation with time fluctuation, the sound case generator 27 uses the same sound source as f1 and f2 for sound image localization. To do. Similarly, the sound case component processing unit 27 performs sound image localization using the frequency component f3 shown in FIG. 4 and the frequency component f4 shown in FIG. 5 as the same sound source.

  The sound source signal calculation processing unit 26 uses the frequency components f1 to f4 shown in FIG. 2 to FIG. 5 separated from the original audio signal by general harmonic signal analysis as the same sound source. A control parameter for localizing to the same position in the reproduction space is given. The sound case component processing unit 27 re-synthesizes the given control parameter by superimposing it on the extracted frequency component, for example, on f1 and f2. Then, a parameter for localizing the sound image as the same sound source is given to the frequency component of the extracted sound source and re-synthesized as the frequency component of the predetermined instrument.

  The subsequent multi-channel amplifier 21 amplifies the multi-channel signal for the array speaker, and performs amplification for reproducing the sound source signal that has not been recombined among the audio signals in two channels or 5.1 channels. For example, when the vocal signal is separated to re-synthesize the sound field as described above, the multi-channel amplifier 21 amplifies the vocal signal with multiple channels for the array speaker, and after the vocal signal component is separated. Audio signal is amplified by two channels of an L channel signal and an R channel signal.

  An operation of reproducing the content data read from the optical disc by the signal reproducing apparatus 1 having the above-described configuration will be described.

  The optical disk reproducing unit 11 reads the data recorded on the DVD by rotating the optical disk. The data read by the optical disc reproducing unit 11 is separated into compressed audio data, compressed video data, caption data, other data, and the like by the signal separation circuit 12. The compressed audio data is decoded by the audio decoder 13 and then sent to the audio signal reproduction processing unit 23 of the audio signal processing circuit 14. The audio signal reproduction processing unit 23 reproduces the decoded audio data and sends it to the multi-channel amplifier 21.

  The data read by the optical disk reproducing unit 11 is also sent to the sound image information extraction processing unit 24 of the audio signal processing circuit 14. The sound image information extraction processing unit 24 extracts sound image information and sound image position information for each sound source from the reproduction signal by general harmonic signal analysis, and uses the extracted sound image information and sound image position information as the virtual sound source position calculation processing unit 25 and the sound source signal. The data is sent to the calculation processing unit 26. The virtual sound source position calculation processing unit 25 calculates the sound source position where the sound field should be synthesized according to the sound image position information extracted by the sound image information extraction processing unit 24. Further, the sound source signal calculation processing unit 26 converts a predetermined sound source into the sound source position calculated by the virtual sound source position calculation processing unit 25 into the audio data separated for each sound source by the sound image information extraction processing unit 24 by the general harmonic signal analysis. Control parameters are given to

  Based on the sound source signal calculated by the sound source signal calculation processing unit 26 and the control parameter, the sound case generation processing unit 27 performs sound case formation to be arranged at the virtual sound source position calculated by the virtual sound source position calculation processing unit 25. At this time, the sound case formation processing unit 27 performs sound case formation by rearranging the extracted sound source signal at the virtual sound source position by one or a combination of pitch change, timing change, and envelope generator. The audio signal in which the sound field is recombined in the sound case generating unit 27 is sent to the multi-channel amplifier 21. In the multi-channel amplifier 21, when a specific audio signal, for example, a vocal signal component is separated, the separated vocal signal is amplified by multi-channel, and the audio signal after the separation of the vocal signal component is converted into an L channel signal, R Amplifies with two channels of channel signal.

  On the other hand, the video data and the caption data are sent to the video decoder 17 and the caption decoder 15 and reproduced by the subsequent caption reproducing circuit 16 or video signal reproducing circuit 18. The subtitle signal and the video signal are combined with the video in synchronism with the video signal in the subtitle synthesis circuit 19. The audio signal in which the sound field is re-synthesized is synchronized with each other from the audio signal output circuit 22 and the video signal with the captions synthesized is synchronized with each other from the video signal output circuit 20 and output to an external speaker system, display device, or the like.

  Therefore, in the signal reproduction apparatus 1, the sound image information extraction processing unit 24 separates and calculates the musical sound for each sound source included in the audio data, and the sound source signal calculation processing unit 26 detects the sound source close to the listening point in the listener's distance perception. The parameter is changed to the position, the parameter for rearranging the sound image localized far away is calculated, the specific sound source signal is re-synthesized with the virtual sound source position calculated by the virtual sound source position calculation processing unit 25, and reproduced. By reconstructing the virtual sound field, the viewer's sense of presence can be enhanced. The signal reproducing apparatus 1 described above realizes the basic configuration of the present invention.

  In the following, other application examples of the present invention will be described. FIG. 6 shows a signal reproducing device 2 as a second specific example of the present invention. The signal reproduction apparatus 2 shown in FIG. 2 is characterized in that sound source information and sound image position information for each sound image are prepared as metadata for each content such as a song, a movie, etc., independently of multi-channel audio data. It is. This metadata is stored in advance in a predetermined area corresponding to an identification code for identifying the content to be associated with the content. Here, the identification code is information unique to each content generated from a part of content data such as audio data using a predetermined rule.

  For this reason, the signal reproduction apparatus 2 uses the same procedure as that for storing the identification code necessary for reading the sound source information and the sound image position information separated from the audio data and stored in a predetermined area for each content on the optical disc. And a search processing unit 29 for searching for metadata corresponding to the generated identification code from the optical disc. In the signal reproducing device 2 shown in FIG. 6, the same functions as those of the signal reproducing device 1 shown in FIG.

  The identification code generation unit 28 generates an identification code necessary for reading sound source information and sound image position information separated from the audio data and stored as metadata in a predetermined area from a part of the multi-channel data. The identification code includes the number of tracks recorded in the TOC, data such as the performance time of each track, data obtained by combining these, data obtained by performing predetermined encoding on the multi-channel data itself, etc. Can be given.

  The search processing unit 29 searches the multiplexed data for the identification code of the music to be played according to the identification code for specifying the content generated by the identification code generation unit 28.

  An operation of reproducing the content data read from the optical disc by the signal reproducing apparatus 2 having the above-described configuration will be described.

  The data read by the optical disc reproducing unit 11 is separated into compressed audio data, compressed video data, caption data, other data, and the like by the signal separation circuit 12. The compressed audio data is decoded by the audio decoder 13 and then sent to the audio signal reproduction processing unit 23 of the audio signal processing circuit 14. The audio signal reproduction processing unit 23 reproduces the decoded audio data and sends it to the multi-channel amplifier 21.

  The identification code generation unit 28 of the audio signal processing circuit 14 generates an identification code from the data read by the optical disc reproduction unit 11 according to the same rule as that at the time of creation. Then, the search processing unit 29 searches the multiplexed data for the sound source information and sound image position information data of the musical composition to be played in accordance with the generated identification code.

  In the signal reproduction device 2, the sound image information extraction processing unit 24 extracts sound image information and sound image position information for each sound source from the metadata obtained by the search processing unit 29, and extracts the extracted sound image information and sound image position information. This is sent to the virtual sound source position calculation processing unit 25 and the sound source signal calculation processing unit 26. The virtual sound source position calculation processing unit 25 calculates the sound source position where the sound field should be synthesized according to the sound image position information extracted by the sound image information extraction processing unit 24. In the sound source signal calculation processing unit 26, control parameters for localizing a predetermined sound source at the sound source position calculated by the virtual sound source position calculation processing unit 25 to the audio data separated for each sound source by the sound image information extraction processing unit 24. Is given.

  Based on the sound source signal calculated by the sound source signal calculation processing unit 26 and the control parameter, the sound case generation processing unit 27 performs sound case formation to be arranged at the virtual sound source position calculated by the virtual sound source position calculation processing unit 25. At this time, the sound case formation processing unit 27 performs sound case formation by rearranging the extracted sound source signal at the virtual sound source position by one or a combination of pitch change, timing change, and envelope generator. The audio signal in which the sound field is re-synthesized in the sound case synthesis unit 27 is sent to the multi-channel amplifier 21, and the separated specific sound source signal is amplified in multi-channels and the specific sound source signal component is separated. Audio signal is amplified by two channels. The caption signal and the video signal are combined with the video in synchronism with the video signal in the caption synthesizing circuit 19, and the audio signal whose sound field is re-synthesized is sent from the audio signal output circuit 22 and the video signal with which the caption is synthesized. Are output from the video signal output circuit 20 to an external speaker system, a display device or the like in synchronization with each other.

  As described above, the signal reproduction device 2 prepares sound source information and sound image position information for each sound image as metadata independent of multi-channel audio data, and further stores the metadata in a predetermined area corresponding to the identification code. By storing, it is possible to reduce the amount of calculation required for the sound source extraction processing by general harmonic signal analysis or the like. In addition, the sound image information extraction processing unit 24 separates the signal for each sound source, and the sound source signal calculation processing unit 26 changes the sound source close to the listening point to a position where the distance perception of the listener is sensitive so that the sound image localized in the distance can be obtained. By calculating parameters to be rearranged further, re-synthesize a specific sound source signal with the virtual sound source position calculated by the virtual sound source position calculation processing unit 25, and reconstruct the virtual sound field to be reproduced. Can increase the sense of reality.

  Next, a signal reproducing device 3 is shown in FIG. 7 as a third specific example of the present invention. 7, components having the same functions as those of the signal reproduction device shown in FIGS. 1 and 6 are assigned the same reference numerals and detailed description thereof is omitted. The signal reproduction apparatus 3 shown in FIG. 7 is not the case where the above-described multi-channel audio data is provided as a so-called packaging medium such as an optical disc, but is a case where the multi-channel audio data is sent via a network. Further, in the signal reproduction device 3, sound source information and sound image position information for each sound image are prepared as metadata independently of multi-channel audio data, and the multi-channel data and metadata are mixed and transmitted via a network. It is characteristic that it is provided.

  Therefore, instead of the optical disc playback unit 11, the signal playback device 3 is a network interface (hereinafter referred to as a network I / F) 31 connected to a network such as a local area network, an original network, or a so-called Internet that is wirelessly or wiredly connected. And a reception buffer 32 for temporarily storing content data such as audio data sent via the network. Network communication protocols include general-purpose protocols such as TCP / IP.

  Further, the signal reproduction device 3 generates an identification code necessary for reading the metadata of the sound source information and the sound image position information created for each content in the same procedure as when the identification code of this content is transmitted. And a search processing unit 34 for searching for metadata corresponding to the identification code generated here from data received from the network.

  In the signal reproduction device 3 shown in FIG. 7, the audio data transmitted via the network may be stream data that enables real-time reproduction, or may be batch transmission data such as so-called download. Good. Further, the signal reproduction device 3 may include an identification code input unit 35, and the identification code may be directly input by the user.

  An operation of reproducing the content data received via the network by the signal reproduction device 3 having the above-described configuration will be described.

  Data received by the network I / F 31 is temporarily stored in the reception buffer 32, and is separated into compressed audio data, compressed video data, caption data, other data, and the like by the signal separation circuit 12. The compressed audio data is decoded by the audio decoder 13 and then sent to the audio signal reproduction processing unit 23 of the audio signal processing circuit 14. The audio signal reproduction processing unit 23 reproduces the decoded audio data and sends it to the multi-channel amplifier 21.

  The identification code generation unit 33 of the audio signal processing circuit 14 generates an identification code from the data received by the network I / F 31 according to the same rule as that at the time of creation. Then, the search processing unit 34 searches for the metadata of the musical composition to be played from the multiplexed data according to the generated identification code.

  In the signal reproduction device 3, the sound image information extraction processing unit 24 extracts sound image information and sound image position information for each sound source from the metadata obtained by the search processing unit 29, and extracts the extracted sound image information and sound image position information. This is sent to the virtual sound source position calculation processing unit 25 and the sound source signal calculation processing unit 26. The virtual sound source position calculation processing unit 25 calculates the sound source position where the sound field should be synthesized according to the sound image position information extracted by the sound image information extraction processing unit 24. In the sound source signal calculation processing unit 26, control parameters for localizing a predetermined sound source at the sound source position calculated by the virtual sound source position calculation processing unit 25 to the audio data separated for each sound source by the sound image information extraction processing unit 24. Is given.

  The sound case formation processing unit 27 performs sound case formation in which the sound source signal calculated by the sound source signal calculation processing unit 26 and the sound source signal extracted based on the control parameter are rearranged at the virtual sound source position. The audio signal in which the sound field is re-synthesized in the sound case synthesis unit 27 is sent to the multi-channel amplifier 21, and the separated specific sound source signal is amplified in multi-channels and the specific sound source signal component is separated. Audio signal is amplified by two channels. The caption signal and the video signal are combined with the video in synchronism with the video signal in the caption synthesizing circuit 19, and the audio signal whose sound field is re-synthesized is sent from the audio signal output circuit 22 and the video signal with which the caption is synthesized. Are output from the video signal output circuit 20 to an external speaker system, a display device or the like in synchronization with each other.

  As described above, the signal reproduction device 3 prepares sound source information and sound image position information for each sound image as metadata independent of multi-channel audio data, and further stores the metadata in a predetermined area in advance corresponding to the identification code. By storing, it is possible to reduce the amount of calculation required for the sound source extraction processing by general harmonic signal analysis or the like. Therefore, the signal reproduction device 3 separates the signal for each sound source, changes the sound source close to the listening point to a position where the distance perception of the listener is sensitive, and relocates the sound image localized far away. Signal processing that calculates, re-synthesizes a specific sound source signal at the virtual sound source position, and reconstructs the virtual sound field to be played back is also possible in real time for content data such as audio data transmitted over the network become.

  Next, a signal reproducing apparatus 4 is shown in FIG. 8 as a fourth specific example of the present invention. In the signal reproduction device 4 shown in FIG. 8, the sound source information and the sound image position information for each sound image are prepared as metadata independently from the multi-channel audio data, but the audio data is obtained from the optical disk or from the optical disk. The sound source information and the sound image position information are transmitted to the signal reproduction device 4 through a network.

  Therefore, the signal reproduction device 4 generates an identification code necessary for reading the metadata of the sound source information and the sound image position information created for each content in the same procedure as when the identification code of this content is transmitted. A network interface (hereinafter referred to as a network I / F) 36 connected to a network 36 such as a local area network or an original network that is wirelessly or wiredly connected, so-called Internet, and the identification generated by the identification code generator 36 And a search processing unit 38 that searches the data corresponding to the code from the data received from the network. Although not shown in FIG. 8, the signal reproduction device 4 includes a reception buffer that temporarily stores content data such as audio data transmitted via a network. Network communication protocols include general-purpose protocols such as TCP / IP. In the signal reproducing device 4 shown in FIG. 8, the same reference numerals are given to the components having the same functions as those of the signal reproducing device shown in FIGS. 1, 6, and 7, and detailed description thereof is omitted.

  An operation of reproducing the content data read from the optical disc by the signal reproducing device 4 having the above-described configuration will be described.

  The data read by the optical disk reproducing unit 11 is separated into compressed audio data, compressed video data, caption data, other data, and the like by the signal separation circuit 12. The compressed audio data is decoded by the audio decoder 13 and then sent to the audio signal reproduction processing unit 23 of the audio signal processing circuit 14. The audio signal reproduction processing unit 23 reproduces the decoded audio data and sends it to the multi-channel amplifier 21.

  The identification code generation unit 36 of the audio signal processing circuit 14 sends the identification code received by the network I / F 37 to the search processing unit 38. The search processing unit 38 searches the metadata of the musical composition to be played from the multiplexed data according to the generated identification code.

  The data read from the optical disk reproduction unit 11 is separated into compressed audio data, compressed video data, caption data, other data, and the like by the signal separation circuit 12. The compressed audio data is decoded by the audio decoder 13 and then sent to the audio signal reproduction processing unit 23 of the audio signal processing circuit 14. The audio signal reproduction processing unit 23 reproduces the decoded audio data and sends it to the multi-channel amplifier 21.

  In the signal reproduction device 4, the sound image information extraction processing unit 24 extracts sound image information and sound image position information for each sound source from the metadata obtained by the search processing unit 29, and extracts the extracted sound image information and sound image position information. This is sent to the virtual sound source position calculation processing unit 25 and the sound source signal calculation processing unit 26. The virtual sound source position calculation processing unit 25 calculates the sound source position where the sound field should be synthesized according to the sound image position information extracted by the sound image information extraction processing unit 24. In the sound source signal calculation processing unit 26, control parameters for localizing a predetermined sound source at the sound source position calculated by the virtual sound source position calculation processing unit 25 to the audio data separated for each sound source by the sound image information extraction processing unit 24. Is given.

  The sound case formation processing unit 27 performs sound case formation in which the sound source signal calculated by the sound source signal calculation processing unit 26 and the sound source signal extracted based on the control parameter are rearranged at the virtual sound source position. The audio signal in which the sound field is re-synthesized in the sound case synthesis unit 27 is sent to the multi-channel amplifier 21, and the separated specific sound source signal is amplified in multi-channels and the specific sound source signal component is separated. Audio signal is amplified by two channels. The caption signal and the video signal are combined with the video in synchronism with the video signal in the caption synthesizing circuit 19, and the audio signal whose sound field is re-synthesized is sent from the audio signal output circuit 22 and the video signal with which the caption is synthesized. Are output from the video signal output circuit 20 to an external speaker system, a display device or the like in synchronization with each other.

  As described above, the signal reproduction device 4 prepares sound source information and sound image position information for each sound image as metadata independent of multi-channel audio data, and receives this from the network. Then, the audio data is provided on the optical disc, so that the amount of calculation required for the sound source extraction processing by general harmonic signal analysis or the like can be reduced. Therefore, the signal reproduction device 4 separates the signal for each sound source, changes the sound source close to the listening point to a position where the distance perception of the listener is sensitive, and relocates the sound image localized far away. Signal processing that calculates, re-synthesizes a specific sound source signal at the virtual sound source position, and reconstructs the virtual sound field to be played back is also possible in real time for content data such as audio data transmitted over the network become.

  Also, for example, in the signal reproduction device 4, metadata describing sound source information and sound image position information about content such as CD and DVD released in the past by a provider who provides content data such as audio data is displayed on the Web page. It is possible to enhance the viewer's sense of presence by executing signal processing for reconstructing a virtual sound field even for content that is a legacy asset, for example, by providing it on the Internet.

  A specific example of a speaker system that can be applied as an output destination of the multi-channel amplifier 21 and the audio signal output circuit 22 of the signal reproduction device shown in FIGS. 1, 6, 7, and 8 will be described.

  The speaker system 50 shown in FIG. 9 includes a sound channel reproduction multi-channel sound amplification circuit 51, a 5.1 channel multi-channel sound amplification circuit 52, sound wave surface reproduction speakers 531, 532,. A 5.1-channel speaker system including six speakers: a front speaker 541 of the listener, a right front speaker 542, a left front speaker 543, a right rear speaker 544, a left rear speaker 545, and a low-frequency output subwoofer speaker 546. ing.

  The speaker system 50 includes the multi-channel amplifier 21 and the audio signal output circuit 22 in the signal reproduction apparatus shown in FIGS. 1, 6, 7, and 8. The multi-channel amplifier 21 is used for the sound wave surface. This corresponds to the multi-channel acoustic amplifier circuit 51 and the 5.1-channel acoustic amplifier circuit 52. Accordingly, the sound source multi-channel acoustic amplification circuit 51 and the 5.1-channel acoustic amplification circuit 52 have the virtual sound source position calculation processing unit 25 based on the sound source signal calculated by the sound source signal calculation processing unit 26 and the control parameters. An audio signal subjected to sound case formation for rearrangement at the calculated virtual sound source position is sent from the sound case formation processing unit 27.

  The audio data is amplified for each channel by the sound wave front multi-channel sound amplification circuit 51 and the 5.1-channel sound amplification circuit 52 and output from a speaker as an audio output device. In the case of the 5.1 channel surround system, the sound is output from six speakers: the front, right front, left front, right rear, left rear, and bass subwoofer speakers for the listener. Sound wave reproduction speakers 531 to 53n output sounds having different wavefronts, and perform wavefront synthesis that localizes a sound image at an arbitrary position.

  For example, when the planar array speaker 50 is used in combination with the projection projector 60, the user can obtain a more realistic sound field. The image light emitted from the projection type projector 60 is projected onto a screen 61 placed on the front listener side of the sound wave surface reproduction speaker 53. When the synthesized sound signal synthesized by the sound case processor 27 is output from each speaker from the speaker system arranged behind the screen 61, a sound field re-synthesized by the signal reproduction device described above is constructed.

It is a block diagram explaining the signal reproducing | regenerating apparatus shown as a 1st example of this invention. It is a wave form diagram which shows the intensity | strength with respect to time of the frequency component f1 isolate | separated by the sound image information extraction circuit of the said signal reproduction apparatus. It is a wave form diagram which shows the intensity | strength with respect to the time of the frequency component f2 isolate | separated by the sound image information extraction circuit of the said signal reproduction apparatus. It is a wave form diagram which shows the intensity | strength with respect to the time of the frequency component f3 isolate | separated by the sound image information extraction circuit of the said signal reproduction apparatus. It is a wave form diagram which shows the intensity | strength with respect to time of the frequency component f4 isolate | separated by the sound image information extraction circuit of the said signal reproduction apparatus. It is a block diagram explaining the signal reproducing | regenerating apparatus shown as the 2nd example of this invention. It is a block diagram explaining the signal reproducing | regenerating apparatus shown as the 3rd example of this invention. It is a block diagram explaining the signal reproducing | regenerating apparatus shown as the 4th example of this invention. It is a block diagram explaining an example of the speaker system which can reproduce | regenerate the acoustic signal synthesize | combined with the signal reproduction apparatus shown as a specific example of this invention.

Explanation of symbols

  1, 2, 3, 4 signal playback device, 11 optical disc playback unit, 12 signal separation circuit, 13 audio decoder, 14 audio signal processing circuit, 15 subtitle decoder, 16 subtitle playback circuit, 17 video decoder, 18 video signal playback circuit, 19 caption synthesis circuit, 20 video signal output circuit, 21 multi-channel amplifier, 22 audio signal output circuit, 23 audio signal reproduction processing unit, 24 sound image information extraction processing unit, 25 virtual sound source position calculation processing unit, 26 sound source signal calculation processing unit 27 sound case generator, 28, 33, 36 identification code generator, 29, 34, 38 search processor, 31, 37 network I / F, 32 reception buffer, 35 identification code input unit

Claims (11)

In a signal reproducing apparatus for reproducing multi-channel data in which audio signals are multiplexed,
Multi-channel data acquisition means for acquiring the multi-channel data;
Signal analysis means for extracting sound source information and sound image position information of a specific sound image from the multi-channel data;
A sound source signal calculating means for calculating a sound source signal from the sound source information;
A signal reproduction apparatus comprising: sound case forming means for changing sound source information of the extracted specific sound image and arranging sound source signals of the changed specific sound image at an arbitrary virtual sound source position.   Virtual sound source position calculating means for calculating a virtual sound source position of the specific sound image according to the extracted sound source information of the specific sound image and the sound image position information is provided, and the sound case generating means is calculated by the sound source signal calculating means. 2. The signal reproduction apparatus according to claim 1, wherein the sound source signal is arranged at the virtual sound source position.   3. The sound source signal calculating unit calculates a new sound source signal based on the sound source information, and the sound case generating unit arranges the new sound source signal at an arbitrary virtual sound source position. Signal reproduction device.   2. The signal reproduction apparatus according to claim 1, wherein the sound source signal calculating means calculates a sound source signal obtained by rearranging the sound source signal of the specific sound image in the depth direction.   The multi-channel data is an intensity stereo signal, and the sound source signal calculation means separates only a sound source signal that is localized at a central position from the intensity stereo signal based on an analysis result in the signal analysis means, The sound case generating means reproduces the intensity stereo signal from which the sound source signal localized at the central position is removed, and reproduces the intensity stereo signal in a normal intensity stereo, and rearranges the separated sound source signal at the central position. 2. The signal reproducing apparatus according to 1.   2. The signal reproduction apparatus according to claim 1, wherein sound source information and sound image position information for each sound image are prepared independently from the multi-channel data.   In the recording medium, the sound source information and sound image position information are stored in a predetermined area in association with data identification information unique to each content generated from a part of the multi-channel data, and data identification for specifying the multi-channel data Data identification information generating means for generating information from a part of the multi-channel data, and search means for searching the sound source information and sound image position information corresponding to the generated data identification information from the area. The signal reproducing apparatus according to claim 6.   7. The signal reproducing apparatus according to claim 6, wherein the multi-channel data and the sound source information and sound image position information for each sound image prepared independently from the multi-channel data are provided via a network.   7. The signal reproducing apparatus according to claim 6, wherein the multi-channel data is provided by being stored in a recording medium, and the sound source information and sound image position information for each sound image are provided via a network. In a signal reproduction method for reproducing multi-channel data in which audio signals are multiplexed,
A multi-channel data acquisition step for acquiring the multi-channel data;
A signal analysis step of extracting sound source information and sound image position information of a specific sound image from the multi-channel data;
A sound source signal calculating step of calculating a sound source signal from the sound source information;
A signal reproduction method comprising: a sound case generating step for performing sound case formation in which the sound image information of the extracted specific sound image is changed and the sound source signal of the changed specific sound image is arranged at an arbitrary virtual sound source position. A virtual sound source position calculating step of calculating a virtual sound source position of the specific sound image according to the extracted sound image information of the specific sound image and the sound image position information;
11. The signal reproduction method according to claim 10, wherein, in the sound case generating step, the sound source signal calculated in the sound source signal calculating step is arranged at the virtual sound source position.

Images