CN115278506A - Audio processing method and audio processing device - Google Patents

Abstract

The present application provides an audio processing method, which separates left and right channel audio into center left and right channel audio and side left and right channel audio, and performs center and side head transfer functions on the center and side left and right channel audio respectively. processing, so as to simulate the above-mentioned audio as multiple sound source positions relative to the user, and then perform frequency compensation on the processed audio according to the user's hearing characteristics to synthesize two-channel audio. The present application further provides an audio processing apparatus for executing the above audio processing method.

Description

Audio processing method and audio processing device

technical field

The present application relates to an audio processing technology, in particular to an audio processing method and an audio processing device for simulating two-channel audio into multi-channel audio.

Background technique

Nowadays, multimedia information is increasingly developed, whether it is movies, TV dramas or games, etc. are inseparable from people's lives. Because of this, people often wear headphones to listen to music, watch movies or TV shows when commuting, and also wear headphones to play 3D games for a more immersive sound experience. However, the general traditional binaural earphones can only give the user two-channel listening experience, so that when watching movies or TV shows, the sound heard by the user does not have a sense of surround, or When playing games, it is also impossible to distinguish sounds from multiple directions. In addition, each user's hearing performance is different, and different users will have different responses to different sound frequencies. Therefore, if there is an earphone that can process a two-channel sound source into multiple channels, and can adjust the frequency of the output sound according to the hearing characteristics of different users, it will definitely bring users a better listening experience.

Contents of the invention

The technical problem to be solved by this application is to provide an audio processing method that can process a two-channel sound source into multiple channels, and can compensate for different frequencies of audio for different users

The present application further provides an audio processing device capable of executing the above-mentioned audio processing method

The present application proposes an audio processing method, including: separating left channel audio into center left channel audio and side left channel audio; separating right channel audio into center right channel audio and side right channel audio ; Process the center left channel audio and the center right channel audio with the center head transfer function to simulate the center left channel audio and the center right channel audio as relative to the user's first sound source position and second sound source position ;Subject the side-left and side-right audio to the side-head transfer function to simulate the side-left and side-right audio as a third source position relative to the user and the fourth sound source position; and performing frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the hearing characteristics of the user to synthesize binaural audio.

The present application further proposes an audio processing device, which includes a channel separation unit, an audio computing unit, and an audio synthesis unit. The channel separation unit receives left channel audio and right channel audio, separates left channel audio into center left channel audio and side left channel audio, and separates right channel audio into center right channel audio and side Right channel audio. The audio operation unit processes the central left channel audio and the central right channel audio with the central head transfer function, so as to simulate the central left channel audio and the central right channel audio as relative to the user's first sound source position and second The position of the sound source, and the side left channel audio and the side right channel audio are processed by the side head transfer function, so that the side left channel audio and the side right channel audio are simulated as relative to the user's first Three sound source positions and a fourth sound source position. The audio synthesis unit performs frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the user's hearing characteristics to synthesize binaural audio.

In some embodiments, the audio processing method includes playing a plurality of audio with different frequencies to the user, and generating a plurality of frequency response values in response to the audio with different frequencies to obtain the hearing characteristics of the user, and then converting the frequency response values Comparing with the default value to generate at least one frequency response difference, and then compensating audio at different frequencies according to the frequency response difference.

In some embodiments, the audio processing device further includes an audio playback unit and a comparison unit. The audio playback unit plays a plurality of audios with different frequencies to the user, and the aforementioned audio calculation unit generates a plurality of frequency response values in response to the audios with different frequencies to obtain the hearing characteristics of the user. The comparison unit compares the frequency response values with at least one default value to generate at least one frequency response difference, and the aforementioned audio computing unit compensates the audio with different frequencies according to the frequency response difference.

To sum up, the audio processing method and the audio processing device provided by the present application can process the audio of the left and right channels into audio of four different source orientations, and can compensate the audio of different frequencies according to the hearing performance of the user, so that Two-channel audio can also have a surround sound listening experience.

Details about other functions and embodiments of the present application are described below with reference to the accompanying drawings.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in this application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a flowchart of an audio processing method according to an embodiment of the present application;

2 is a block diagram of an audio processing device according to an embodiment of the present application;

FIG. 3 is a schematic diagram of audio source distribution according to an embodiment of the application.

Symbol Description

101-105 steps 200 audio processing device

201 stereo separation unit 202 equalizer

203 channel separation unit 204 audio computing unit

205 audio synthesis units 300 users

301-304 sound source position SA stereo audio

L Left channel audio R Right channel audio

L_Eq Bass Enhanced Left Channel Audio

R_Eq Bass Enhanced Right Channel Audio

Cent_L center left channel audio

Cent_R center right channel audio

Side_L side left channel audio

Side_R side right channel audio

Center left channel audio processed by Cent_LH head transfer function

Center right channel audio processed by Cent_RH head transfer function

Side left channel audio processed by Side_LH head transfer function

Side_RH head transfer function processed side right channel audio

Detailed ways

The positional relationships described in the following embodiments include: up, down, left and right. Unless otherwise specified, they are all based on the directions in which the components are drawn in the drawings.

FIG. 1 is a flowchart of an audio processing method according to an embodiment of the application. As shown in FIG. 1 , the audio processing method of the present application includes steps 101-105. Step 101: Separate the left channel audio into center left channel audio and side left channel audio. Step 102: Separate the right channel audio into center right channel audio and side right channel audio. Step 103: The central left channel audio and the central right channel audio are processed by the central head transfer function, so that the central left channel audio and the central right channel audio are simulated relative to the user's first sound source position and second The location of the sound source. Step 104: Perform side head transfer function processing on the side left channel audio and the side right channel audio, so as to simulate the side left channel audio and the side right channel audio as the third relative to the user. The position of the sound source and the position of the fourth sound source. Step: 105: Perform frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the user's hearing characteristics to synthesize binaural audio.

FIG. 2 is a block diagram of an audio processing device according to an embodiment of the application. FIG. 3 is a schematic diagram of audio source distribution according to an embodiment of the application. The blocks in FIG. 2 will be used below to illustrate how the audio processing device of the present application executes the audio processing method in FIG. 1 . Please refer to Figure 1, Figure 2 and Figure 3. As shown in FIG. 2 , the audio processing device 200 includes a stereo separation unit 201 , an equalizer 202 , a channel separation unit 203 , an audio calculation unit 204 and an audio synthesis unit 205 . The stereo separation unit 201 receives stereo audio SA and separates it into left channel audio L and right channel audio R. In this embodiment, the stereo audio SA includes, for example, the left channel audio L and the right channel audio R, but the application is not limited thereto, and it may also include more channels. The equalizer 202 receives the left-channel audio L and the right-channel audio R, and the left-channel audio L and the right-channel audio R processed by the equalizer 202 can enhance their bass effect, thereby producing a left channel with a better bass effect Audio L_Eq and right channel audio R_Eq. The channel separation unit 203 receives the left channel audio L_Eq and the right channel audio R_Eq, and separates the left channel audio L_Eq into the central left channel audio Cent_L and the side left channel audio Side_L, and separates the right channel audio R_Eq It is the center right channel audio Cent_R and the side right channel audio Side_R.

Specifically, the center left channel audio Cent_L and the center right channel audio Cent_R are equivalent to the left and right sound sources from the front for the user, while the side left channel audio Side_L and the side right channel audio Side_R are Equivalent to left and right sound sources from the sides. In particular, the above-mentioned left channel sound source and right channel sound source are the audio heard by the user's left ear and right ear respectively. The audio computing unit 204 will receive the above-mentioned central left channel audio Cent_L, central right channel audio Cent_R, side left channel audio Side_L and side right channel audio Side_R, and perform central head transfer function and side Edge head transfer function processing. Head Related Transfer Functions (HRTF) is a sound localization algorithm. The process of its localization and calculation is known to those skilled in the art, so it will not be repeated here. Through the calculation of the head transfer function, the center left channel audio Cent_L, the center right channel audio Cent_R, the side left channel audio Side_L and the side right channel audio Side_R will be virtualized as the sound source position 301 relative to the user 300 -304, as shown in Figure 3. The audio synthesis unit 205 will receive the aforementioned audio Cent_LH, Cent_RH, Side_LH, and Side_RH processed by the head transfer function, and then perform frequency compensation on the received audio according to the user's hearing characteristics and then synthesize them into two-channel audio. Now, even through binaural headphones, users can listen to the equivalent of multi-channel surround sound.

Further, the audio processing device 200 is, for example, a two-channel earphone, and the stereo separation unit 201, the equalizer 202, the channel separation unit 203, the audio computing unit 204, and the audio synthesis unit 205 are independent or integrated in the earphone. component, circuit or chip. The audio processing device 200 further includes an audio playback unit and a comparison unit (not shown in the figure). The audio playback unit will play multiple audios with different frequencies to the user, and the user will respond to these audios with different frequencies after listening. This produces a number of frequency response values that may be representative of the user's hearing personality. The comparison unit then compares these frequency response values with a default value to generate at least one frequency response difference value. The meaning of this frequency response difference value is that the user may have a weak ability to distinguish sounds of certain frequencies. Therefore, through the audio calculation The unit 204 can compensate the corresponding audio frequency according to the frequency response difference. In this way, the user can not only have the experience of surround sound effect when listening, but also can obtain hearing compensation for the frequency audio with poor identification ability, further optimizing listening feelings. In particular, there are many methods of compensation. In this embodiment, for example, the compensation is the default value, but this application is not limited thereto. Those skilled in the art can also compensate according to the design or the hearing characteristics of the user. for other values.

More specifically, the audio computing unit 204 performs filtering processing on the received center left channel audio Cent_L, center right channel audio Cent_R, side left channel audio Side_L, and side right channel audio Side_R. In the embodiment, for example, high-pass filtering is used to filter out bands that are relatively insensitive to human hearing sensitivity and retain bands that are more sensitive to human hearing sensitivity, but this application is not limited thereto. Those skilled in the art can also perform filtering in different bands according to design or user characteristics. The audio operation unit 204 performs convolution operation on the filtered audio and a default head transfer function. The convolution operation is known to those skilled in the art, so details are not repeated here. After the convolution operation, the audio can be virtualized to various directions relative to the user, thereby generating a listening experience of surround sound. In particular, the audio operation unit 204 can also multiply the audio after the convolution operation by a default parameter, and the preset parameter can be any value, which can optimize the intensity of the audio according to requirements.

In addition, the audio synthesis unit 205 can also receive the center left channel audio Cent_L and the center right channel audio Cent_R without the head transfer function processing, and combine the received audio without the head function transfer processing with the head transfer function The audio processed by the function is synthesized together, that is, the band that is less sensitive to human hearing sensitivity is synthesized with the band that is more sensitive. In this way, the synthesized audio can cover a wider band, thereby increasing the final The richness of the output audio.

To sum up, the audio processing method and audio processing device provided by the present application simulate the audio of the original left and right channels into multi-channel audio through audio separation and calculation of the head transfer function, and the simulated audio can be further adjusted. The audio is relative to the user's source direction, and can further compensate the corresponding audio frequency for the user's hearing characteristics, so that the user can experience the optimized experience of surround sound effect and hearing compensation.

The embodiments and/or implementations described above are only used to illustrate the preferred embodiments and/or implementations of the technology of the present application, and are not intended to limit the implementation of the technology of the application in any form. Personnel, without departing from the scope of the technical means disclosed in this application, may make some changes or modifications to other equivalent embodiments, but it should still be regarded as the technology or embodiment substantially the same as this application.

Claims (10)

1. An audio processing method, comprising:

separating a left channel audio into a central left channel audio and a side left channel audio;

separating a right channel audio into a center right channel audio and a side right channel audio;

performing a center head transfer function process on the center left channel audio and the center right channel audio to simulate the center left channel audio and the center right channel audio as a first sound source position and a second sound source position relative to a user;

performing a side head transfer function on the side left channel audio and the side right channel audio to simulate the side left channel audio and the side right channel audio into a third audio source position and a fourth audio source position relative to the user; and

and performing frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the hearing characteristic of the user to synthesize the audio into a binaural audio.

2. The audio processing method according to claim 1, wherein the step of performing the frequency compensation according to the hearing ability characteristic of the user comprises:

playing a plurality of audio frequencies with different frequencies to the user;

generating a plurality of frequency response values corresponding to the audios with different frequencies to obtain the hearing ability characteristic of the user;

comparing the frequency response values with at least one default value to generate at least one frequency response difference value; and

compensating the audios with different frequencies according to the at least one frequency response difference.

3. The audio processing method of claim 2, wherein the step of compensating the audios with different frequencies compensates the frequency response values corresponding to the audios with different frequencies to the default value.

4. The audio processing method of claim 1, wherein the audio processed by the center header transfer function and the side header transfer function is processed by a filtering and a convolution operation.

5. The audio processing method according to claim 4, wherein the audio after the filtering and the convolution operation is further multiplied by a default parameter.

6. An audio processing apparatus, comprising:

a channel separation unit, receiving a left channel audio and a right channel audio, separating the left channel audio into a center left channel audio and a side left channel audio, and separating the right channel audio into a center right channel audio and a side right channel audio;

an audio operation unit, receiving and performing a center head transfer function process on the center left channel audio and the center right channel audio, so as to simulate the center left channel audio and the center right channel audio into a first sound source position and a second sound source position relative to a user, and receiving and performing a side head transfer function process on the side left channel audio and the side right channel audio, so as to simulate the side left channel audio and the side right channel audio into a third sound source position and a fourth sound source position relative to the user; and

an audio synthesizing unit, which receives and carries out a frequency compensation to the audio processed by the central head function transfer and the side head transfer function according to the hearing characteristic of the user so as to synthesize a binaural audio.

7. The audio processing device as claimed in claim 6, wherein the audio computing unit has a filter for performing a filtering process on the center left channel audio, the center right channel audio, the side left channel audio and the side right channel audio.

8. The audio processing device as claimed in claim 7, wherein the audio arithmetic unit performs a convolution operation on the filtered center left channel audio and the center right channel audio and the center head transfer function.

9. The audio processing apparatus of claim 7, wherein the audio arithmetic unit performs a convolution operation on the filtered side-left channel audio and the filtered side-right channel audio and the side-header transfer function.

10. The audio processing device according to claim 7, further comprising:

the audio frequency operation unit generates a plurality of frequency response values according to the audio frequencies with different frequencies so as to obtain the hearing characteristic of the user; and

the audio frequency operation unit compensates the audio frequencies with different frequencies according to the at least one frequency response difference.

Images