CN115604646B - Panoramic deep space audio processing method - Google Patents
Panoramic deep space audio processing method Download PDFInfo
- Publication number
- CN115604646B CN115604646B CN202211487247.1A CN202211487247A CN115604646B CN 115604646 B CN115604646 B CN 115604646B CN 202211487247 A CN202211487247 A CN 202211487247A CN 115604646 B CN115604646 B CN 115604646B
- Authority
- CN
- China
- Prior art keywords
- audio signal
- spatial audio
- measuring
- sound source
- spatial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/16—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
- G10K11/175—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound
- G10K11/178—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase
- G10K11/1781—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions
- G10K11/17813—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions characterised by the analysis of the acoustic paths, e.g. estimating, calibrating or testing of transfer functions or cross-terms
- G10K11/17819—Methods or devices for protecting against, or for damping, noise or other acoustic waves in general using interference effects; Masking sound by electro-acoustically regenerating the original acoustic waves in anti-phase characterised by the analysis of input or output signals, e.g. frequency range, modes, transfer functions characterised by the analysis of the acoustic paths, e.g. estimating, calibrating or testing of transfer functions or cross-terms between the output signals and the reference signals, e.g. to prevent howling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/307—Frequency adjustment, e.g. tone control
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
A panoramic deep space audio processing method belongs to the technical field of signal processing. The method is realized based on an audio processing system comprising a human head spherical simulator, a plurality of measuring microphones and a processor; the method is performed by a processor; the method comprises the following steps: step S01, receiving spatial audio signals collected by a plurality of measuring microphones; s02, carrying out fast Fourier transform algorithm analysis on the spatial audio signal obtained in the step S01 to obtain the frequency spectrum and the waveform of the spatial audio signal; step S03, reducing the spatial audio signal converted in the step S02 by using an anti-howling algorithm; and S04, positioning the sound source position based on the spatial audio signal acquired in the step S01, and carrying out equalization adjustment on the amplitude of the spatial audio signal reduced in the step S03 according to the sound source position. The invention can simulate the spatial audio collection of the head-mounted equipment at low cost, and adjust the collected spatial audio based on the spatial position of the sound source, so that the sound volume demand of most users is met by playing.
Description
Technical Field
The invention relates to the technical field of signal processing, in particular to a panoramic deep space audio processing method.
Background
In daily human activities, we are constantly in full depth of field spatial audio, i.e. sound can be transmitted from any direction of all directions. However, most of the currently used sound collection devices can only collect sound mixed in a certain direction or a certain point, and there are few methods for collecting panoramic deep spatial audio, and there are various defects in processing collected spatial audio signals. For example, VR glasses using the FOA (first order audio) technology can only collect a small amount of data although they are small in size, and need to perform very complicated algorithm processing to finally restore the current sound field information, so that the degree of restoration is general, and the playback audio cannot be adjusted according to the spatial position of the sound source. Although VR glasses adopting QB (quad baural) technology can collect more data and have better reduction degree, the equipment volume is overlarge and is only used in some large occasions; and also fails to adjust the playback audio according to the spatial location of the sound source.
The invention patent application CN201610550721.9 discloses a panoramic audio processing method, and specifically discloses that the method comprises the following steps: the server side obtains audio signals with different formats, transcodes the audio signals with different formats to obtain audio signals with an intermediate format, and transcodes the audio signals with different formats to uniform audio signals with the intermediate format; superposing the intermediate format audio signals to obtain intermediate format audio signals output to the client; the client acquires the intermediate format audio signal output to the client from the server; acquiring the rotation direction of the head; decoding to obtain a panoramic sound binaural signal; the audio signals of different formats comprise multi-directional binaural recording signals, sound field recording signals and audio object signals; the intermediate format audio signal comprises a plurality of paths of binaural signals and Ambisonic signals; the server side is used for the condition that the intermediate format audio signal is a multi-channel binaural signal: uniformly transcoding the input multidirectional binaural recording signal, the sound field recording signal and the audio object signal into a multipath binaural signal; the audio signals with different formats are transcoded into multi-channel binaural format audio signals which are superposed and output to a client; the client receives the multi-channel binaural format audio signals from the server, and interpolates and reduces the played panoramic sound field according to the head rotating direction; for the condition that the intermediate format audio signal is an Ambisonic signal, the server side comprises the following steps: uniformly transcoding the input multidirectional binaural recording signal, the sound field recording signal and the audio object signal into an Ambisonic signal; overlapping Ambisonic format signals transcoded into different formats and outputting the signals to the client; the client receives the Ambisonic format signal from the server, decodes the Ambisonic format signal and restores and plays the panoramic sound field according to the head rotation direction; and the server side transcodes the audio signals with different formats into a plurality of intermediate format audio signals at the same time in a mixed manner or dynamically. The invention mainly aims at carrying out panoramic processing on audio signals with different formats acquired by a server side and playing back the audio signals to a client side, but not aims at carrying out panoramic processing on the audio signals acquired in a real-time state.
The invention patent application CN201910867044.7 discloses a 360-degree sound source real-time playback system, and specifically discloses that the system comprises: the main body is composed of an audio data helmet, the audio data helmet is composed of a helmet frame, a microphone array, earphones and a processing system, wherein the earphones are arranged according to the positions of human ears, a 360-degree sound source real-time playback processing hardware module realizes the functions of head position perception, sound source positioning and HRTF-based audio reconstruction processing, and the audio data helmet is assembled inside the helmet frame along the inner contour by adopting a flexible manufacturing process; the microphone array is used for the audio data acquisition function of the sound source positioning function and is arranged on the helmet; the 360-degree sound source real-time playback processing hardware module comprises an HRTF (head related transfer function) calculation module and a signal processing module; the HRTF calculation module is used for positioning the head posture by using a position sensor, positioning a sound source signal according to the microphone array, further determining the direction of the current sound relative to the head, then retrieving an HRTF database by using position information and calculating HRTF data of the direction by using an HRTF interpolation algorithm; the HRTF database is preset and stored in a memory of the 360-degree sound source real-time playback processing hardware module when leaving a factory; the signal processing module is used for filtering the data by utilizing the calculated HRTF so as to generate a sound effect with azimuth sensation; the read interval of the head angle information is required to be consistent with the length of one extracted frame of data, and each frame of data corresponds to one set of read head angle information. The microphone array is arranged at the helmet to collect audio in the environment, then sound source positioning is carried out on the audio data collected in real time, and finally the generated sound effect is the sound effect with the azimuth. However, the invention does not adjust the sound pressure according to the spatial position of the sound source so as to output the playback audio meeting the sound volume requirement.
The invention patent application CN202110982814.X discloses a howling suppression method, and specifically discloses a method comprising the following steps: judging whether a howling event exists according to an audio signal collected by a microphone; if yes, determining a howling type corresponding to the howling event; if the howling type is shock wave type howling, reducing amplitude gain of a frequency band where the howling event is located, and filtering the audio signal with the amplitude gain reduced to obtain a target audio signal; and controlling the loudspeaker to play the target audio signal. The method mainly processes the collected audio signals and reduces the amplitude gain of the frequency band according to the howling type. The invention does not expand on how to reduce and to what extent, and the sound pressure is not adjusted according to the space position of the sound source.
Disclosure of Invention
The invention aims to provide a panoramic deep space audio processing method which can simulate the spatial audio collection of head-mounted equipment at low cost and adjust the collected spatial audio based on the spatial position of a sound source so that the played audio meets the sound volume demand of most users.
The invention provides a panoramic deep space audio processing method, which is realized based on an audio processing system comprising a human head spherical simulator, a plurality of measuring microphones uniformly distributed on the spherical surface of the human head spherical simulator and a processor; the method is performed by a processor; the method comprises the following steps:
step S01, receiving spatial audio signals collected by a plurality of measuring microphones;
s02, carrying out fast Fourier transform algorithm analysis on the spatial audio signal obtained in the step S01 to obtain the frequency spectrum and the waveform of the spatial audio signal;
step S03, reducing the spatial audio signal converted in the step S02 by using an anti-howling algorithm;
and S04, positioning the sound source position based on the spatial audio signal acquired in the step S01, and carrying out equalization adjustment on the amplitude of the spatial audio signal reduced in the step S03 according to the sound source position.
The invention utilizes the human head spherical simulator distributed with the measuring microphones to simulate the spatial audio signal acquisition of the head-mounted equipment, arranges different numbers of measuring microphones according to different types of the head-mounted equipment, and arranges a plurality of measuring microphones in different required areas.
Then, the collected spatial audio signals are processed, and the reduction processing is carried out by utilizing the sound with overhigh howling resisting algorithm, so that the damage to a human body caused by overhigh sound pressure level of the sound is avoided. Meanwhile, the reduced spatial audio signals are equalized based on the sound source position. Since the sound sources are from various positions in space, the received spatial audio signals are different due to different positions, and most obviously, the spatial audio amplitude is different. If each spatial audio signal is subjected to unified anti-howling processing, the processed signal does not consider the influence of the spatial position of the sound source on the spatial audio signal. Especially in confined spaces, such as the area of a room, can affect the acoustic transmission. Therefore, the invention carries out equalization adjustment on the spatial audio signal after the anti-howling processing by combining the spatial position of the sound source through positioning the position of the sound source so as to generate the sound effect which better meets the sound volume demand of a user.
Preferably, the step S03 includes:
step S31, when the peak value in the space audio signal transformed in the step S02 is judged to be larger than the processing threshold value, half-wave rectification processing is carried out on the space audio signal;
step S32, subtracting the average noise from the half-wave rectified spatial audio signal in step S31 to obtain a spatial audio signal after the clipping processing.
Preferably, the process of locating the sound source position in step S04 is as follows:
step S41, microphone signals of measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator are defined as positive and negative opposite signals by utilizing a polarity test algorithm;
step S42, sequencing and coding the measuring microphones on the human head spherical simulator according to the time sequence of the received space audio signals;
step S43, judging the time sequence of receiving the space audio signal between two symmetrical measuring microphones in the same group, and determining the approximate direction of the sound source through the microphone signals by taking the measuring microphone which receives the space audio signal firstly as a reference; and the spatial position of the sound source is located based on the distance between the two measuring microphones, the spatial positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively.
Preferably, step S41 specifically includes:
step S411, dividing a plurality of measuring microphones distributed on the human head spherical simulator into a plurality of groups, wherein each group of measuring microphones comprises two measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator;
step S412, defining microphone signals of a plurality of measuring microphones symmetrically distributed on the front hemisphere of the human head sphere simulator as +1, and defining microphone signals of a plurality of measuring microphones symmetrically distributed on the rear hemisphere of the human head sphere simulator as-1; or defining microphone signals of a plurality of measuring microphones symmetrically distributed on a rear hemisphere of the human head sphere simulator as +1, and defining microphone signals of a plurality of measuring microphones symmetrically distributed on a front hemisphere of the human head sphere simulator as-1;
step S413, the spatial position and the microphone signal of each measurement microphone in each group of measurement microphones are stored in association with each other in units of groups.
Preferably, the step S42 includes:
step S421, amplifying the time of receiving the spatial audio signal by using a time delay algorithm;
step S422, the amplified time of receiving the spatial audio signal is arranged according to the receiving sequence, and the measuring microphones corresponding to the receiving time are encoded according to the receiving sequence.
Preferably, the step S43 includes:
step S431, acquiring one measuring microphone which receives the spatial audio signal at the earliest in all the measuring microphones in the step S42, and determining the approximate direction of the sound source according to the microphone signal of the measuring microphone;
and step S432, obtaining another measuring microphone which is symmetrical to the measuring microphone in the step S431, and positioning the space position of the sound source according to the distance between the two measuring microphones, the space positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively.
Preferably, in step S04, the process of performing equalization adjustment on the amplitude of the spatial audio signal after being reduced in step S03 according to the sound source position is as follows:
obtaining influence frequencies according to the sound source positions obtained by positioning by contrasting a sound-frequency response curve library;
obtaining an amplitude adjustment coefficient according to the influence frequency and the amplitude of the spatial audio signal reduced in the step S03 by referring to the sound frequency band adjustment table;
adjusting the amplitude of the spatial audio signal reduced in the step S03 according to the amplitude adjustment coefficient;
wherein the sound-frequency response curve library stores sound-frequency response curves at different spatial positions; the sound band adjustment table is a table in which frequency, amplitude and amplitude adjustment coefficients are stored.
Preferably, the method further includes step S05, obtaining a loudness curve by using a loudness curve algorithm based on the spatial audio signal after equalization adjustment.
Preferably, the step S05 further includes: selecting a curve which is closest to the loudness curve in the equal loudness curve library, and adjusting the loudness curve by taking the curve as a reference curve; the equal loudness curve storage stores a plurality of equal loudness curves.
Preferably, the error between the adjusted loudness curve and the reference curve is ± 3dB.
The invention has the following beneficial effects:
the invention relates to a panoramic deep space audio processing method, which comprises the following steps that on one hand, a human head spherical simulator with a plurality of measuring microphones distributed thereon is utilized to carry out real-time space audio acquisition, and the space audio acquisition of head-mounted equipment can be simulated at low cost; on the other hand, the acquired spatial audio signals are subjected to howling resistance treatment, average noise is introduced to carry out high-frequency amplitude reduction treatment, and then the amplitude is subjected to balance adjustment by combining the spatial position of a sound source so as to generate a sound effect which is more in line with the sound volume demand of a user; in addition, the loudness is adjusted according to the equal loudness curve, so that a user can obtain better hearing feeling.
Drawings
FIG. 1 is a flow chart of a panoramic deep space audio processing method according to the present invention;
fig. 2 is a block diagram of a human head spherical simulator, above which a plurality of measuring microphones are distributed.
Detailed Description
The following are specific embodiments of the present invention and are further described with reference to the drawings, but the present invention is not limited to these embodiments.
The invention relates to a panoramic deep space audio processing method which is realized based on an audio processing system comprising a human head spherical simulator, a plurality of measuring microphones uniformly distributed on the spherical surface of the human head spherical simulator and a processor.
In fig. 2 a human head sphere simulator is shown, comprising a human head simulator structure 1 and a head and neck simulator structure 2. The spherical surface of the human head simulation structure 1 is distributed with a plurality of measuring microphones 3. The measuring microphone needs to meet the following requirements: sensitivity error + -2 dB (e.g., -26dB + -2 dB); the dynamic range of frequency response is from 20Hz to 169hz; background noise inherent: <20dBA; phase position: 20Hz to 5kHz < -3 degrees; the vibrating diaphragm of the measuring microphone can be 0 to 40mm higher than the spherical surface of the human head simulator; measuring that the error of the installation height between the microphones is less than 1mm; the number of measuring microphones may be 5 to 128 (47 are shown); the invention utilizes the principle that the time for transmitting sound to each microphone is different to carry out panoramic deep acoustic positioning; the human head spherical simulator is internally hollow and is used for arranging a circuit and a cable; the diameter of the human head simulation structure is 180mm-240mm; the head and neck structure is hollow in design, the height is 100mm-200mm, and the head and neck structure is used for fixing the invention and can be internally provided with circuits such as circuits or cable routing and the like. The measuring microphone is connected with the processor through a line, and the acquired spatial audio signals are sent to the processor for processing and analysis.
As shown in fig. 1, the method of the present invention comprises:
step S01, receiving spatial audio signals collected by a plurality of measuring microphones;
s02, carrying out fast Fourier transform algorithm analysis on the spatial audio signal obtained in the step S01 to obtain the frequency spectrum and the waveform of the spatial audio signal;
step S03, reducing the spatial audio signal converted in the step S02 by using an anti-howling algorithm;
and S04, positioning the sound source position based on the spatial audio signal acquired in the step S01, and carrying out equalization adjustment on the amplitude of the spatial audio signal reduced in the step S03 according to the sound source position.
In step S02, the spatial audio signal is processed by using FFT algorithm, and the time domain signal sequence is processed in the processing process
According to two parts, i.e. front and rear, i.e.
For the above formula to be adjusted as
. Thereafter, the parity by k is divided into two groups, i.e.
And
in which
、
、
The number of the sampling points is Fourier series, and k, 2l and 2l +1 are sampling points k, 2l and 2l +1 respectively; n represents the total number of sampling points, N represents the sampling point selected at the moment, and W is the unitThe strength of the frequency signal. After two N/2 point DFT operations are obtained, the decomposition is carried out in this way, iteration is carried out, and finally the frequency spectrum and the waveform of the spatial audio signal are obtained.
The step S03 includes:
step S31, when the peak value in the space audio signal transformed in the step S02 is judged to be larger than the processing threshold value, half-wave rectification processing is carried out on the space audio signal;
step S32, subtracting the average noise from the half-wave rectified spatial audio signal in step S31 to obtain a spatial audio signal after the clipping processing.
Since the frequency point corresponding to the sound which generates a harsh or roaring sound is the frequency corresponding to the peak point on the curve, the peak point is howling first. In order to avoid damage to a human body due to the fact that the sound pressure level of the sound is too high, the excessive sound is reduced through the howling resisting algorithm. Firstly, the spatial audio signal processed in step S02 needs to be judged, and when the peak value is judged to be greater than the processing threshold (such as 160 dB), the spatial audio signal is processed, and half-wave rectification processing is performed first
，
Wherein
In order to be the amplitude of the signal,
for a signal amplitude larger than 0 and,
the signal amplitude is processed for a half-wave,
the amplitude of the signal after the half-wave rectification process. Thereafter, an offset subtraction operation is performed:
where k is the kth band frequency in the L band,
to subtract the signal amplitude after the offset,
for the total signal amplitude received after half-wave rectification,
is offset, i.e. average noise.
After avoiding the peak position, the average sound pressure in the language range can be obviously improved, thereby meeting the requirements of most listeners on sound volume perception, and simultaneously ensuring the system to be stable. However, we have found that the location where the sound source occurs also has some effect on the perception of sound volume. The sound pressure-frequency response curve of a room refers to the response curve of any point position in the room space under the condition that the space position and the angle of sound generated by a room sound box are fixed. When the spatial position changes, the curve also changes, and the corresponding peak-to-valley frequency position on the curve also changes. In practice, the sound pressure-frequency response curve of a reference point of a room is influenced more heavily by the large-area reflecting surface of the room, and the rest of the peak-valley points are influenced more by the reflecting surface near the reference point. For this purpose, after the processing of step S03, the present invention further performs equalization adjustment on the amplitude of the spatial audio signal by introducing the sound source position.
And the step S04 comprises positioning to obtain the sound source position, and then performing equalization adjustment on the amplitude of the spatial audio signal reduced in the step S03 according to the sound source position.
The sound source position locating process can be obtained through various existing locating algorithms, and can also be realized by adopting a simple locating method under the scheme. The process of locating the sound source position in step S04 is as follows:
and S41, defining microphone signals of the measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator as positive and negative opposite signals by using a polarity test algorithm.
Specifically, firstly, a plurality of measuring microphones distributed on a human head spherical simulator are divided into a plurality of groups, and each group of measuring microphones comprises two measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator; the connecting line between the two ear parts is used as a central axis to distinguish the front hemisphere from the rear hemisphere, and all the measuring microphones are symmetrically distributed. Secondly, microphone signals of a plurality of measuring microphones symmetrically distributed on the front hemisphere of the human head sphere simulator are defined as +1, and microphone signals of a plurality of measuring microphones symmetrically distributed on the rear hemisphere of the human head sphere simulator are defined as-1; or, the microphone signals of a plurality of measuring microphones symmetrically distributed on the rear hemisphere of the human head sphere simulator are defined as +1, and the microphone signals of a plurality of measuring microphones symmetrically distributed on the front hemisphere of the human head sphere simulator are defined as-1. And finally, the spatial position of each measuring microphone in each group of measuring microphones and the microphone signals are stored in a related mode by taking the group as a unit. For example, the microphones 1 and 2 are symmetrically arranged, the microphone 1 is located in the front hemisphere, and the microphone 2 is located in the rear hemisphere, and the storage is performed in the manner of (C1, +1, (X1, Y1); C2, -1, (X2, Y2)).
Step S42, sequencing and coding the measuring microphones on the human head spherical simulator according to the time sequence of the received space audio signals;
for example, the sorting is performed according to the time sequence, and the sorting code is the time sequence code.
Step S43, judging the time sequence of receiving the space audio signal between two symmetrical measuring microphones in the same group, and determining the approximate direction of the sound source through the microphone signals by taking the measuring microphone which receives the space audio signal firstly as a reference; and the spatial position of the sound source is located based on the distance between the two measuring microphones, the spatial positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively.
The spatial position of the measuring microphone from which the earliest received spatial audio signal originates is obtained, as well as the spatial position of the measuring microphone arranged symmetrically to it. Sound source localization is performed based on the set of measurement microphones.
In order to achieve accurate sequencing, the time of the received spatial audio signal needs to be amplified, for example, by using a time delay algorithm. The step S42 includes: step S421, the time of receiving the spatial audio signal is amplified by using a time delay algorithm, for example, a time delay of 200ms is added for each collected acoustic signal. Step S422, the amplified time of receiving the spatial audio signal is arranged according to the receiving sequence, and the measuring microphones corresponding to the receiving time are encoded according to the receiving sequence.
The specific positioning manner is as follows, and the step S43 specifically includes:
step S431, acquiring one measuring microphone which receives the spatial audio signal at the earliest in all the measuring microphones in the step S42, and determining the approximate direction of the sound source according to the microphone signal of the measuring microphone;
and step S432, obtaining another measuring microphone which is symmetrical to the measuring microphone in the step S431, and positioning the space position of the sound source according to the distance between the two measuring microphones, the space positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively.
The step S432 specifically includes: determining a first local spherical surface where a sound source space position in the approximate direction of the sound source is located by taking a first measuring microphone in the same group of measuring microphones as a reference point and taking the distance from the first measuring microphone to the sound source as a radius; determining a second local spherical surface where the space position of the sound source in the approximate direction of the sound source is located by taking a second measuring microphone in the same group of measuring microphones as a reference point and the distance of the second measuring microphone as a radius; and determining the intersection part of the first local spherical surface and the second local spherical surface as the sound source space position. The specific space position is determined by utilizing a trigonometric function according to two fixed points (two measuring microphones in the same group) and two distance (the distance length is determined by the time and the sound speed of each measuring microphone receiving the sound source).
The process of performing equalization adjustment on the amplitude of the spatial audio signal after being reduced in the step S03 according to the sound source position in the step S04 is as follows:
obtaining influence frequencies according to the sound source positions obtained by positioning by contrasting a sound-frequency response curve library;
comparing with the sound frequency band adjusting table, and obtaining an amplitude adjusting coefficient according to the influence frequency and the amplitude of the space audio signal reduced in the step S03;
adjusting the amplitude of the spatial audio signal reduced in the step S03 according to the amplitude adjustment coefficient;
the sound-frequency response curve base stores sound-frequency response curves at different spatial positions, the sound-frequency response curves are obtained according to actual detection and stored in the base, and each sound-frequency response curve is stored corresponding to a spatial position. After the spatial position of the sound source is obtained, a sound-frequency response curve at the spatial position is obtained from the library, and then peak/valley point frequencies are selected as influence frequencies. The sound band adjustment table is a table (as shown in the table) storing frequency, amplitude and amplitude adjustment coefficients, and the table is provided with 15 sections of amplitude adjustment coefficients under different frequencies and amplitudes.
Table-sound frequency band regulating table
For example, when the influencing frequency is 1000Hz, a corresponding amplitude adjustment coefficient is selected according to the amplitude of the spatial audio signal after being reduced in step S03, and when the amplitude is 140dB, the adjustment process of subtracting 20 is performed on the amplitude.
The method also comprises a step S05 of obtaining a loudness curve by using a loudness curve algorithm based on the space audio signal after the equalization adjustment. The loudness curve of the acoustic signal is obtained, for example, according to the loudness curve algorithm of ISO-532.
The step S05 further includes: selecting a curve which is closest to the loudness curve in the equal loudness curve library, and adjusting the loudness curve by taking the curve as a reference curve; the equal loudness curve library stores a plurality of equal loudness curves, wherein the equal loudness curves range from 0Phon (the lowest curve) to 90Phons (the highest curve), the curve intervals are 10Phons, the curves are stored in the processor as standard curves, and after the loudness curves are detected, the curve correction is carried out on the test curves by taking the closest equal loudness curve as a reference curve, so that the error between the detected loudness curves and the reference curve is +/-3 dB.
Based on the process, the invention can detect the spatial audio signals collected by various head-mounted devices, adjust the collected signals in terms of sound pressure and loudness, and output sound effects meeting the requirements of the user on sound volume and hearing comfort under the consideration of the spatial position of a sound source.
It will be appreciated by persons skilled in the art that the embodiments of the invention described above and shown in the drawings are given by way of example only and are not limiting of the invention. The objects of the present invention have been fully and effectively accomplished. The functional and structural principles of the present invention have been shown and described in the examples, and any variations or modifications of the embodiments of the present invention may be made without departing from the principles.
Claims (7)
1. A panoramic deep space audio processing method is characterized in that the method is realized based on an audio processing system comprising a human head spherical simulator, a plurality of measuring microphones uniformly distributed on the spherical surface of the human head spherical simulator and a processor; the method is performed by a processor; the method comprises the following steps:
step S01, receiving spatial audio signals collected by a plurality of measuring microphones;
s02, carrying out fast Fourier transform algorithm analysis on the spatial audio signal obtained in the step S01 to obtain the frequency spectrum and the waveform of the spatial audio signal;
step S03, reducing the spatial audio signal converted in the step S02 by using an anti-howling algorithm; the step S03 includes:
step S31, when the peak value in the space audio signal transformed in the step S02 is judged to be larger than the processing threshold value, half-wave rectification processing is carried out on the space audio signal;
step S32, subtracting the average noise from the half-wave rectified spatial audio signal in step S31 to obtain a spatial audio signal after reduction processing;
s04, positioning a sound source position based on the spatial audio signal acquired in the S01, and carrying out balance adjustment on the amplitude of the spatial audio signal reduced in the S03 according to the sound source position;
the process of locating the sound source position in step S04 is as follows:
step S41, microphone signals of the measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator are defined as positive and negative opposite signals by utilizing a polarity testing algorithm;
step S42, according to the time sequence of the received spatial audio signals, sequencing and coding the measuring microphones on the head spherical simulator;
step S43, judging the time sequence of receiving the space audio signal between two symmetrical measuring microphones in the same group, and determining the approximate direction of the sound source through the microphone signals by taking the measuring microphone which receives the space audio signal firstly as a reference; positioning the spatial position of the sound source based on the distance between the two measuring microphones, the spatial positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively;
in step S04, the process of performing equalization adjustment on the amplitude of the spatial audio signal reduced in step S03 according to the sound source position is as follows:
obtaining influence frequencies according to the sound source positions obtained by positioning by contrasting a sound-frequency response curve library;
obtaining an amplitude adjustment coefficient according to the influence frequency and the amplitude of the spatial audio signal reduced in the step S03 by referring to the sound frequency band adjustment table;
adjusting the amplitude of the spatial audio signal reduced in the step S03 according to the amplitude adjustment coefficient;
wherein the sound-frequency response curve library stores sound-frequency response curves at different spatial positions; the sound band adjustment table is a table in which frequency, amplitude and amplitude adjustment coefficients are stored.
2. The method for processing the panoramic deep space audio according to claim 1, wherein step S41 specifically comprises:
step S411, dividing a plurality of measuring microphones distributed on the human head spherical simulator into a plurality of groups, wherein each group of measuring microphones comprises two measuring microphones symmetrically distributed on the front hemisphere and the rear hemisphere of the human head spherical simulator;
step S412, defining microphone signals of a plurality of measuring microphones symmetrically distributed on the front hemisphere of the human head sphere simulator as +1, and defining microphone signals of a plurality of measuring microphones symmetrically distributed on the rear hemisphere of the human head sphere simulator as-1; or defining microphone signals of a plurality of measuring microphones symmetrically distributed on a rear hemisphere of the human head sphere simulator as +1, and defining microphone signals of a plurality of measuring microphones symmetrically distributed on a front hemisphere of the human head sphere simulator as-1;
step S413, the spatial position and the microphone signal of each measurement microphone in each group of measurement microphones are stored in association with each other in units of groups.
3. The method of claim 1, wherein the step S42 comprises:
step S421, amplifying the time of receiving the spatial audio signal by using a time delay algorithm;
step S422, the amplified time of receiving the spatial audio signal is arranged according to the receiving sequence, and the measuring microphones corresponding to the receiving time are encoded according to the receiving sequence.
4. The method of claim 1, wherein the step S43 comprises:
step S431, acquiring one measuring microphone which receives the spatial audio signal at the earliest in all the measuring microphones in the step S42, and determining the approximate direction of the sound source according to the microphone signal of the measuring microphone;
and step S432, obtaining another measuring microphone which is symmetrical to the measuring microphone in the step S431, and positioning the space position of the sound source according to the distance between the two measuring microphones, the space positions of the two measuring microphones and the distances between the two measuring microphones and the sound source respectively.
5. The method of claim 1, further comprising a step S05 of obtaining a loudness curve by a loudness curve algorithm based on the spatial audio signal after equalization adjustment.
6. The method for panoramic deep space audio processing according to claim 5, wherein said step S05 further comprises: selecting a curve which is closest to the loudness curve in the equal loudness curve library, and adjusting the loudness curve by taking the curve as a reference curve; the equal loudness curve storage stores a plurality of equal loudness curves.
7. The method of claim 6, wherein the adjusted loudness curve has an error of ± 3dB from the reference curve.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211487247.1A CN115604646B (en) | 2022-11-25 | 2022-11-25 | Panoramic deep space audio processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211487247.1A CN115604646B (en) | 2022-11-25 | 2022-11-25 | Panoramic deep space audio processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115604646A CN115604646A (en) | 2023-01-13 |
| CN115604646B true CN115604646B (en) | 2023-03-21 |
Family
ID=84853229
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211487247.1A Active CN115604646B (en) | 2022-11-25 | 2022-11-25 | Panoramic deep space audio processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115604646B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5710823A (en) * | 1994-09-09 | 1998-01-20 | Yamaha Corporation | Howling remover having cascade connected equalizers suppressing multiple noise peaks |
| EP1843636A1 (en) * | 2006-04-05 | 2007-10-10 | Harman Becker Automotive Systems GmbH | Method for automatically equalizing a sound system |
| CN111556425A (en) * | 2020-04-20 | 2020-08-18 | 华南理工大学 | Tone equalization method for virtual sound reproduction of loudspeaker |
| WO2021203603A1 (en) * | 2020-04-10 | 2021-10-14 | 南京拓灵智能科技有限公司 | Howling suppression method and apparatus, and electronic device |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2119306A4 (en) * | 2007-03-01 | 2012-04-25 | Jerry Mahabub | Audio spatialization and environment simulation |
| CN102687536B (en) * | 2009-10-05 | 2017-03-08 | 哈曼国际工业有限公司 | System for the spatial extraction of audio signal |
| CN107340498A (en) * | 2016-05-03 | 2017-11-10 | 深圳光启合众科技有限公司 | The determination method and apparatus of robot and sound source position |
| CN106772244A (en) * | 2016-11-25 | 2017-05-31 | 北京明泰朗繁精密设备有限公司 | Sound source positioning system and method |
| GB201802850D0 (en) * | 2018-02-22 | 2018-04-11 | Sintef Tto As | Positioning sound sources |
| CN108616789B (en) * | 2018-04-11 | 2021-01-01 | 北京理工大学 | Personalized virtual audio playback method based on double-ear real-time measurement |
| CN110536215B (en) * | 2019-09-09 | 2021-06-29 | 普联技术有限公司 | Method and apparatus for audio signal processing, calculation device, and storage medium |
| CN113015078A (en) * | 2021-04-02 | 2021-06-22 | 广东思派康电子科技有限公司 | Anti-howling earphone debugging method and device in transparent mode |
| CN113645531B (en) * | 2021-08-05 | 2024-04-16 | 高敬源 | Earphone virtual space sound playback method and device, storage medium and earphone |
| CN114501224A (en) * | 2022-03-10 | 2022-05-13 | 北京小米移动软件有限公司 | Sound playback method, device, wearable device and storage medium |
| CN114664321A (en) * | 2022-04-20 | 2022-06-24 | 深圳魔耳智能声学科技有限公司 | Audio signal processing method, audio signal processing device, electronic equipment and storage medium |
-
2022
- 2022-11-25 CN CN202211487247.1A patent/CN115604646B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5710823A (en) * | 1994-09-09 | 1998-01-20 | Yamaha Corporation | Howling remover having cascade connected equalizers suppressing multiple noise peaks |
| EP1843636A1 (en) * | 2006-04-05 | 2007-10-10 | Harman Becker Automotive Systems GmbH | Method for automatically equalizing a sound system |
| WO2021203603A1 (en) * | 2020-04-10 | 2021-10-14 | 南京拓灵智能科技有限公司 | Howling suppression method and apparatus, and electronic device |
| CN111556425A (en) * | 2020-04-20 | 2020-08-18 | 华南理工大学 | Tone equalization method for virtual sound reproduction of loudspeaker |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115604646A (en) | 2023-01-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7489788B2 (en) | Recording a three dimensional auditory scene and reproducing it for the individual listener | |
| KR101333031B1 (en) | Method of and device for generating and processing parameters representing HRTFs | |
| EP3320692B1 (en) | Spatial audio processing apparatus | |
| CN110537221A (en) | Two stages audio for space audio processing focuses | |
| CN103165136A (en) | Audio processing method and audio processing device | |
| KR20050083928A (en) | Method for processing audio data and sound acquisition device therefor | |
| Sakamoto et al. | Sound-space recording and binaural presentation system based on a 252-channel microphone array | |
| CN113574596B (en) | Audio signal encoding method, audio signal decoding method, program, encoding device, audio system, and decoding device | |
| KR20180051411A (en) | Audio signal processing method and audio system | |
| CN110890100B (en) | Voice enhancement method, multimedia data acquisition method, multimedia data playing method, device and monitoring system | |
| CN115604646B (en) | Panoramic deep space audio processing method | |
| CN115376527A (en) | Three-dimensional audio signal coding method, device and coder | |
| WO2021212287A1 (en) | Audio signal processing method, audio processing device, and recording apparatus | |
| Vennerød | Binaural reproduction of higher order ambisonics-a real-time implementation and perceptual improvements | |
| Shabtai et al. | Spherical array beamforming for binaural sound reproduction | |
| Yuan et al. | Sound image externalization for headphone based real-time 3D audio | |
| Olgun et al. | Sound field interpolation via sparse plane wave decomposition for 6DoF immersive audio | |
| Kearney et al. | Depth perception in interactive virtual acoustic environments using higher order ambisonic soundfields | |
| CN110099351B (en) | Sound field playback method, device and system | |
| US20240292171A1 (en) | Systems and methods for efficient and accurate virtual accoustic rendering | |
| Ishi et al. | A sound-selective hearing support system using environment sensor network | |
| Oreinos et al. | Objective analysis of higher-order Ambisonics sound-field reproduction for hearing aid applications | |
| Giurda et al. | Evaluation of an ILD-based hearing device algorithm using Virtual Sound Environments | |
| Schulze-Forster | The B-Format–Recording, Auralization, and Absorption Measurements | |
| AU2002325063B2 (en) | Recording a three dimensional auditory scene and reproducing it for the individual listener |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |