CN101001485A - Finite sound source multi-channel sound field system and sound field analogy method - Google Patents

Abstract

A limited sound source multi-channel sound field system and sound field simulation method, comprising: a microphone array for recording M channels of audio information and detecting sound field characteristics; an audio collection subsystem for analog-to-digital conversion of each channel of audio information and packaging of audio data And the channel number, time stamp; the server processes the audio data of each microphone, completes the separation and processing of the sound source, and compresses and saves the data; according to the M-channel audio source data and the characteristics of the reconstructed sound field, mixes the audio source data and converts them into N-channel speakers output data and control signals; the audio reproduction subsystem, which composes audio data from different sources into multi-channel analog signals, and controls the audio synchronization of each speaker; the speaker array is used to play N audio signals. The invention can realize the separation and collection of sound sources, the dynamic weighted matching of M and N, the all-round accurate reproduction of the original sound field, the reduction of the influence of the phase distortion of the sound field, and the avoidance of intermodulation interference and other distortions in the processing, amplification and broadcasting of various sound sources.

Description

A limited sound source multi-channel sound field system and sound field simulation method

technical field

The invention relates to audio collection and restoration technology, in particular to a limited sound source multi-channel sound field system.

Background technique

Music is an important part of the history of human civilization. Since the invention of recording technology, people have been trying to record its beautiful melody faithfully. Acoustic reproduction technology has experienced the development from monophonic to dual-channel, five-channel and more channels. One of the goals that has always been pursued is to reproduce the original sound field or sense of space more realistically. Compared with two-channel stereo, although the multi-channel system has enabled people to obtain wonderful and extraordinary listening enjoyment, in fact, such a system cannot restore the sound source without distortion, and there are sound field phase distortion and sound source distortion. Issues such as intermodulation interference and dynamic range compression have encountered unprecedented technical complexity and theoretical challenges in the development of more advanced multi-channel sound field systems. Needless to say Bakelite records and tapes, even if the two-channel stereo sound source recorded by digital technology is played on the more popular 5.1/6.1-channel sound reproduction system today, it cannot truly reproduce the three-dimensional sense of space. In addition, traditional speakers all have a certain radiation axial angle, so there is a more obvious "emperor position" in the two-channel stereo system. This position is on the vertical line connecting the two speakers. Or when it deviates from this axis to the right, the sound image will obviously shift in proportion. Whether it is a home audio system or a professional theater sound reproduction system, there is a common problem of distortion of the direction of the sound source. In a multi-channel surround sound system, limited by each speaker, the sweet spot becomes a narrower point, that is, the midpoint of the area surrounded by each speaker. As long as you leave this point, the listener will be "swallowed" by the sound field of a certain speaker, resulting in a serious imbalance in the sound field.

Current stereo recording and playback formats are being replaced by surround sound formats. Various surround sound technologies have been developed in many artistic and technical applications to provide a sense of space to the ear through appropriate recording and playback methods. In this research, there are mainly two types of methods, one is perceptual simulation, and the other is sound field simulation.

Perceptual simulation method:

Binaural sound technology belongs to the perceptual simulation method. This type of technology believes that only by reproducing the sound pressure on the listener's eardrums can effectively reproduce the sense of auditory space. This is based on the well-known and verified facts - binaural and The head and chest can distinguish the direction and distance of the location of the sound source. Crosstalk cancellation technology is also binaural technology, which can eliminate crosstalk between the left speaker and the listener's right ear. The traditional techniques of sound recording and stereophony are based on observations of perceptual phenomena and some experience, generating guiding principles for artificial synthesis. At present, common multi-channel sound reproduction systems are designed using techniques based on perceptual simulation. At present, there are many technical specifications for various sound fields used by people. The most common surround sound formats are Dolby (Dolby), DTS (Digital Theater System, digital theater system), SACD (SuperAudio CD, Super Audio CD System) and DVD Audio ( DVD-Audio). Among them, SACD and DVD Audio are both high-resolution record formats. Movies do not use these two formats. The surround formats of movies mainly use DTS ES 6.1 and Dolby Digital EX 6.1.

Sound field simulation method:

Systems based on sound field simulations are rare because the technical and physical concepts are complex and require a deep background in acoustics and signal processing. Berkhout proposed Wave Field Synthesis (WFS) technology and Wave Field Analysis (WFA) theory in 1988 and 1997 respectively. Berkhout and J. Meyer carried out research on sound field analysis and sound field recording by microphone arrays around this theory, and Paul D. Henderson and X. Shen conducted research on sound field restoration using speaker arrays.

The basic assumption of sound field simulation is: to reproduce the sound field with spatially distributed sound pressure in the reproduction space, so that a complete auditory system (outer ear) is naturally stimulated. This stimulus is a virtual stimulus, which is exactly what is to be reproduced. . Obviously, this task is more natural in physics, but it is difficult to achieve it only by theoretical or intuitive spatial sound perception methods. Wave Field Synthesis (WFS) and holophonic (holophonic) are two sound field simulation methods, they both use an area surrounded by speakers to reproduce the sound field. Ambisonic is also a sound field simulation method in which the sound field can be partially reproduced at the center of the ambisonic surround speaker array. Generating a pattern around the reproduced sound source is also a sound field simulation technique.

The above studies are based on the Huygens'Principle to decompose and synthesize the sound field. Such a sound field reconstruction system theoretically needs an infinite number of secondary sound sources, which cannot be realized in practice, and requires a lot of simplification and approximation. At the same time, it is easy to cause sound source information distortion, sound field phase distortion, sound source intermodulation interference, and dynamic range compression. And other issues.

In order for the sound reproduction system to truly reproduce the original sound field, the research on recording technology is a key issue. How to record the music played by a symphony orchestra composed of instruments with different sounding methods and loudness in a harmonious, balanced, clear and accurate way is the key to the success of the recording. Modern recording technology often uses high-fidelity pickup microphones, adopts multi-microphone and multi-track recording methods, which is as close as possible to the real collected sound, and then remixes to balance the volume of each track, so that various instruments can be heard in one voice. A state of harmony and precision is displayed. Distortion-free sound source extraction is the problem of sound source separation, the purpose of which is to separate the target sound source from other interfering sound sources and noise signals. Statistical methods, such as neural networks, Hidden Markov Model (HMM), and Support Vector Machines (SVM), are currently commonly used methods in the field of sound source separation. However, the current statistical method in the application research of sound source separation has the limitation of assuming that the noise signal is Gaussian white noise. Therefore, the existing research on sound source separation can only be regarded as a kind of signal enhancement research, and when the signal noise is the interference of other sound sources (such as musical instruments), the existing solutions lose their application value due to the non-Gaussian nature of the system .

Contents of the invention

The technical problem to be solved by the present invention is to provide a limited sound source multi-channel sound field system and a sound field simulation method to realize undistorted sound source separation and collection, and to accurately reproduce the original sound field in all directions.

In order to solve the above technical problems, the present invention provides a limited sound source multi-channel sound field system, including: a microphone array with multiple microphones, a speaker array with multiple speakers, and also includes: an audio collection subsystem, a server, an audio sound subsystem, where,

The microphone array is used to record M-channel audio information and detect sound field characteristics;

The audio collection subsystem is used to perform analog-to-digital conversion on each audio signal collected from the microphone array, and mark the converted audio data with the collection channel number and time stamp, package and send it;

The server is used to receive and analyze the audio data packets sent by the audio acquisition subsystem during the sound field acquisition process, convert the audio data collected from each microphone array into different single sound source data, and convert and compress the converted single sound source data In the sound field restoration process, it is used to read the saved audio file, according to the characteristics of the M-channel sound source data and the reconstructed sound field, the audio source data is mixed and converted into N-channel speaker output through intelligent matching Data and control signals are sent to the audio reproduction subsystem;

The audio reproduction subsystem is used to synchronize the output data of each speaker received from the server according to the control signal received from the server, and restore it into a multi-channel analog audio signal, and send it to the speaker array for playback;

The speaker array is used to play N channels of audio signals and reconstruct the sound field.

In a preferred embodiment, the audio collection subsystem further includes: a plurality of audio collection sub-boards and an audio collection motherboard; each audio collection sub-board includes 1 or more audio collection channels, an analog-to-digital conversion device group and a logic processing device; the audio acquisition motherboard includes: an acquisition sub-board data interface and a server communication interface; wherein, each audio acquisition sub-board collects audio signals from the microphone array through an audio acquisition channel, and collects the audio acquisition channel to The audio signal of the analog-to-digital converter is sent to the analog-to-digital converter group, and the analog-to-digital converter group converts the audio information into audio data and sends it to the logic processing device, and marks the audio data output by each analog-to-digital converter in the analog-to-digital converter group with the channel number and time stamp, and send it to the data interface of the acquisition sub-board in the audio acquisition motherboard, and then send the audio data to the server through the server communication interface in the audio acquisition motherboard.

In a preferred embodiment, the audio collection motherboard further includes: a collection sub-board control interface; the server sends a control command to the audio collection sub-board through the collection sub-board control interface in the audio collection The sub-board control interface obtains the status information fed back by the audio collection sub-board.

In a preferred embodiment, the server further includes: a monitoring collection module, which is used to monitor whether audio data arrives at the server, and collects after listening to the arrival of audio data; an audio data processing module, including a particle filter and an equalizer The device is used to convert the collected audio data into different single audio source data; the storage module is used to convert and compress the converted single audio source data into an audio file format, add file description information and save it; the playback control module is used to read Take the saved audio files, mix the audio source data according to the characteristics of the M-channel sound source data and the reconstructed sound field, and convert them into the output data and control signals of the N-channel speakers through intelligent matching, and send them to the audio reproduction subsystem.

In a preferred embodiment, the audio sound reproduction subsystem further includes: a plurality of audio sound reproduction sub-boards and an audio sound reproduction motherboard; each audio sound reproduction sub-board includes one or more audio sound reproduction channels , a digital-to-analog converter group and a logic processing device; the audio reproduction motherboard includes: a sound reproduction sub-board control interface, a sound reproduction sub-board data interface and a server communication interface; wherein, the audio reproduction motherboard receives through the server communication interface Output data and control signals from each speaker of the server, and send the output data of each speaker to the audio reproduction sub-board through the data interface of the sound reproduction sub-board, and at the same time send the control signal to the audio reproduction sub-board through the control interface of the sound reproduction sub-board Daughter board; the logic processing device in the audio reproduction daughter board synchronizes the output data of each speaker received from the audio reproduction motherboard according to the control signal received from the audio reproduction motherboard, and sends it to the digital-to-analog converter group It is converted into a multi-channel analog audio signal and sent to the speaker array for playback through the audio reproduction channel.

In a preferred embodiment, the playback control module further includes a speaker volume control sub-module for controlling the volume of the speaker array.

In a preferred embodiment, the speaker volume control sub-module further includes a single speaker volume control unit for controlling the volume of a single speaker for the speaker array, a group speaker volume control unit for controlling the volume of group speakers, or a volume control unit for all speakers All speaker volume control units.

In a preferred embodiment, the playback control module further includes a loudspeaker array network monitoring sub-module for network monitoring of the loudspeaker array.

In order to solve the above-mentioned technical problems, the present invention also provides a limited sound source multi-channel sound field simulation method, comprising the following steps:

(a) The M-channel audio signals collected by the microphone array are subjected to analog-to-digital conversion by the audio collection subsystem, and the converted audio data is marked with a collection channel number and a time stamp, packed and sent;

(b) A server receives and analyzes the audio data packets sent by the audio collection subsystem, converts the audio data collected by the microphone array into different single sound source data, converts and compresses the converted single sound source data into an audio file format and saves it ;

(c) During the sound field reproduction process, the server reads the saved audio file, mixes the sound source data and converts the audio source data into the output data and control of the N-way speakers through adaptive matching according to the M-way sound source data and the characteristics of the reconstructed sound field The signal is sent to the audio reproduction subsystem;

(d) The audio reproduction subsystem synchronizes the output data of each speaker received from the server according to the control signal received from the server, and restores it into N-channel multi-channel analog audio signals, and sends them to the speaker array for playback.

In a preferred embodiment, when the audio data collected in the step (b) is converted into different single sound source data, a particle filter is used to separate noise and interference from the audio channel, that is, other sound source signals As a non-Gaussian noise interference, the problem of undistorted sound source extraction is transformed into a waveform tracking problem.

The present invention also provides an audio data packet structure, which is used to identify audio data attribute information during the audio collection process, including: a packet header part used to represent audio data attributes and a packet data part used to represent audio data, wherein the packet header Part includes packet start identification bit, channel identification bit, time stamp bit; packet data part includes audio data bit. A check digit may further be included.

As can be seen from the above, the system of the present invention converts the collected audio data into audio data of multiple single sound sources through the audio processing module in the server, thereby realizing the separation and collection of sound sources without distortion, and avoiding the single sound source The distortion of the sound field reduces the influence of the phase distortion of the sound field, and completely avoids the intermodulation interference between the sound sources; the audio data of the separated M-channel single sound source is converted according to the characteristics of the reconstructed sound field through the playback control module in the server The data outputted by N-channel speakers is converted into and the necessary control signals are provided, thereby accurately reproducing the original sound field in all directions. Through the audio frequency reproduction subsystem and the speaker array technology, the narrow phenomenon of the best listening area is avoided. The present invention clearly records the channel number from which the audio data comes from and the time of collection by adopting the audio data packet structure with the channel number and time stamp identification during audio collection, and provides space and time for all-round and accurate sound reproduction. important basis.

The technical problems to be solved by the present invention, the key points of the technical solutions and the beneficial effects will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

Description of drawings

Fig. 1 is a schematic structural diagram of the system described in the embodiment of the present invention;

Fig. 2 is the structural representation of the audio frequency acquisition sub-board in the audio frequency acquisition subsystem of Fig. 1;

Fig. 3 is the structural representation of the audio frequency acquisition motherboard in the audio frequency acquisition subsystem of Fig. 1;

Fig. 4 is the schematic diagram of the structure of the audio data packet after processing by the audio acquisition sub-board;

Fig. 5 is a schematic diagram of the structure of an audio data packet with a check code;

Fig. 6 is the flowchart of server audio data monitoring, audio data processing and storage in Fig. 1;

Fig. 7 is a schematic structural diagram of the audio data processing module in the server of Fig. 1;

Fig. 8 is a schematic structural diagram of the MINO intelligent matching module in the server of Fig. 1;

Fig. 9 is a structural schematic diagram of the audio sound reproduction sub-board in the audio sound reproduction subsystem of Fig. 1;

FIG. 10 is a schematic structural diagram of the audio reproduction motherboard in the audio reproduction subsystem of FIG. 1;

FIG. 11 is a schematic structural diagram of the playback control module in the server in FIG. 1 .

Detailed ways

Referring to FIG. 1 , it is a schematic structural diagram of the system described in the embodiment of the present invention.

The system described in the embodiment of the present invention includes: a microphone array 1 with multiple microphones, a speaker array 5 with multiple speakers, and also includes: an audio collection subsystem 2 , a server 3 , and an audio reproduction subsystem 4 .

in,

The audio acquisition subsystem 2 is used to perform analog-to-digital conversion on the various audio signals collected from the microphone array, and mark the converted audio data with the acquisition channel number and time stamp, package and send it;

Server 3 is used to receive and analyze the audio data packets sent by the audio acquisition subsystem during the sound field acquisition process, convert the audio data collected from each microphone array into different single sound source data, and convert the converted single sound source data Compress it into an audio file format, and add file description information, including song name, recording time, author, performer, venue, microphone positioning information, etc. to save; in the process of sound field reproduction, it is used to read the saved audio file, according to M The audio source data and the characteristics of the reconstructed sound field (including the number of speakers, the placement position, the occasion of the projection, etc.), the audio source data is mixed and converted into the output data and control signals of the N-channel speakers through intelligent matching, and sent to the audio frequency converter system.

The audio reproduction subsystem 4 is used to synchronize the output data of each speaker received from the server according to the control signal received from the server, including information such as the NM mapping matrix and the optimal structure of the speaker, and restore it to multi-channel analog audio The signal is sent to the speaker array for playback.

Through the audio collection subsystem, the data collected by multiple audio channels (such as 256 channels) can be transmitted to the Ethernet through the data interface, and then transmitted to the server through the Ethernet. The server processes the audio data of each microphone according to different sound field processing algorithms, including improved filter and equalizer processing, and finally converts the audio data collected by each microphone into audio data of different single sound sources, and converts these data into The audio file format can be played, and saved to the local SCSI (SmallComputer System Interface, small computer system interface) hard disk or other storage media. The remote client can access and play the audio files in the SCSI disk through the ISCSI (Internet Small Computer System Interface, Internet Small Computer System Interface) protocol or access the corresponding storage media through other communication protocols, and reproduce the original sound field realistically through the audio reproduction subsystem.

In the process of audio collection, transmission and storage, in order to avoid the distortion of a single sound source, reduce the influence of the phase distortion of the sound field, and completely avoid the intermodulation interference between sound sources, this program adopts an improved particle filter to combine noise and interference Detach from this audio channel. That is, other sound source signals are regarded as a non-Gaussian noise interference, and the problem of undistorted sound source extraction is transformed into a waveform tracking problem, thus a new method of sound source separation based on particle filters is proposed. The sound pickup range of this program includes both recording studios and on-site recordings, such as large-scale symphony concerts, sports competitions and party recordings.

By separating the main finite sound sources, the analysis and representation of other sound fields are made simpler and feasible. A multi-solution sound field system with M input and N output is constructed by means of finite sound source restoration technology and Huygens' principle, and a sound reproduction system that is superior and more flexible than a single sound field synthesis method and suitable for various actual environments is obtained. The audio reproduction subsystem and speaker array technology of this solution can better solve the existing problem of the narrow best listening area, and can be applied to the recording and broadcasting systems of major theaters, concert halls, stadiums, and squares , and can obtain a more realistic sound reproduction experience than the current multi-channel sound reproduction system.

The audio collection subsystem described in the embodiment of the present invention includes a plurality of audio collection sub-boards and an audio collection motherboard. Wherein, each audio collection sub-board includes one or more audio collection channels, an analog-to-digital converter group, and a field programmable gate array FPGA (or other logic processing device); the audio collection motherboard includes: collection sub-board control Interface (including serial interface, parallel interface), data acquisition interface of sub-board (including high-speed serial interface, parallel interface) and server communication interface (such as wired Ethernet interface, wireless ultra-wideband interface, wireless IP interface, etc.) (such as Figure 3).

Referring to Fig. 2, it is a schematic structural diagram of the audio collection sub-board in the audio collection subsystem according to the embodiment of the present invention, each audio collection sub-board collects audio signals from the microphone array through audio collection channels (CH0-CH7), and collects audio signals The audio signal collected by the channel is sent to the analog-to-digital converter group (4 analog-to-digital converter A/D groups are shown in the figure, and each A/D group is responsible for the analog-to-digital conversion of the audio signals collected by two audio collection channels ), the analog-to-digital converter group converts the audio information into audio data and sends it to the field programmable gate array, and the field programmable gate array marks the channel number and time on the audio data output by each analog-to-digital converter in the analog-to-digital converter group stamp, and send it to the data interface of the acquisition sub-board in the audio acquisition motherboard, and then send the audio data to the server through the server communication interface in the audio acquisition motherboard.

Here, the audio collection subsystem adopts a modular design, that is, the mode of an audio collection sub-board (as shown in Figure 2) and an audio collection motherboard (as shown in Figure 3), and each audio collection sub-board is used as a collection terminal. It can collect the data of 8 audio acquisition channels (when there are more than 8 audio acquisition channels, it can be extended). The Ethernet interface or other channels are read by the server and saved to the SCSI hard disk or other storage media. The server 3 sends control commands to the audio collection sub-board through the control interface of the audio collection sub-board on the audio collection motherboard, and obtains status information fed back by the audio collection sub-board from the control interface of the audio collection sub-board.

In Figure 2, the role of the A/D group is to convert multi-channel audio analog signals into digital audio data. Users can selectively sample audio data of one or several channels. The data of each channel must contain the channel number parameter. Since the single board supports up to 256 channels, an integer of one byte length can be used as the channel number parameter, that is, the channel number is from 0 to 255, and 0x00~0xff exceeds 256 channels. , it is distinguished by the server record file. In addition, the data collected by each channel must be time-stamped to ensure the correct order of the collected data and to facilitate selective playback by time. In order to ensure the correctness and reliability of the data in the sound reproduction system, error correction coding can also be performed on the audio data.

As shown in FIG. 4 , it is a schematic diagram of the structure of the audio data packet processed by the audio acquisition sub-board. After the field programmable gate array FPGA stamps the channel number and time stamp on the audio data after the analog-to-digital conversion, the audio data packet consists of two parts, the packet header and the packet data. The length of each data packet is fixed at 128 bytes, that is, the total length is 1024 bits. The structure of the data packet is shown in FIG. 4 . In network transmission, audio data packets will also be encapsulated into IP packets in the form of UDP (User Datagram Protocol) or TCP/IP (Transmission Control Protocol/Internet Protocol, Transmission Control Protocol/Internet Protocol). In the 128-byte audio data packet, there are 6 bytes in the header, including 1 byte for the packet start identification bit, 1 byte for the channel identification bit, 4 bytes for the timestamp bit, and 122 bytes for the audio data bit .

in,

The packet start identification bit adopts 1 byte:

The first byte (0x77) of the packet header, the packet start identifier, is used for synchronization to represent the beginning of a data packet.

The channel identification bit adopts 1 byte:

The data of each channel must contain a channel number parameter. Since a maximum of 256 channels are supported, an integer of one byte length can be used as the channel number parameter, that is, the channel number is from 0 to 255, and the decoder detects the packet header from 0x0 to 0xff. In the next byte, it is possible to know which of the 256 audio signals the received audio signal is.

The timestamp bits take 4 bytes:

The data collected by each channel must be time-stamped to ensure the correct order of the collected data and to facilitate selective playback by time. The time stamp here is a relative time stamp, which defines the affiliation relationship of each audio signal. Select the first channel of each group of audio signals as the main audio channel, and the other channels of audio signals as slave audio. Put a time stamp on each unit of the master audio, and according to the performance of the unit of the master audio at the same time, put the same time stamp on the unit corresponding to the slave audio, and the time stamp on each unit of the slave audio is Relative to the timestamp of the main audio unit.

In order to ensure the correctness and reliability of the data in the audio frequency reproduction subsystem, forward error correction can also be performed on the audio signal. In our scheme, T=8, truncated RS (Reed-Solomon, Reed-Solomon) encoding is adopted, and 16 check bytes are added to each data packet. At this time, the audio data bits only have 106 words Festival. The frame structure of the RS error protection packet is shown in FIG. 5 .

RS encoding is also applied to the synchronization byte of the data packet. Here, the truncated RS (144, 128) code is realized by adding 111 bytes and setting them as all 0s before inputting the information bytes at the input end of the RS (255, 239) encoder. After encoding, these zero bytes are then discarded. Similarly, the audio playback subsystem adds 111 bytes before inputting information bytes at the input terminal of the RS (255, 239) decoder, and sets them as all 0s. After decoding, these zero bytes are then discarded.

The server described in the embodiment of the present invention includes: a listening and collecting module, an audio data processing module, a storage module and a playback control module.

The monitoring acquisition module is used to monitor whether audio data arrives at the server, and collects when the audio data arrives;

The audio data processing module is used to convert the collected audio data into different single sound source data; here, the audio processor adopts a particle filter and an equalizer, as shown in Figure 7, and finally converts the audio signals collected by each microphone into different single source signal. Each audio processor adopts a modular plug-in design, so that different audio processors can be added and deleted as needed without affecting the entire system. At the same time, better audio processors can be continuously designed with the deepening of research.

The storage module is used for lossy or lossless conversion and compression of the converted single audio source data into an audio file format, adding relevant file information, including marking and distinguishing systems greater than 256 channels, and saving them;

The playback control module is used to read the saved audio files, mix the audio source data and convert them into output data and control signals of N-channel speakers through intelligent matching according to the M-channel audio source data and the characteristics of the reconstructed sound field, and send them to the audio-returning sub system. As shown in FIG. 8 , in this embodiment, the playback control module adopts the intelligent conversion module to realize MINO matching, and M and N dynamic weighted matching can be performed during the conversion process.

At the front end of the server, the audio signals collected by each microphone are input to the server through the Ethernet port (communication interface) on the server. Therefore, the server establishes port monitoring through the monitoring collection module, as shown in FIG. 6 . When data arrives at any Ethernet port, read the data packet with the TCP/IP protocol, and determine the channel attribute of the audio stream data packet according to the channel identification of the data packet header, and then transfer the audio stream data of different channels Store them in different hard disk files. The remote client can access and play these audio files on the SCSI disk through the ISCSI protocol or access the corresponding storage medium through other transmission protocols.

During the sound reproduction process, the server reads the saved audio files through the playback control module, mixes the sound source data according to the characteristics of the M-channel sound source data and the reconstructed sound field, and converts the audio source data into output data and control signals of the N-channel speakers through intelligent matching, and sends the to the audio reproduction subsystem.

The audio sound reproduction subsystem described in the embodiment of the present invention includes: a plurality of audio sound reproduction sub-boards and an audio sound reproduction motherboard.

Wherein, each audio sound reproduction sub-board includes one or more audio sound reproduction channels, a digital-to-analog converter group and a field programmable gate array.

The audio reproduction motherboard includes: a sound reproduction sub-board control interface, a sound reproduction sub-board data interface and a server communication interface, as shown in FIG. 9 .

The audio reproduction motherboard receives the output data and control signals from the speakers of the server through the server communication interface, and sends the output data of each speaker to the audio reproduction daughter board through the data interface of the reproduction sub-board, and at the same time passes the The control interface sends the control signal to the audio reproduction sub-board, as shown in Figure 10; please refer to Figure 9 at the same time, the field programmable gate array in the audio reproduction sub-board Synchronize the output data of each speaker received from the audio reproduction motherboard, and send to the digital-to-analog converter D/A group (4 dual-channel digital-to-analog converter D/A are shown in Figure 9) to convert into multi-channel The analog audio signal is sent to the speaker array for playback through audio reproduction channels (CH0-CHN, generally no more than 8 audio reproduction channels).

As shown in Figure 11, a speaker volume control submodule can also be added to the playback control module in the server to control the volume of the speaker array, such as adding a single speaker volume control unit, a group speaker volume control unit and all speaker volume control units, to perform individual speaker volume control, group speaker volume control, or all speaker volume control for the speaker array individually. At the same time, since the server and each loudspeaker are connected through a network, a loudspeaker array network monitoring sub-module can also be added, so that the server can monitor the connection status of each loudspeaker, so as to find abnormally connected loudspeakers in time. Here, the loudspeaker volume control submodule and the loudspeaker array network monitoring submodule can be implemented using existing mature technologies, such as in the audio power amplifier, the digital or analog volume control used is to control the volume of the loudspeaker, which can be Volume control for single, group and all speakers.

The monitoring of the loudspeaker includes the measurement of the input voltage of the loudspeaker and the measurement of the output of the sound field of the loudspeaker. The measurement results are sent to the server through the monitoring network to ensure normal operation. When playing remotely, you can choose not to detect the speaker.

A limited sound source multi-channel sound field system and audio data packet structure described in the present invention are not limited to the use listed in the description and the implementation, and it can be applied to various fields suitable for the present invention. Additional advantages and modifications will readily occur to those skilled in the art, so the invention is not to be limited to the specific details, representative examples, and the like without departing from the spirit and scope of the general concept defined by the claims and equivalents thereof. equipment and illustrated examples shown and described herein.

Claims (12)

1. A limited source multi-channel soundfield system comprising: a microphone array having a plurality of microphones, a speaker array having a plurality of speakers, and further comprising: an audio acquisition subsystem, a server, and an audio playback subsystem, wherein,

the microphone array is used for recording M paths of audio information and detecting sound field characteristics;

the audio acquisition subsystem is used for carrying out analog-to-digital conversion on each path of audio signals acquired from the microphone array, marking acquisition channel numbers and time stamps on the converted audio data, and packaging and sending the audio data;

the server is used for receiving and analyzing the audio data packets sent by the audio acquisition subsystem in the sound field acquisition process, converting the audio data acquired from each microphone array into different single sound source data, converting the converted single sound source data into an audio file format and storing the audio file format; in the sound field sound reproduction process, the system is used for reading the stored audio file, mixing sound source data according to M paths of sound source data and the characteristics of a reconstructed sound field, converting the mixed sound source data into output data and control signals of N paths of loudspeakers through intelligent matching, and sending the output data and the control signals to the audio sound reproduction subsystem;

the audio sound reproduction subsystem is used for synchronizing the output data of each loudspeaker received from the server according to the control signal received from the server, restoring the output data into a multi-channel analog audio signal and sending the multi-channel analog audio signal to the loudspeaker array for playing;

and the loudspeaker array is used for playing the N paths of audio signals and reconstructing a sound field.

2. The sound field system of claim 1, wherein said audio acquisition subsystem further comprises: the system comprises a plurality of audio acquisition daughter boards and an audio acquisition mother board; each audio acquisition sub-board comprises 1 or more audio acquisition channels, an analog-to-digital converter group and a logic processing device; the audio acquisition motherboard includes: collecting a daughter board data interface and a server communication interface; the system comprises a microphone array, an audio acquisition daughter board, a logic processing device, a server communication interface and a server, wherein each audio acquisition daughter board acquires audio signals from the microphone array through an audio acquisition channel and sends the audio signals acquired by the audio acquisition channel to an analog-to-digital converter group, the analog-to-digital converter group converts audio information into audio data and sends the audio data to the logic processing device, the audio data output by each analog-to-digital converter in the analog-to-digital converter group is marked with a channel number and a timestamp and sent to the acquisition daughter board data interface in the audio acquisition mother board, and the audio data are sent to the server through the.

3. The sound field system of claim 2, wherein said audio capture motherboard further comprises: collecting a daughter board control interface; the server sends a control command to the audio acquisition daughter board through an acquisition daughter board control interface in the audio acquisition mother board, and acquires state information fed back by the audio acquisition daughter board from the acquisition daughter board control interface.

4. The sound field system of claim 1, wherein the server further comprises: the monitoring acquisition module is used for monitoring whether audio data arrives at the server or not, and acquiring the audio data after the audio data arrives; the audio data processing module comprises a particle filter and an equalizer and is used for converting the collected audio data into different single sound source data; the storage module is used for converting the converted single sound source data into an audio file format, adding file description information and storing the file description information; and the playing control module is used for reading the stored audio file, mixing the sound source data according to the M paths of sound source data and the characteristics of the reconstructed sound field, converting the mixed sound source data into output data and control signals of the N paths of loudspeakers through intelligent matching, and sending the output data and the control signals to the audio sound reproduction subsystem.

5. The sound field system of claim 1, wherein said audio sound subsystem further comprises: a plurality of audio sound reproduction daughter boards and an audio sound reproduction mother board; each audio sound playing board comprises 1 or more audio sound playing channels, a digital-to-analog converter group and a logic processing device; the audio repayment mother board includes: the system comprises a sound reproduction board control interface, a sound reproduction board data interface and a server communication interface; the audio sound reproduction mother board receives output data and control signals of all loudspeakers from the server through the server communication interface, sends the output data of all the loudspeakers to the audio sound reproduction board through the sound reproduction board data interface, and sends the control signals to the audio sound reproduction board through the sound reproduction board control interface; the logic processing device in the audio sound reproduction motherboard synchronizes the output data of each loudspeaker received from the audio sound reproduction motherboard according to the control signal received from the audio sound reproduction motherboard, and sends the output data to the digital-to-analog converter group to be converted into a multi-channel analog audio signal, and the multi-channel analog audio signal is sent to the loudspeaker array to be played through the audio sound reproduction channel.

6. The sound field system of claim 4, wherein said playback control module further comprises a speaker volume control sub-module for controlling the volume of the speaker array.

7. The sound field system of claim 6, wherein said speaker volume control sub-module further comprises a single speaker volume control unit for single speaker volume control of the speaker array, a group speaker volume control unit for group speaker volume control, or a total speaker volume control unit for total speaker volume control.

8. The sound field system according to claim 4, wherein said playback control module further comprises a speaker array network monitoring sub-module for performing network monitoring on the speaker array.

9. A method for simulating a limited sound source multi-channel sound field comprises the following steps:

(a) the audio acquisition subsystem performs analog-to-digital conversion on the M audio signals acquired by the microphone array, marks acquisition channel numbers and timestamps on the converted audio data, and packs and sends the audio data;

(b) the server receives and analyzes the audio data packet sent by the audio acquisition subsystem, converts the audio data acquired by the microphone array into different single sound source data, converts the converted single sound source data into an audio file format and stores the audio file format;

(c) in the sound field sound reproduction process, the server reads the stored audio file, mixes sound source data according to M paths of sound source data and the characteristics of a reconstructed sound field, converts the mixed sound source data into output data and control signals of N paths of loudspeakers through self-adaptive matching, and sends the output data and the control signals to the audio sound reproduction subsystem;

(d) and the audio sound reproduction subsystem synchronizes the output data of each loudspeaker received from the server according to the control signal received from the server, restores the output data into N-channel multi-channel analog audio signals and sends the signals to the loudspeaker array for playing.

10. The method of claim 9, wherein step (b) converts the collected audio data into different single source data by using a particle filter to separate noise and interference from the audio channel, i.e., treating other source signals as a non-gaussian noise interference, and converting the distortion-free source extraction problem into a waveform tracking problem.

11. An audio data packet structure for identifying audio data attribute information during audio acquisition, comprising: the audio data processing device comprises a header part used for representing audio data attributes and a packet data part used for representing audio data, wherein the header part comprises a packet start identification bit, a channel identification bit and a time stamp bit; the packet data portion includes audio data bits.

12. The data packet structure according to claim 11, further comprising: and checking the bits.

Images