CN115966216A - Audio stream processing method and device - Google Patents

Abstract

The present application provides an audio stream processing method and device, wherein the audio stream processing method includes: obtaining the audio stream to be processed, and identifying the encoding format of the audio stream to be processed; when the encoding format of the audio stream to be processed is panoramic sound In the case of the encoding format, obtain the set audio loudness parameter; according to the set audio loudness parameter, transcode the audio stream to be processed into an audio stream in the stereo encoding format; determine the audio stream in the stereo encoding format according to the audio stream in the stereo encoding format Describes the target audio stream after transcoding the audio stream to be processed. Through the obtained set audio loudness parameters, the audio stream to be processed is transcoded into an audio stream in stereo encoding format, and then the target video stream is obtained, which can effectively solve the problem of low volume generated when panoramic sound is converted to stereo, and improve audio quality. At the same time, the audio processing process is simplified, the processing efficiency is improved, and it is compatible with the normal production of stereo audio.

Description

Audio stream processing method and device

technical field

The present application relates to the field of computer technology, in particular to an audio stream processing method. The present application also relates to an audio stream processing device, a computing device, and a computer-readable storage medium.

Background technique

With the rapid development of computer technology, signal processing technology has also advanced by leaps and bounds, among which audio processing technology is the most prominent. In the current on-demand streaming media, the most widely used audio is stereo audio and panoramic audio. In order to be compatible with the audio playback of all devices, when producing audio, generate audio in both panoramic and stereo formats.

In the prior art, usually the panoramic sound audio is directly transcoded to generate two audio streams in the panoramic sound encoding format and the stereo encoding format, which are provided to different hardware devices for playback. However, because the mixing method of Atmos audio is very different from that of stereo audio, after transcoding Atmos audio to stereo audio, the volume will become lower and the audio quality will be reduced, making stereo audio almost impossible to play. Therefore, there is an urgent need for an effective solution to solve the above problems.

Contents of the invention

In view of this, an embodiment of the present application provides an audio stream processing method. The present application also relates to an audio stream processing device, a computing device, and a computer-readable storage medium, so as to solve the technical defect of low audio quality after transcoding existing in the prior art.

According to the first aspect of the embodiments of the present application, an audio stream processing method is provided, including:

Obtain the audio stream to be processed, and identify the encoding format of the audio stream to be processed;

In the case where the encoding format of the audio stream to be processed is the panoramic sound encoding format, acquiring and setting the audio loudness parameter;

Transcoding the audio stream to be processed into an audio stream in a stereo encoding format according to the set audio loudness parameter;

Determine, according to the audio stream in the stereo encoding format, a target audio stream after transcoding the audio stream to be processed.

Optionally, transcoding the audio stream to be processed into an audio stream in a stereo encoding format according to the set audio loudness parameter includes:

Decoding the audio stream to be processed to obtain audio sample data;

According to the set audio loudness parameter and according to the set stereo encoding strategy, the audio sampling data is encoded to obtain an audio stream in a stereo encoding format.

Optionally, performing encoding processing on the audio sampling data according to the set audio loudness parameter according to the set stereo encoding strategy to obtain the audio stream in the stereo encoding format, including:

Encoding the audio sample data according to the set stereo encoding strategy, and adjusting the sound level of the audio sample data within the set duration according to the set audio loudness parameter, to obtain the audio stream in the stereo encoding format .

Optionally, the audio stream to be processed includes a plurality of audio data packets;

Decoding the audio stream to be processed to obtain audio sample data includes:

Decoding each of the audio data packets to obtain sub-audio sample data corresponding to each of the audio sample data;

According to the set audio loudness parameter, according to the set stereo encoding strategy, the audio sampling data is encoded to obtain the audio stream in stereo encoding format, including:

For each sub-audio sample data, according to the set audio loudness parameter, according to the set stereo encoding strategy, the sub-audio sample data is encoded to obtain a sub-audio stream in stereo encoding format;

Splicing the sub-audio streams in the stereo encoding format to obtain the audio stream in the stereo encoding format.

Optionally, the acquiring the audio stream to be processed includes:

Obtaining a video stream to be processed, wherein the video stream to be processed includes an audio stream to be processed and an image sequence to be processed;

After the acquisition of the multimedia stream to be processed, it also includes:

Transcoding the image sequence to be processed to obtain a target image sequence;

performing alignment processing on the target audio stream and the target image sequence to determine a target video stream.

Optionally, before acquiring and setting the audio loudness parameter, it also includes:

Acquiring audio track information of the audio stream to be processed;

If the audio track information is a single audio track, identify the encoding format of the audio stream to be processed.

Optionally, after acquiring the audio track information of the audio stream to be processed, the method further includes:

In the case where the audio track information is a dual audio track, respectively identify the encoding format of the audio stream to be processed corresponding to each audio track;

Perform file format transcoding on the audio stream to be processed whose encoding format is stereo encoding format to obtain a target audio stream in stereo encoding format, and/or perform file format conversion on the audio stream to be processed whose encoding format is panoramic sound encoding format Format transcoding to obtain the target audio stream in the Atmos encoding format.

According to a second aspect of the embodiments of the present application, an audio stream processing device is provided, including:

The first identification module is configured to obtain the audio stream to be processed, and identify the encoding format of the audio stream to be processed;

The acquisition module is configured to acquire and set the audio loudness parameter when the encoding format of the audio stream to be processed is the panoramic sound encoding format;

The first transcoding module is configured to transcode the audio stream to be processed into an audio stream in a stereo encoding format according to the set audio loudness parameter;

The determining module is configured to determine a target audio stream after transcoding the audio stream to be processed according to the audio stream in the stereo encoding format.

According to a third aspect of the embodiments of the present application, there is provided a computing device, including a memory, a processor, and computer instructions stored in the memory and operable on the processor. When the processor executes the computer instructions, the computer instructions are implemented. The steps of the audio stream processing method are described.

According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, which stores computer instructions, and implements the steps of the audio stream processing method when the computer instructions are executed by a processor.

The audio stream processing method provided in this application obtains the audio stream to be processed, and identifies the encoding format of the audio stream to be processed; when the encoding format of the audio stream to be processed is the panoramic sound encoding format, obtains the set audio Loudness parameter; according to the set audio loudness parameter, transcode the audio stream to be processed into an audio stream in stereo encoding format; determine the target audio after transcoding the audio stream to be processed according to the audio stream in stereo encoding format flow. By obtaining the set audio loudness parameters, the audio stream to be processed is transcoded into an audio stream in stereo encoding format, and then the target video stream is obtained, which can effectively solve the problem of low volume generated when panoramic sound is converted to stereo, and improve audio quality. At the same time, it simplifies the audio processing process, improves the processing efficiency, and is compatible with the normal production of stereo audio.

Description of drawings

Fig. 1 is the processing flowchart of a kind of audio stream processing method that prior art provides;

FIG. 2 is a schematic structural diagram of an audio stream processing system provided according to an embodiment of the present application;

Fig. 3 is a flowchart of an audio stream processing method provided by an embodiment of the present application;

FIG. 4A is a processing flowchart of an audio stream processing method provided by an embodiment of the present application;

FIG. 4B is a processing flowchart of an audio stream processing method applied to a movie scene provided by an embodiment of the present application;

Fig. 5 is a schematic structural diagram of an audio stream processing device provided by an embodiment of the present application;

Fig. 6 is a structural block diagram of a computing device provided by an embodiment of the present application.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the application. However, the present application can be implemented in many other ways different from those described here, and those skilled in the art can make similar promotions without violating the connotation of the present application. Therefore, the present application is not limited by the specific implementation disclosed below.

Terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only, and are not intended to limit the one or more embodiments of the present application. As used in one or more embodiments of this application and the appended claims, the singular forms "a", "the", and "the" are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and/or" used in one or more embodiments of the present application refers to and includes any and all possible combinations of one or more associated listed items.

It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second, and similarly, second may also be referred to as first, without departing from the scope of one or more embodiments of the present application. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination."

First, terms and terms involved in one or more embodiments of the present application are explained.

Stereo: It uses two or more independent sound channels to appear on a pair of speakers (commonly known as speakers) configured in a symmetrical manner. The sound produced in this way can still be natural and pleasant in different directions. Stereo sound with multiple channels is also called stereo surround sound, such as common 5.1 surround sound and 7.1 surround sound.

Atmos: Also known as Dolby Atmos, it is a three-dimensional surround sound technology. It expands on the basis of stereo surround sound and adds the surround sound effect of the sky channel. It can present the content of 64 independent speakers, or simultaneously Send up to 128 channels or objects, more detailed than 7.1 surround channels. Dolby Atmos is an implementation of spatial audio.

LUFS (LoudnessUnitsRelativetoFullScale) stands for Loudness Units Relative to Full Scale or Full Scale Loudness Units (ie, the maximum level the system can handle). This is a standardized measure of the loudness of sound that takes into account both human perception and electrical signal strength.

With the rapid development of computer technology, signal processing technology has also advanced by leaps and bounds, among which audio processing technology is the most prominent. In the current on-demand streaming media, the most widely used audio is stereo audio and panoramic audio. Traditional stereo can display sound on a horizontal plane, and sound positioning has two dimensions, front and back, left and right, which can be called two-dimensional (2D, 2-Dimension) audio. When an audio has two dimensions of front and back, left and right, as well as up and down dimensions, it can be called three-dimensional (3D, 3-Dimension) audio, that is, spatial audio.

In the current on-demand streaming media, the most widely used audio is the audio in stereo encoding format. Because of its simplicity and high compatibility, many Internet companies have adopted this solution. With the development of hardware technology, more and more user devices support playing spatial audio. Therefore, in order to be compatible with the audio playback of all devices, when producing audio, audio in both panoramic and stereo formats is generated.

In the prior art, usually the panoramic sound audio is directly transcoded to generate two audio streams in the panoramic sound encoding format and the stereo encoding format, which are provided to different hardware devices for playback. Referring to FIG. 1, FIG. 1 shows a schematic flowchart of an audio stream processing method provided by the prior art: video or audio, that is, a generator of media resources, generates panoramic sound media resources, and then directly processes the panoramic sound media resources Transcoding to obtain the target media resource of the panoramic sound and the target media resource of the stereo sound, so that the listener can use the playback device to play.

However, due to the mixing and method of panoramic sound audio, it is very different from stereo audio. Ordinary stereo generally only has two fixed channels, and the sound only has the sound sources on the left and right sides during playback. The sound source of panoramic sound includes not only the left and right sides, but also the front, rear, up, down, and all directions, which is a kind of 360-degree surround sound. As a result, after transcoding Atmos audio to stereo audio, the volume becomes lower and the audio quality degrades, making stereo audio almost unplayable.

Therefore, this specification provides an audio stream processing method, which acquires the audio stream to be processed, and identifies the encoding format of the audio stream to be processed; when the encoding format of the audio stream to be processed is the panoramic sound encoding format, Obtain the set audio loudness parameter; according to the set audio loudness parameter, transcode the audio stream to be processed into an audio stream in stereo encoding format; determine the transcoding of the audio stream to be processed according to the audio stream in stereo encoding format After the target audio stream. By obtaining the set audio loudness parameters, the audio stream to be processed is transcoded into an audio stream in stereo encoding format, and then the target video stream is obtained, which can effectively solve the problem of low volume generated when panoramic sound is converted to stereo, and improve audio quality. At the same time, it simplifies the audio processing process, improves the processing efficiency, and is compatible with the normal production of stereo audio.

In the present application, an audio stream processing method is provided, and the present application also relates to an audio stream processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments one by one.

The execution subject of the audio stream processing method provided in the embodiment of the present application may be a terminal, a server, or a terminal and a server jointly assisting, which is not limited in the embodiment of the present application. Moreover, the terminal can be any electronic product capable of man-machine interaction with the user, such as a PC (Personal Computer, personal computer), a mobile phone, a Pocket PC (Pocket PC), a tablet computer, and the like. The server may be one server, or a server cluster composed of multiple servers, or a cloud computing service center, which is not limited in this embodiment of the present application.

Taking the joint assistance of the terminal and the server as an example, see FIG. 2, which is a schematic structural diagram of an audio stream processing system provided according to an embodiment of the present application:

The terminal uploads the audio stream to be processed to the server.

The server identifies the encoding format of the audio stream to be processed; when the encoding format of the audio stream to be processed is a panoramic sound encoding format, obtains a set audio loudness parameter; according to the set audio loudness parameter, the described The audio stream to be processed is transcoded into an audio stream in a stereo coding format; and a target audio stream after the transcoding of the audio stream to be processed is determined according to the audio stream in a stereo coding format.

Correspondingly, the server may also send the target audio stream to the playback device corresponding to the listener, so that the listener can play it.

Applying the scheme of this embodiment, by obtaining the set audio loudness parameters, the audio stream to be processed is transcoded into an audio stream in stereo encoding format, and then the target video stream is obtained, which can effectively solve the problem of low volume generated when panoramic sound is converted to stereo The problem is that while improving the audio quality, it simplifies the audio processing process, improves processing efficiency, and is compatible with the normal production of stereo audio.

Fig. 3 shows a flow chart of an audio stream processing method provided according to an embodiment of the present application, which specifically includes the following steps:

Step 302: Obtain the audio stream to be processed, and identify the encoding format of the audio stream to be processed.

Specifically, audio refers to a file storing sound content. Audio Stream can control the output quality of "Stream" sync type audio. The audio stream to be processed refers to the audio stream that needs to be processed or transcoded.

In practical applications, there are many ways to obtain the audio stream to be processed. For example, a user may send an acquisition instruction for the audio stream to be processed to the execution subject, or send an audio stream processing instruction. Correspondingly, the execution subject receives the acquisition instruction After the instruction, start to obtain the audio stream to be processed; it can also be that the execution subject automatically obtains the audio stream to be processed every preset time period. For example, after the preset time period, the server with audio stream processing function automatically obtains the designated access area The pending audio stream within the . This manual does not make any limitation on the way to obtain the audio stream to be processed.

After the audio stream to be processed is obtained, the encoding format of the audio stream to be processed can be identified according to the data attribute information of the audio stream, or the audio stream to be processed can be detected by the media file information viewing tool to identify the encoding format.

For example, you can call the mediainfo command to identify the encoding format of the audio stream to be processed. Among them, MediaInfo is a very practical video parameter detection tool. In addition to encoding, analyzing and querying video, it can also detect the encoding and information of audio files.

Step 304: When the encoding format of the audio stream to be processed is the panoramic sound encoding format, acquire and set audio loudness parameters.

Specifically, encoding is to use a predetermined method to encode text, numbers or other objects into numbers, or to convert information and data into prescribed electrical pulse signals. The panoramic sound coding format refers to a coding format corresponding to panoramic sound, such as the E-AC-3JOC format. Setting the audio loudness parameter refers to the parameter for adjusting the loudness of the audio data, such as LUFS (loudness unit of relative full scale or full-scale loudness unit), decibel, etc.

In practical applications, if it is identified that the encoding format of the audio stream to be processed is the panoramic sound encoding format, the set audio loudness parameter can be obtained, and the set audio loudness parameter can be set in advance, or it can be obtained after the audio stream to be processed is identified. After the coding format is the panoramic sound coding format, the panoramic sound coding format is displayed on the display, and the audio loudness parameter is set by the user in real time, thereby obtaining the set audio loudness parameter.

Step 306: Transcode the audio stream to be processed into an audio stream in stereo encoding format according to the set audio loudness parameter.

Specifically, the stereo encoding format refers to an encoding format corresponding to stereo, such as the AACLC format.

In practical applications, the audio stream to be processed may be transcoded into an audio stream in a stereo encoding format based on a set audio loudness parameter according to a set transcoding strategy.

In one or more optional embodiments of this specification, based on the set audio loudness parameter, the audio stream to be processed may be decoded and encoded, so as to obtain an audio stream in a stereo encoding format. That is to say, according to the set audio loudness parameter, the audio stream to be processed is transcoded into an audio stream in a stereo encoding format, and the specific implementation process can be as follows:

Decoding the audio stream to be processed to obtain audio sample data;

According to the set audio loudness parameter and according to the set stereo encoding strategy, the audio sampling data is encoded to obtain an audio stream in a stereo encoding format.

Specifically, the audio sampling data is audio data in PCM (Pulse Code Modulation, Pulse Code Modulation) format, which is the format closest to the original audio, and is also called naked audio. Setting a stereo coding strategy refers to a coding method, scheme, etc. for converting panoramic sound into stereo sound.

In practical applications, you can first use the decoder to decode the audio stream to be processed to obtain the original audio data, that is, audio sampling data, then set the audio loudness parameters, follow the set stereo encoding strategy, and use the encoder to process the audio The sampling data is encoded to obtain an audio stream in a stereo encoding format, and the difference between the loudness of the audio stream in the stereo encoding format and the set audio loudness parameter is within a set range. In this way, the audio transcoding efficiency can be improved, and the quality of the audio stream in the stereo encoding format can be improved, thereby improving user satisfaction.

In one or more optional embodiments of this specification, while encoding, the loudness of the audio stream may be adjusted based on the set audio loudness parameter. That is to say, according to the set audio loudness parameter, according to the set stereo encoding strategy, the audio sampling data is encoded to obtain the audio stream in the stereo encoding format. The specific implementation process can be as follows:

Encoding the audio sample data according to the set stereo encoding strategy, and adjusting the sound level of the audio sample data within the set duration according to the set audio loudness parameter, to obtain the audio stream in the stereo encoding format .

Specifically, the sound level within the set duration refers to an average level of the sound within a period of time.

In practical applications, the encoder is set by setting the stereo encoding strategy, and then the set encoder encodes the audio sample data, and at the same time, the set audio loudness parameter is input into the encoder, so that the encoder adjusts the audio sample data in the set The sound level within the duration, that is, the loudness level, and then obtain the audio stream in the stereo encoding format. In this way, the accuracy and quality of the transcoded audio stream in the stereo encoding format can be improved, thereby improving the quality of the target audio stream.

In one or more optional embodiments of this specification, the audio stream to be processed includes multiple audio data packets; at this time, each audio data packet is decoded and encoded respectively. That is to say, the audio stream to be processed is decoded to obtain audio sampling data. The specific implementation process can be as follows:

Decoding each of the audio data packets to obtain sub-audio sample data corresponding to each of the audio sample data;

Correspondingly, according to the set audio loudness parameter, according to the set stereo encoding strategy, the audio sampling data is encoded to obtain an audio stream in a stereo encoding format, including:

For each sub-audio sample data, according to the set audio loudness parameter, according to the set stereo encoding strategy, the sub-audio sample data is encoded to obtain a sub-audio stream in stereo encoding format;

Splicing the sub-audio streams in the stereo encoding format to obtain the audio stream in the stereo encoding format.

Specifically, an audio data packet, that is, an audiopacket, refers to a component unit that constitutes an audio stream, that is, each audio stream is composed of N audiopackets, and encoding and decoding can be processed for the audiopackets.

In practical applications, for each audio data packet in the audio stream to be processed, a decoder can be used to decode the current audio data packet to obtain the original sub-audio data, that is, the sub-audio sampling data, and then set the audio Loudness parameter, according to the set stereo encoding strategy, and use the encoder to encode the sub-audio sample data to obtain the sub-audio stream in the stereo encoding format, and the difference between the loudness of the sub-audio stream in the stereo encoding format and the set audio loudness parameter The value is within the set range. After traversing all the audio data packets, the sub-audio streams in the stereo encoding format corresponding to the audio data packets are spliced to obtain an audio stream in the stereo encoding format. In this way, the audio transcoding efficiency can be further improved, and the quality of the audio stream in the stereo coding format can be improved, thereby improving user satisfaction.

It should be noted that the audio information in each audiopacket is different, such as loudness and bit rate. When transcoding, a single audiopacket will be processed, and fixed encoding parameters will be specified, that is, the loudness parameter will be set. The actual data of the final encoded audiopacket will also be different, and will be processed separately according to the stream information of the audio packet itself.

Optionally, for each sub-audio sample data, according to the set audio loudness parameter, according to the set stereo encoding strategy, the sub-audio sample data is encoded to obtain a sub-audio stream in a stereo encoding format; Splicing the sub-audio streams in the stereo encoding format to obtain the audio stream in the stereo encoding format can also be: for each sub-audio sample data, encode the sub-audio sample data according to the set stereo encoding strategy, and according to the set The audio loudness parameter adjusts the sound level of the sub-audio sampling data within the set duration to obtain a sub-audio stream in a stereo encoding format; splicing the sub-audio streams in each stereo encoding format to obtain an audio stream in a stereo encoding format.

In a practicable embodiment of the present specification, on the basis of obtaining the audio stream to be processed, the encoding format of the audio stream to be processed can be directly identified, thereby simplifying the processing flow and improving processing efficiency.

In another realizable embodiment of this specification, on the basis of obtaining the audio stream to be processed, the audio track information of the audio stream to be processed can also be obtained first, and then based on the audio track information, the encoding of the audio stream to be processed can be identified Format. That is, before obtaining and setting the audio loudness parameter, it also includes:

Acquiring audio track information of the audio stream to be processed;

If the audio track information is a single audio track, identify the encoding format of the audio stream to be processed.

Specifically, an audio track refers to a music track, and each or each audio track corresponds to an audio stream. A single audio track means that the audio stream to be processed corresponds to one audio stream.

In practical applications, the audio track information of the audio stream to be processed can be identified according to the data attribute information of the audio stream, and the audio track information of the audio stream to be processed can also be detected by a media file information viewing tool.

For example, the mediainfo command can be called to identify the audio track information of the audio stream to be processed.

In this way, by first identifying the audio track information of the audio stream to be processed, and then identifying the encoding format of the audio stream to be processed in the case of a single audio track, it is possible to avoid two different encodings in the case of a dual audio track. format of the pending audio stream, resulting in recognition errors, thereby improving recognition efficiency.

Optionally, the audio track information may also be two-way audio tracks. In this case, it is necessary to perform transcoding processing on the pending audio streams corresponding to each track. That is, after the acquisition of the audio track information of the audio stream to be processed, it also includes:

In the case where the audio track information is a dual audio track, respectively identify the encoding format of the audio stream to be processed corresponding to each audio track;

Perform file format transcoding on the audio stream to be processed whose encoding format is stereo encoding format to obtain a target audio stream in stereo encoding format, and/or perform file format conversion on the audio stream to be processed whose encoding format is panoramic sound encoding format Format transcoding to obtain the target audio stream in the Atmos encoding format.

Specifically, a single audio track means that the audio stream to be processed corresponds to two audio streams, that is, there are two audio streams to be processed.

In practical applications, when the audio track information is a dual audio track, you can call the media file information viewing tool to identify the encoding format of the audio stream corresponding to each audio track. For the audio stream whose encoding format is stereo encoding format Process the audio stream and transcode it to obtain the target audio stream in the stereo encoding format, for example, convert the stereo audio stream in the acc file format (the audio stream to be processed in the stereo encoding format) into a stereo audio stream in the mp3 file format; for The encoding format is the audio stream to be processed in the panoramic sound encoding format, and it is transcoded to obtain the target audio stream in the panoramic sound encoding format.

Since it is a dual audio track, it is only necessary to transcode the audio stream to be processed in the stereo encoding format to the target audio stream in the stereo encoding format, and to transcode the audio stream to be processed in the atmos encoding format to the target audio stream in the atmos encoding format. For the conversion between the stereo encoding format and the panoramic encoding format, the change in loudness before and after transcoding is minimal, so there is no need to obtain and set the audio loudness parameter. In this way, not only can the transcoding quality be guaranteed, but also the processing flow can be simplified and the processing efficiency can be improved.

In addition, after identifying the coding format of the audio stream to be processed, it also includes: transcoding the audio stream to be processed into a panoramic sound coding format when the coding format of the audio stream to be processed is a panoramic sound coding format The target audio stream for .

It should be noted that whether the audio stream to be processed in the stereo coding format is transcoded into the target audio stream in the stereo coding format, or the audio stream in the panoramic sound coding format is transcoded into the target audio stream in the panoramic sound coding format, the process is similar. That is, the audio stream to be processed in the specified encoding format is decoded to obtain audio sample data; according to the target file format, the audio sample data is encoded to obtain the target audio stream in the specified encoding format. Wherein, the specified encoding format is a stereo encoding format or panoramic sound encoding format, and the target file format is a common file format, so that various playback devices can recognize the target audio stream for playback.

Step 308: According to the audio stream in the stereo encoding format, determine a target audio stream after transcoding the audio stream to be processed.

In practical applications, after the audio stream in the stereo encoding format is obtained, the audio stream in the stereo encoding format may be converted to a file format to obtain a target audio stream in the stereo encoding format.

In one or more optional embodiments of this specification, the video stream to be processed may be acquired, and the audio stream to be processed in the video stream to be processed may be processed. That is, the acquisition of the audio stream to be processed includes:

Obtaining a video stream to be processed, wherein the video stream to be processed includes an audio stream to be processed and an image sequence to be processed;

Correspondingly, after the acquisition of the multimedia stream to be processed, it also includes:

Transcoding the image sequence to be processed to obtain a target image sequence;

performing alignment processing on the target audio stream and the target image sequence to determine a target video stream.

In practical applications, the object to be processed is a video stream to be processed, and the video stream to be processed includes an audio stream to be processed and an image sequence to be processed, wherein the image sequence to be processed refers to multiple images arranged in a certain order. Then, identify the encoding format of the audio stream to be processed; if the encoding format of the audio stream to be processed is the panorama encoding format, obtain the set audio loudness parameter; according to the set audio loudness parameter, the to-be-processed The audio stream is transcoded into an audio stream in a stereo encoding format; and a target audio stream after the transcoding of the audio stream to be processed is determined according to the audio stream in a stereo encoding format. The image sequence to be processed is also transcoded to obtain a target image sequence in a target image format, wherein the target image format is an approved image format, that is, an image format that can be displayed by a display device. Further, the target audio stream and the target image sequence are aligned according to the corresponding time stamps to obtain the target video stream.

Referring to FIG. 4A, FIG. 4A shows a processing flowchart of an audio stream processing method provided by an embodiment of the present application:

In the video production stage, detect the original audio meta information uploaded by users, that is, the audio stream to be processed: use mediainfo to pre-detect the audio stream (meta information detection), and obtain the encoding format and corresponding audio track information of the audio stream to be processed. When the encoding format is E-AC-3JOC (panoramic sound coding format), it means that the audio stream to be processed is panoramic sound; when the coding format is AACLC (stereo sound coding format), it means that the audio stream to be processed is stereo. When multiple audio tracks are used, the encoding formats of different audio streams to be processed can be obtained at the same time.

If it is detected that the audio stream to be processed is a single-channel audio track and the encoding format is E-AC-3JOC, that is, a single-channel Dolby Atmos Process the transcoding of the audio stream into an AACLC target audio stream (atmos -> stereo) with an audio loudness parameter of -18LUFS and a common file format. That is, when the original audio stream is a single-track Dolby Atmos, when transcoding the audio, first adjust the loudness parameter to restore the sound volume to the normal one. The actual loudness parameter is a fixed loudness value -18LUFS, which affects the dynamic range Larger audio is more friendly, the effect in actual application is that the bass is not lost, and the treble is clearer. In addition, the audio stream to be processed of E-AC-3JOC needs to be transcoded in file format to obtain the E-AC-3JOC target audio stream in a common file format (atmosound->atmosound).

If it is detected that the audio stream to be processed is a dual audio track, and the encoding format of the first audio track is E-AC-3JOC, and the encoding format of the second audio track is AACLC, that is, one Dolby panoramic sound track + one Stereo soundtrack. When transcoding, the appropriate audio track is automatically selected for transcoding based on the expected result. Use the transcoding of the stereo track in the original audio to generate the corresponding stereo audio, and use the transcoding of the atmos track in the original audio to generate the corresponding atmos audio. Just transcode the file format of the audio stream to be processed corresponding to the first audio track to obtain the E-AC-3JOC target audio stream (atmos->atmosound) in the general file format; convert the audio stream corresponding to the second audio track to The audio stream to be processed can be transcoded in the file format to obtain the AACLC target audio stream in the common file format (stereo -> stereo).

The audio stream processing method provided in this application obtains the audio stream to be processed, and identifies the encoding format of the audio stream to be processed; when the encoding format of the audio stream to be processed is the panoramic sound encoding format, obtains the set audio Loudness parameter; according to the set audio loudness parameter, transcode the audio stream to be processed into an audio stream in stereo encoding format; determine the target audio after transcoding the audio stream to be processed according to the audio stream in stereo encoding format flow. By obtaining the set audio loudness parameters, the audio stream to be processed is transcoded into an audio stream in stereo encoding format, and then the target video stream is obtained, which can effectively solve the problem of low volume generated when panoramic sound is converted to stereo, and improve audio quality. At the same time, it simplifies the audio processing process, improves the processing efficiency, and is compatible with the normal production of stereo audio.

In the following, the audio stream processing method will be further described by taking the application of the audio stream processing method provided in this application in a film scene as an example in conjunction with FIG. 4B . Wherein, FIG. 4B shows a processing flowchart of an audio stream processing method applied to a movie scene provided by an embodiment of the present application, which specifically includes the following steps:

Step 402: Obtain a video stream to be processed, wherein the video stream to be processed includes an audio stream to be processed and an image sequence to be processed.

Step 404: Transcoding the image sequence to be processed to obtain a target image sequence.

Step 406: Obtain audio track information of the audio stream to be processed.

Step 408: If the audio track information is a single audio track, identify the encoding format of the audio stream to be processed.

Step 410: When the encoding format of the audio stream to be processed is the panoramic sound encoding format, acquire and set audio loudness parameters.

Step 412: Decode the audio stream to be processed to obtain audio sample data.

Optionally, the audio stream to be processed includes a plurality of audio data packets;

Decode the audio stream to be processed to obtain audio sample data, including:

Each audio data packet is decoded to obtain sub-audio sample data corresponding to each audio sample data.

Step 414: Encode the audio sample data according to the set stereo encoding strategy, and adjust the sound level of the audio sample data within the set duration according to the set audio loudness parameter to obtain an audio stream in stereo encoding format.

Optionally, the audio stream to be processed includes a plurality of audio data packets;

According to the set stereo encoding strategy, the audio sample data is encoded, and according to the set audio loudness parameter, the sound level of the audio sample data is adjusted within the set duration, and the audio stream in the stereo encoding format is obtained, including:

For each sub-audio sample data, encode the sub-audio sample data according to the set stereo encoding strategy, and adjust the sound level of the sub-audio sample data within the set duration according to the set audio loudness parameters, and obtain the sub-audio in the stereo encoding format flow;

The sub-audio streams in the stereo encoding format are spliced to obtain an audio stream in the stereo encoding format.

Step 416: According to the audio stream in the stereo encoding format, determine the target audio stream after transcoding the audio stream to be processed.

Step 418: In the case that the audio track information is a dual audio track, respectively identify the encoding format of the audio stream to be processed corresponding to each audio track.

Step 420: Perform file format transcoding on the audio stream to be processed whose encoding format is the stereo encoding format, to obtain a target audio stream in the stereo encoding format.

Step 422: Perform file format transcoding on the audio stream to be processed whose encoding format is the panoramic sound encoding format, to obtain a target audio stream in the panoramic sound encoding format.

Step 424: Align the target audio stream and the target image sequence to determine the target video stream.

The audio stream processing method provided by this application transcodes the audio stream to be processed into an audio stream in stereo encoding format by obtaining the set audio loudness parameters, and then obtains the target video stream, which can effectively solve the volume generated when panoramic sound is converted to stereo The problem of too low, while improving the audio quality, simplifies the audio processing process, improves the processing efficiency, and is compatible with the normal production of stereo audio.

Corresponding to the above method embodiment, the present application also provides an embodiment of an audio stream processing device, and FIG. 5 shows a schematic structural diagram of an audio stream processing device provided by an embodiment of the present application. As shown in Figure 5, the device includes:

The first identifying module 502 is configured to acquire the audio stream to be processed, and identify the encoding format of the audio stream to be processed;

The obtaining module 504 is configured to obtain a set audio loudness parameter when the encoding format of the audio stream to be processed is the panoramic sound encoding format;

The first transcoding module 506 is configured to transcode the audio stream to be processed into an audio stream in a stereo coding format according to the set audio loudness parameter;

The determination module 508 is configured to determine a target audio stream after transcoding the audio stream to be processed according to the audio stream in the stereo coding format.

Optionally, the first transcoding module 506 is further configured to:

Decoding the audio stream to be processed to obtain audio sample data;

According to the set audio loudness parameter and according to the set stereo encoding strategy, the audio sampling data is encoded to obtain an audio stream in a stereo encoding format.

Optionally, the first transcoding module 506 is further configured to:

Encoding the audio sample data according to the set stereo encoding strategy, and adjusting the sound level of the audio sample data within the set duration according to the set audio loudness parameter, to obtain the audio stream in the stereo encoding format .

Optionally, the audio stream to be processed includes a plurality of audio data packets;

The first transcoding module 506 is further configured to:

Decoding each of the audio data packets to obtain sub-audio sample data corresponding to each of the audio sample data;

For each sub-audio sample data, according to the set audio loudness parameter, according to the set stereo encoding strategy, the sub-audio sample data is encoded to obtain a sub-audio stream in stereo encoding format;

Splicing the sub-audio streams in the stereo encoding format to obtain the audio stream in the stereo encoding format.

Optionally, the first identification module 502 is further configured to:

Obtaining a video stream to be processed, wherein the video stream to be processed includes an audio stream to be processed and an image sequence to be processed;

The device also includes a second transcoding module configured to:

Transcoding the image sequence to be processed to obtain a target image sequence;

performing alignment processing on the target audio stream and the target image sequence to determine a target video stream.

Optionally, the device further includes a second identification module configured to:

Acquiring audio track information of the audio stream to be processed;

If the audio track information is a single audio track, identify the encoding format of the audio stream to be processed.

Optionally, the device further includes a third identification module configured to:

In the case where the audio track information is a dual audio track, respectively identify the encoding format of the audio stream to be processed corresponding to each audio track;

Perform file format transcoding on the audio stream to be processed whose encoding format is stereo encoding format to obtain a target audio stream in stereo encoding format, and/or perform file format conversion on the audio stream to be processed whose encoding format is panoramic sound encoding format Format transcoding to obtain the target audio stream in the Atmos encoding format.

The audio stream processing device provided by this application transcodes the audio stream to be processed into an audio stream in a stereo encoding format by obtaining the set audio loudness parameters, and then obtains a target video stream, which can effectively solve the volume generated when panoramic sound is converted to stereo The problem of too low, while improving the audio quality, simplifies the audio processing process, improves the processing efficiency, and is compatible with the normal production of stereo audio.

The foregoing is a schematic solution of an audio stream processing apparatus in this embodiment. It should be noted that the technical solution of the audio stream processing device and the technical solution of the above-mentioned audio stream processing method belong to the same idea, and details of the technical solution of the audio stream processing device that are not described in detail can be found in the above-mentioned audio stream processing method. Description of the technical solution.

Fig. 6 shows a structural block diagram of a computing device provided according to an embodiment of the present application. Components of the computing device 600 include, but are not limited to, memory 610 and processor 620 . The processor 620 is connected to the memory 610 through the bus 630, and the database 650 is used for saving data.

Computing device 600 also includes an access device 640 that enables computing device 600 to communicate via one or more networks 660 . Examples of these networks include a public switched telephone network (PSTN, PublicSwitchedTelephoneNetwork), a local area network (LAN, LocalAreaNetwork), a wide area network (WAN, WideAreaNetwork), a personal area network (PAN, PersonalAreaNetwork) or a combination of communication networks such as the Internet. The access device 640 may include one or more of wired or wireless network interfaces of any type (for example, a network interface card (NIC, NetworkInterfaceController)), such as an IEEE802.11 wireless local area network (WLAN, WirelessLocalAreaNetwork) wireless interface, a global microwave Internet access (Wi-MAX, Worldwide Interoperability for Microwave Access) interface, Ethernet interface, target serial bus (USB, UniversalSerialBus) interface, cellular network interface, Bluetooth interface, near field communication (NFC, NearFieldCommunication) interface, etc.

In an embodiment of the present application, the above-mentioned components of the computing device 600 and other components not shown in FIG. 6 may also be connected to each other, for example, through a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 6 is only for the purpose of illustration, rather than limiting the scope of the application. Those skilled in the art can add or replace other components as needed.

Computing device 600 may be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile telephones (e.g., smartphones), ), wearable computing devices (eg, smart watches, smart glasses, etc.), or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. Computing device 600 may also be a mobile or stationary server.

Wherein, the processor 620 implements the steps of the audio stream processing method when executing the computer instructions.

The foregoing is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of the computing device and the above-mentioned technical solution of the audio stream processing method belong to the same idea, and details of the technical solution of the computing device that are not described in detail can be found in the description of the technical solution of the above-mentioned audio stream processing method .

An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the aforementioned audio stream processing method are implemented.

The foregoing is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned audio stream processing method belong to the same idea, and details of the technical solution of the storage medium that are not described in detail can be found in the description of the above-mentioned technical solution of the audio stream processing method .

The foregoing describes specific embodiments of the present application. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

The computer instructions include computer program code, which may be in source code form, object code form, executable file or some intermediate form, and the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-OnlyMemory), Random access memory (RAM, RandomAccessMemory), electric carrier signal, telecommunication signal and software distribution medium, etc.

It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Depending on the application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

The preferred embodiments of the present application disclosed above are only used to help clarify the present application. Alternative embodiments are not exhaustive in all detail, nor are the inventions limited to specific implementations described. Obviously, many modifications and changes can be made according to the content of this application. This application selects and specifically describes these embodiments in order to better explain the principles and practical applications of this application, so that those skilled in the art can well understand and use this application. This application is to be limited only by the claims, along with their full scope and equivalents.

Claims (10)

1. An audio stream processing method, comprising:

acquiring an audio stream to be processed, and identifying the coding format of the audio stream to be processed;

acquiring a set audio loudness parameter under the condition that the coding format of the audio stream to be processed is a panoramic sound coding format;

according to the set audio loudness parameter, the audio stream to be processed is coded into an audio stream in a stereo coding format;

and determining a target audio stream after transcoding the audio stream to be processed according to the audio stream in the stereo coding format.

2. The method of claim 1, wherein the transcoding the audio stream to be processed into an audio stream in a stereo encoding format according to the set audio loudness parameter comprises:

decoding the audio stream to be processed to obtain audio sampling data;

and according to the set audio loudness parameter and a set stereo coding strategy, carrying out coding processing on the audio sampling data to obtain an audio stream in a stereo coding format.

3. The method according to claim 2, wherein the encoding the audio sample data according to the set audio loudness parameter and the set stereo coding strategy to obtain the audio stream in the stereo coding format comprises:

and coding the audio sampling data according to a set stereo coding strategy, and adjusting the sound level of the audio sampling data within a set time length according to the set audio loudness parameter to obtain an audio stream in the stereo coding format.

4. A method according to claim 2 or 3, wherein the audio stream to be processed comprises a plurality of audio data packets;

the decoding processing of the audio stream to be processed to obtain audio sample data includes:

decoding each audio data packet to obtain sub-audio sampling data corresponding to each audio sampling data;

the step of coding the audio sampling data according to the set audio loudness parameter and the set stereo coding strategy to obtain an audio stream in a stereo coding format includes:

for each piece of sub-audio sampling data, according to the set audio loudness parameter and a set stereo coding strategy, coding the sub-audio sampling data to obtain a sub-audio stream in a stereo coding format;

and splicing the sub audio streams of the stereo coding format to obtain the audio stream of the stereo coding format.

5. The method of claim 1, wherein the obtaining the audio stream to be processed comprises:

acquiring a video stream to be processed, wherein the video stream to be processed comprises an audio stream to be processed and an image sequence to be processed;

after the obtaining of the multimedia stream to be processed, the method further includes:

transcoding the image sequence to be processed to obtain a target image sequence;

and aligning the target audio stream and the target image sequence to determine a target video stream.

6. The method of claim 1, wherein before obtaining the set audio loudness parameter, further comprising:

acquiring audio track information of the audio stream to be processed;

and identifying the coding format of the audio stream to be processed under the condition that the audio track information is a single-channel audio track.

7. The method according to claim 6, wherein after the obtaining of the track information of the audio stream to be processed, the method further comprises:

under the condition that the audio track information is two paths of audio tracks, respectively identifying the coding formats of the audio streams to be processed corresponding to the audio tracks;

and/or, carrying out file format transcoding on the audio stream to be processed with the coding format being a panoramic sound coding format to obtain a target audio stream with the panoramic sound coding format.

8. An audio stream processing apparatus, comprising:

the device comprises a first identification module, a second identification module and a processing module, wherein the first identification module is configured to acquire an audio stream to be processed and identify the coding format of the audio stream to be processed;

the acquisition module is configured to acquire a set audio loudness parameter under the condition that the coding format of the audio stream to be processed is a panoramagram coding format;

a first transcoding module configured to transcode the audio stream to be processed into an audio stream in a stereo coding format according to the set audio loudness parameter;

the determining module is configured to determine a target audio stream after transcoding the audio stream to be processed according to the audio stream in the stereo coding format.

9. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1-7 when executing the computer instructions.

10. A computer-readable storage medium storing computer instructions, which when executed by a processor, perform the steps of the method of any one of claims 1 to 7.

Images