CN118116397A - Audio metadata encoding and decoding method, transmission method, encoder terminal and system - Google Patents

Abstract

The application provides an audio metadata encoding and decoding method, a transmission method, an encoder terminal and a system, wherein metadata information after audio parameter modification by a user is updated to the tail part of an original encoding code stream, so that the new encoding code stream can simultaneously retain the original metadata information and the modified metadata information, and when the audio is rendered, if the modified metadata information cannot be realized due to external factors, the audio rendering can be performed according to the original metadata information, thereby ensuring the flexibility and the reliability of the audio rendering.

Description

Audio metadata encoding and decoding method, transmission method, encoder terminal and system

Technical Field

The present application relates to the field of audio processing, and in particular to an audio metadata encoding and decoding method, a transmission method, an encoder terminal and a system.

Background technique

Transparent transmission technology is applicable to a variety of communication protocols and network devices. It can improve the efficiency and stability of data transmission and reduce problems such as packet loss and delay during data transmission. In addition, transparent transmission technology can make the data transmission process transparent to upper-layer applications and does not require special configuration, thereby improving the flexibility and convenience of the system. Based on the advantages of the above transparent transmission technology, it is also widely used in audio metadata data transmission.

Currently, the technologies that support interactive metadata modification in transparent transmission scenarios mainly include Dolby MS12 and Fraunhofer MPEG-H. However, both have certain disadvantages, which are mainly reflected as follows:

Dolby MS12 method: After decoding the ES stream into metadata and PCM, the PCM is re-encoded into the ES stream for transparent transmission according to user configuration. The disadvantage of this method is that the solution is complex and additional encoding processing from PCM to ES is required;

Fraunhofer MPEG-H method: parse the metadata segments in the ES stream separately, modify the metadata according to the user configuration, re-encode the metadata and update and replace the original metadata segments. The disadvantage is that the original metadata is lost, and decoding and rendering can only be performed according to the modified metadata. Once the modified metadata cannot meet the current scene, no corresponding compensation can be made.

Summary of the invention

In order to solve one of the above-mentioned technical defects, an audio metadata encoding and decoding method, a transmission method, an encoder terminal and a system are provided in an embodiment of the present application.

A first aspect of an embodiment of the present application provides an audio metadata encoding and decoding method, the method comprising:

Obtaining a first encoded code stream, where the first encoded code stream is an original encoded code stream;

Decoding the first encoded bitstream to obtain first metadata information, wherein the first metadata information includes audio parameters modifiable by a user;

Acquire second metadata information, where the second metadata information is metadata information generated by a modified audio parameter set after a user modifies the audio parameters in the first metadata information;

The second metadata information is encoded, and the encoded second metadata information is updated to the end of the first encoding code stream to form a second encoding code stream.

A second aspect of an embodiment of the present application provides an audio transmission method, the method comprising:

The codec terminal obtains a first coded code stream, and decodes the first coded code stream to obtain first metadata information, wherein the first coded code stream is an original coded code stream, and the first metadata information includes audio parameters that can be modified by a user;

The codec terminal sends the first metadata information to the audio interactive terminal, and the audio interactive terminal receives second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information;

The audio interactive terminal sends the second metadata information to the codec terminal, and the codec terminal encodes the second metadata information and updates the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream;

The codec terminal transparently transmits the second encoded code stream to the power amplifier, the power amplifier decodes the second encoded code stream, obtains the first metadata information and the second metadata information, and outputs sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information.

A third aspect of an embodiment of the present application provides a codec terminal, the codec terminal comprising a processor, the processor having built-in operating instructions executable by the processor to execute the audio metadata codec method as described in the first aspect of the embodiment of the present application.

A fourth aspect of the embodiments of the present application provides an audio transmission system, the system comprising a speaker, an audio interactive terminal, a power amplifier and a codec terminal;

The codec terminal is used to obtain a first coded code stream and decode the first coded code stream to obtain first metadata information, wherein the first coded code stream is an original coded code stream, and the first metadata information includes audio parameters that can be modified by a user;

The audio interactive terminal is used to receive the first metadata information sent by the codec terminal, and receive the second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information;

The codec terminal is further used to receive second metadata information sent by the audio interactive terminal, encode the second metadata information, and update the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream and send it to the power amplifier;

The power amplifier is used to receive a second encoded code stream sent by the codec terminal, decode the second encoded code stream, obtain first metadata information and second metadata information, and output sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information.

By adopting the audio metadata encoding and decoding method provided in the embodiment of the present application, the metadata information after the user modifies the audio parameters is updated to the end of the original encoded code stream, so that the new encoded code stream can retain both the original metadata information and the modified metadata information. During audio rendering, if the modified metadata information cannot be realized due to external factors, audio rendering can also be performed according to the original metadata information, thereby ensuring the flexibility and reliability of audio rendering.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a flowchart of an audio metadata encoding and decoding method provided in Embodiment 1 of the present application;

FIG2 is a data format of a first encoding code stream provided in Example 1 of the present application;

FIG3 is a schematic diagram of an operation interface of an audio operation terminal provided in Example 1 of the present application;

FIG4 is a data format of a second encoding code stream provided in Example 1 of the present application;

FIG5 is a flow chart of an audio transmission method provided in Example 2 of the present application;

FIG6 is a schematic diagram of the principles of the audio transmission system provided in Example 4 of the present application.

Detailed ways

In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

Example 1

As shown in FIG1 , this embodiment provides an audio metadata encoding and decoding method, the method comprising:

S101: Obtain a first encoded bitstream.

Specifically, the audio metadata decryption method proposed in this embodiment is applied to a codec terminal, which may be an audio codec. The codec terminal can implement codec operations on audio data. In this embodiment, the codec terminal can be connected to an external device to obtain a coded stream sent by the external device. The coded stream is the first coded stream, which can also be regarded as an original coded stream. The original coded stream contains fixed fields and information related to audio. In this embodiment, the specific method of obtaining the first coded stream includes but is not limited to obtaining the first coded stream from a digital television, a local medium or a network link.

S102: Decode the first encoded bitstream to obtain first metadata information.

Specifically, after obtaining the first encoded code stream, the codec terminal needs to perform a decoding operation on it to obtain the original metadata information in the first encoded code stream, that is, the first metadata information. As shown in Figure 2, the data format of the first encoded code stream includes a frame header at the head, a metadata segment in the middle, and a sound bed and object segment at the end. Among them, the metadata segment in the middle is the content related to the first metadata information, which may specifically include audio parameters such as object volume and sound-related information. Among the audio parameters, some are fixed parameters, that is, default parameters that cannot be changed by the user, and some are variable parameters, that is, parameters that the user can modify according to actual needs.

S103: Acquire second metadata information.

Specifically, in the above process, it is mentioned that the first metadata information includes audio parameters that can be modified by the user. When the user modifies the audio parameters, the metadata information corresponding to the audio parameters will also change, and the data information generated after the change is the second metadata information. It should be noted that the second metadata information in this embodiment does not include the metadata information corresponding to the audio parameters that have not changed. In order to more clearly understand the difference between the first metadata information and the second metadata information in this embodiment, this embodiment will be further introduced.

In this embodiment, the user can modify the audio parameters through an operation interface. First, after the codec terminal decodes the first encoding bit stream to obtain the first metadata information, the audio parameters related to the first metadata information can be displayed on the operation interface through a UI interface, as shown in FIG3. The audio parameters shown in FIG3 are only for example, and the layout or parameter addition and deletion can be freely performed in actual application. In FIG3, the parameters that can be modified by the user include atmosphere, sound object, volume, and sound direction. If FIG3 shows the audio parameter state corresponding to the first metadata information, then, in the first metadata information, the atmosphere is standard, the sound object is Chinese commentary, the volume is mute, and the sound direction adjustment is right. At this time, the user needs to modify the atmosphere and the sound object according to actual needs. For example, the atmosphere is modified to the home and away atmosphere, and the sound object is modified to the English commentary. Then, the atmosphere and the sound object are the audio parameters that the user wants to select to modify, and the home and away atmosphere and the English commentary are the corresponding adjustment information. In this embodiment, the atmosphere, the sound object, the home and away atmosphere, and the English commentary are integrated to generate the second metadata information. At this point, it can be seen that the second metadata information only includes the modified audio parameters and their corresponding adjustment information, but does not include the unmodified audio parameters.

S104: Encode the second metadata information, and update the encoded second metadata information to the end of the first encoded code stream to form a second encoded code stream.

Specifically, after obtaining the second metadata information through the above process, this embodiment can encode the second metadata information through the encoder terminal. Then the encoded second metadata information is updated to the end of the first encoded code stream to form a second encoded code stream, as shown in Figure 4, and the modified metadata segment is the content related to the second metadata information in this embodiment. Since the second metadata information is directly updated at the end of the first encoded code stream, no changes are made to the first encoded code stream itself, so that the first encoded code stream completely retains the original metadata information. At the same time, it also makes it possible that in the second encoded code stream, there is both the first metadata information with the original audio parameter related information and the second metadata information with the modified audio parameter related information. In this way, when rendering audio, if the second metadata information cannot be realized due to external factors, audio rendering can also be performed according to the first metadata information, ensuring the flexibility and reliability of audio rendering.

In addition, in this embodiment, after the user selects the audio parameters to be modified and modifies the audio parameters, the display content of the modified audio parameters can also be adjusted on the operation interface. For example, as shown in Figure 3, the atmosphere is switched from the standard option to the home and away atmosphere, and the sound object is switched from the Chinese commentary option to the English commentary, etc., so that the user can intuitively see the audio parameters selected by him and the adjusted content.

Example 2

As shown in FIG. 5 , based on Embodiment 1, this embodiment proposes an audio transmission method, which includes:

S201: A codec terminal obtains a first coded code stream, and decodes the first coded code stream to obtain first metadata information.

Specifically, in this embodiment, the codec terminal can implement the codec operation of audio data. In this embodiment, the codec terminal can be connected to an external device to obtain a coded stream sent by the external device. The coded stream is the first coded stream, which can also be regarded as an original coded stream. The original coded stream contains fixed fields and information related to audio. In this embodiment, the codec terminal specifically obtains the first coded stream in ways including but not limited to digital television, local media or network links.

After obtaining the first encoded code stream, the codec terminal needs to perform a decoding operation on it to obtain the original metadata information in the first encoded code stream, that is, the first metadata information. The first metadata information may specifically include audio parameters such as object volume and sound-related information. Among the audio parameters, some are fixed parameters, that is, default parameters that cannot be changed by the user, and some are variable parameters, that is, parameters that can be modified by the user according to actual needs.

S202: The codec terminal sends the first metadata information to the audio interactive terminal, and the audio interactive terminal receives second metadata information generated by a modified audio parameter set after the user modifies the audio parameters in the first metadata information.

Specifically, in the above process, it is mentioned that the first metadata information includes audio parameters that can be modified by the user. When the user modifies the audio parameters, the metadata information corresponding to the audio parameters will also change, and the data information generated after the change is the second metadata information. It should be noted that the second metadata information in this embodiment does not include the metadata information corresponding to the audio parameters that have not changed. In order to more clearly understand the difference between the first metadata information and the second metadata information in this embodiment, this embodiment will be further introduced.

In this embodiment, the user can modify the audio parameters through an operation interface. First, after the codec terminal decodes the first encoding code stream to obtain the first metadata information, it sends the first metadata information to the audio interactive terminal. The audio interactive terminal can display the audio parameters related to the first metadata information on its operation interface through a UI interface, as shown in FIG3. The audio parameters shown in FIG3 are only for example, and the layout or parameter addition and deletion can be freely performed in actual application. In FIG3, the parameters available for user modification include atmosphere, sound object, volume, and sound direction. If FIG3 shows the audio parameter state corresponding to the first metadata information, then, in the first metadata information, the atmosphere is standard, the sound object is Chinese commentary, the volume is mute, and the sound direction adjustment is right. At this time, the user needs to modify the atmosphere and the sound object according to actual needs. For example, the atmosphere is modified to the home and away atmosphere, and the sound object is modified to the English commentary. Then, the atmosphere and the sound object are the audio parameters that the user wants to select to modify, and the home and away atmosphere and the English commentary are the corresponding adjustment information. In this embodiment, the atmosphere, the sound object, the home and away atmosphere, and the English commentary are integrated to generate the second metadata information. At this point, it can be seen that the second metadata information only includes the modified audio parameters and their corresponding adjustment information, but does not include the unmodified audio parameters.

S203: The audio interactive terminal sends the second metadata information to the codec terminal, and the codec terminal encodes the second metadata information and updates the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream.

Specifically, after the audio interactive terminal obtains the second metadata information through the above process, the second metadata information is sent to the codec terminal. In this embodiment, the second metadata information can be encoded by the encoder terminal. Then the encoded second metadata information is updated to the end of the first encoding stream to form a second encoding stream. Since the second metadata information is directly updated at the end of the first encoding stream, no changes are made to the first encoding stream itself, so that the first encoding stream completely retains the original metadata information. At the same time, in the second encoding stream, there is both the first metadata information with information related to the original audio parameters and the second metadata information with information related to the modified audio parameters. In this way, when rendering audio, if the second metadata information cannot be realized due to external factors, audio rendering can also be performed according to the first metadata information, thereby ensuring the flexibility and reliability of audio rendering.

In addition, in this embodiment, after the user selects the audio parameters to be modified and modifies the audio parameters, the display content of the modified audio parameters can also be adjusted on the operation interface. For example, as shown in Figure 3, the atmosphere is switched from the standard option to the home and away atmosphere, and the sound object is switched from the Chinese commentary option to the English commentary, etc., so that the user can intuitively see the audio parameters selected by him and the adjusted content.

S204: The codec terminal transparently transmits the second encoded code stream to the power amplifier, the power amplifier decodes the second encoded code stream to obtain first metadata information and second metadata information, and outputs sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information.

Specifically, after the codec terminal encodes the second metadata information to generate the second encoded code stream, the second encoded code stream is transparently transmitted to the power amplifier. The transparent transmission method can be implemented by HDMI, ARC, EARC or SPDIF, which is not specifically limited in this embodiment. After receiving the second encoded code stream, the power amplifier can decode it, and then output the sound through the speakers connected to the power amplifier according to the first metadata information and the second metadata information. In this process, the power amplifier can also choose to use the first metadata information or the second metadata information for audio rendering according to the configuration of the speakers connected to it.

This embodiment updates the metadata information after the user modifies the audio parameters to the end of the original encoded bitstream, so that the new encoded bitstream can retain both the original metadata information and the modified metadata information. During audio rendering, if the modified metadata information cannot be implemented due to external factors, audio rendering can also be performed based on the original metadata information, thereby ensuring the flexibility and reliability of audio rendering.

Example 3

This embodiment provides a codec terminal, which includes a processor, and the processor has built-in operating instructions executable by the processor to perform the following steps:

Obtaining a first encoded code stream, where the first encoded code stream is an original encoded code stream;

Decoding the first encoded bitstream to obtain first metadata information, wherein the first metadata information includes audio parameters modifiable by a user;

Acquire second metadata information, where the second metadata information is metadata information generated by a modified audio parameter set after a user modifies the audio parameters in the first metadata information;

The second metadata information is encoded, and the encoded second metadata information is updated to the end of the first encoding code stream to form a second encoding code stream.

The specific working process of the codec terminal proposed in this embodiment can refer to the content of Embodiment 1, and will not be described in detail in this embodiment.

This embodiment updates the metadata information after the user modifies the audio parameters to the end of the original encoded bitstream, so that the new encoded bitstream can retain both the original metadata information and the modified metadata information. During audio rendering, if the modified metadata information cannot be implemented due to external factors, audio rendering can also be performed based on the original metadata information, thereby ensuring the flexibility and reliability of audio rendering.

Example 4

As shown in FIG6 , this embodiment provides an audio transmission system, which includes a speaker, an audio interactive terminal, a power amplifier, and a codec terminal;

The codec terminal is used to obtain a first coded code stream and decode the first coded code stream to obtain first metadata information, wherein the first coded code stream is an original coded code stream, and the first metadata information includes audio parameters that can be modified by a user;

The audio interactive terminal is used to receive the first metadata information sent by the codec terminal, and receive the second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information;

The codec terminal is further used to receive second metadata information sent by the audio interactive terminal, encode the second metadata information, and update the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream and send it to the power amplifier;

The power amplifier is used to receive a second encoded code stream sent by the codec terminal, decode the second encoded code stream, obtain first metadata information and second metadata information, and output sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information.

The specific working process of the audio transmission system proposed in this embodiment can refer to the content of Example 2, and this embodiment will not be repeated.

This embodiment updates the metadata information after the user modifies the audio parameters to the end of the original encoded bitstream, so that the new encoded bitstream can retain both the original metadata information and the modified metadata information. During audio rendering, if the modified metadata information cannot be implemented due to external factors, audio rendering can also be performed based on the original metadata information, thereby ensuring the flexibility and reliability of audio rendering.

In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (11)

1. An audio metadata encoding and decoding method, characterized in that the method comprises: Obtaining a first encoded code stream, where the first encoded code stream is an original encoded code stream; Decoding the first encoded bitstream to obtain first metadata information, wherein the first metadata information includes audio parameters modifiable by a user; Acquire second metadata information, where the second metadata information is metadata information generated by a modified audio parameter set after a user modifies the audio parameters in the first metadata information; The second metadata information is encoded, and the encoded second metadata information is updated to the end of the first encoding code stream to form a second encoding code stream. 2. The method according to claim 1 is characterized in that the way of obtaining the first encoded code stream includes obtaining the first encoded code stream from a digital television, a local medium or a network link. 3. The method according to claim 1, wherein the process of obtaining the second metadata information comprises: Displaying the first metadata information on an operation interface; Acquire the audio parameters selected by the user through the operation interface and the adjustment information after adjusting the audio parameters; The audio parameter selected by the user and the adjustment information corresponding to the audio parameter selected by the user are integrated to generate second metadata information. 4. The method according to claim 3, characterized in that after obtaining the audio parameters selected by the user through the operation interface and the adjustment information after adjusting the audio parameters, the method further comprises: The display content of the audio parameter selected by the user on the operation interface is adjusted according to the audio parameter selected by the user and the adjustment information corresponding to the audio parameter selected by the user. 5. An audio transmission method, characterized in that the method comprises: The codec terminal obtains a first coded code stream, and decodes the first coded code stream to obtain first metadata information, wherein the first coded code stream is an original coded code stream, and the first metadata information includes audio parameters that can be modified by a user; The codec terminal sends the first metadata information to the audio interactive terminal, and the audio interactive terminal receives second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information; The audio interactive terminal sends the second metadata information to the codec terminal, and the codec terminal encodes the second metadata information and updates the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream; The codec terminal transparently transmits the second encoded code stream to the power amplifier, the power amplifier decodes the second encoded code stream, obtains the first metadata information and the second metadata information, and outputs sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information. 6. The method according to claim 5, characterized in that the codec terminal obtains the first coded code stream from a digital television, a local medium or a network link. 7. The method according to claim 5 is characterized in that the process of the audio interactive terminal receiving the second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information comprises: The audio interactive terminal displays the first metadata information on an operation interface; The audio interactive terminal obtains the audio parameters selected by the user through the operation interface and the adjustment information after adjusting the audio parameters; The audio interactive terminal integrates the audio parameters selected by the user and the adjustment information corresponding to the audio parameters selected by the user to generate second metadata information. 8. The method according to claim 7, characterized in that after the audio interactive terminal obtains the audio parameters selected by the user through the operation interface and the adjustment information after adjusting the audio parameters, the method further comprises: The audio interactive terminal adjusts the display content of the audio parameter selected by the user on the operation interface according to the audio parameter selected by the user and the adjustment information corresponding to the audio parameter selected by the user. 9. The method according to claim 5, characterized in that the codec terminal transparently transmits the second encoded code stream to the power amplifier via HDMI, ARC, EARC or SPDIF. 10. A codec terminal, characterized in that the codec terminal comprises a processor, wherein the processor has built-in operating instructions executable by the processor to execute the audio metadata codec method according to any one of claims 1 to 4. 11. An audio transmission system, characterized in that the system comprises a speaker, an audio interactive terminal, a power amplifier and a codec terminal; The codec terminal is used to obtain a first coded code stream and decode the first coded code stream to obtain first metadata information, wherein the first coded code stream is an original coded code stream, and the first metadata information includes audio parameters that can be modified by a user; The audio interactive terminal is used to receive the first metadata information sent by the codec terminal, and receive the second metadata information generated by the modified audio parameter set after the user modifies the audio parameters in the first metadata information; The codec terminal is further used to receive second metadata information sent by the audio interactive terminal, encode the second metadata information, and update the encoded second metadata information to the end of the first encoding code stream to form a second encoding code stream and send it to the power amplifier; The power amplifier is used to receive a second encoded code stream sent by the codec terminal, decode the second encoded code stream, obtain first metadata information and second metadata information, and output sound through a speaker connected to the power amplifier according to the first metadata information and the second metadata information.