CN118802874A - Audio data transmission method, device, electronic device and readable storage medium - Google Patents

Abstract

The application provides an audio data transmission method, an audio data transmission device, electronic equipment and a readable storage medium, wherein the method comprises the following steps: receiving a real-time transport protocol RTP data packet; the RTP data packet carries first audio data and first indication information, wherein the first indication information is used for indicating that the encoding mode of the first audio data is an audio-video encoding-decoding standard AVS encoding mode; and decoding the first audio data according to the first indication information. The application can realize real-time transmission of the audio data obtained by adopting the AVS coding mode.

Description

Audio data transmission method, device, electronic device and readable storage medium

Technical Field

The present application relates to the field of communication technology, and in particular to an audio data transmission method, device, electronic device and readable storage medium.

Background Art

The rapid development of real-time audio and video transmission provides users with more convenient conditions. For example, live broadcast, video conferencing, remote control, etc. based on Web Real-Time Communications (WebRTC) provide users with convenient real-time audio and video communication scenarios. The Audio Video Coding Standard (AVS) includes four main technical standards such as system, video, audio, and digital rights management, as well as supporting standards such as consistency testing. The AVS encoding method has high encoding efficiency, low implementation complexity, and greater application prospects. However, there is currently no solution for how to achieve real-time transmission of audio data obtained using the AVS encoding method.

Summary of the invention

The present application provides an audio data transmission method, device, electronic device and readable storage medium, which can solve the problem that there is currently no solution for how to achieve real-time transmission of audio data obtained using the AVS encoding method.

An embodiment of the present application provides an audio data transmission method, which is applied to a receiving-side device, and the method includes:

Receive a Real-time Transport Protocol (RTP) data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is an audio and video codec standard AVS encoding method;

The first audio data is decoded according to the first indication information.

Optionally, before receiving the RTP data packet, the method further includes:

The encoding configuration information is determined through a media negotiation process with a sending-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information;

Encoder configuration parameters;

Bit rate;

The encoder identification information includes at least one of the following:

First identification information, used to indicate the encoding category for encoding the audio data;

The second identification information is used to indicate whether the encoder includes a neural network model or not;

The third identification information is used to indicate the category of the neural network model in the encoder.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

Optionally, the decoding the first audio data according to the first indication information includes:

When the timestamps carried in the received multiple RTP data packets are the same, combining the first audio data carried in the multiple RTP data packets to obtain combined audio data;

The combined audio data is decoded according to the first indication information.

Optionally, the decoding the first audio data according to the first indication information includes:

In the case where the marker bit carried in the received first RTP data packet is a first value, determining at least one second RTP data packet; wherein each of the second RTP data packets has the same timestamp as that carried in the first RTP data packet;

Combining the first audio data carried in a plurality of RTP data packets to obtain combined audio data; wherein the plurality of RTP data packets include: the first RTP data packet and the at least one second RTP data packet;

The combined audio data is decoded according to the first indication information.

Optionally, when the marker bit carried in the received first RTP data packet is a first value, determining at least one second RTP data packet includes:

In the case where the marker bit carried in the received first RTP data packet is a first value, determining a target RTP data packet; wherein the marker bit carried in the target RTP data packet is the first value, and the timestamps carried in the target RTP data packet and the first RTP data packet are different;

An RTP data packet that carries a sequence number between a first sequence number and a second sequence number and carries a timestamp that is the same as the timestamp carried in the first RTP data packet is determined as the second RTP data packet; wherein the first sequence number is the sequence number carried in the first RTP data packet, and the second sequence number is the sequence number carried in the target RTP data packet.

Optionally, combining the first audio data carried in the multiple RTP data packets to obtain combined audio data includes:

In the case where the sequence numbers carried in the multiple RTP data packets are continuous, the first audio data carried in the multiple RTP data packets are combined in sequence according to the increasing or decreasing order of the sequence numbers carried in the multiple RTP data packets to obtain combined audio data.

Optionally, the audio data transmission method further includes at least one of the following:

In the case where the sequence numbers carried in the multiple RTP data packets are discontinuous, discarding the multiple RTP data packets and receiving the RTP data packets again;

When the marker bit carried in the received RTP data packet is the second value, continue to receive the RTP data packet until the marker bit carried in the received RTP data packet is the first value or the received RTP data packet is discarded when the timer times out.

Optionally, receiving a real-time transport protocol RTP data packet includes:

The RTP data packet is received through the WebRTC media channel.

The present application provides an audio data transmission method, which is applied to a sending side device, and the method includes:

Obtaining first audio data; wherein the first audio data is obtained by encoding based on the AVS encoding method;

Encapsulate the first audio data according to RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method;

The RTP data packet is sent.

Optionally, before obtaining the first audio data, the method further includes:

The encoding configuration information is determined through a media negotiation process with a receiving-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information;

Encoder configuration parameters;

Bit rate;

The encoder identification information includes at least one of the following:

First identification information, used to indicate the encoding category for encoding the audio data;

The second identification information is used to indicate whether the encoder includes a neural network model or not;

The third identification information is used to indicate the category of the neural network model in the encoder.

Optionally, obtaining the first audio data includes:

Get initial audio data;

According to the encoding configuration information, the initial audio data is encoded using the AVS encoding method to obtain encoded audio data;

The encoded audio data is divided into a plurality of the first audio data.

Optionally, encapsulating the first audio data according to RTP to obtain an RTP data packet includes:

Each of the first audio data is encapsulated according to RTP to obtain multiple RTP data packets; wherein each RTP data packet carries one of the first audio data and the first indication information.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

Optionally, sending the RTP data packet includes:

The RTP data packet is sent through the WebRTC media channel.

The present application provides an audio data transmission device, which is applied to a receiving device, including:

A receiving module, configured to receive an RTP data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method;

A decoding module is used to decode the first audio data according to the first indication information.

The present application provides an audio data transmission device, which is applied to a sending side device, including:

An obtaining module, used to obtain first audio data; wherein the first audio data is obtained by encoding based on the AVS encoding method;

An encapsulation module, used to encapsulate the first audio data according to RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method;

A sending module is used to send the RTP data packet.

An embodiment of the present application provides an electronic device, characterized in that it includes: a processor, a memory, and a computer program stored in the memory and executable on the processor; wherein, when the processor executes the computer program, the steps of the audio data transmission method of the receiving side device as described above are implemented; or when the processor executes the computer program, the steps of the audio data transmission method of the sending side device as described above are implemented.

An embodiment of the present application provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the steps of the audio data transmission method of the receiving side device as described above are implemented; or when the program is executed by a processor, the steps of the audio data transmission method of the sending side device as described above are implemented.

In an embodiment of the present application, the first audio data obtained based on the AVS encoding method is carried by an RTP data packet, and the encoding method of the first audio data is indicated as the AVS encoding method in the RTP data packet, so that the receiving side device can decode the first audio data according to the first indication information to realize the real-time transmission of the first audio data obtained based on the AVS encoding method, which solves the problem that there is currently no solution to how to realize real-time transmission of audio data obtained using the AVS encoding method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart showing an audio transmission method of a receiving device according to an embodiment of the present application;

FIG2 is a schematic diagram showing the RTP data packet header structure of an embodiment of the present application;

FIG3 is a flow chart showing an audio transmission method of a transmitting device according to an embodiment of the present application;

FIG4 is an overall flow chart of the audio transmission method according to an embodiment of the present application;

FIG5 is a block diagram showing an audio transmission device of a receiving side device according to an embodiment of the present application;

FIG6 is a block diagram showing an audio transmission device of a transmitting side device according to an embodiment of the present application;

FIG. 7 is a block diagram of an electronic device according to an embodiment of the present application.

DETAILED DESCRIPTION

In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In various embodiments of the present application, it should be understood that the order of execution of the following processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the terms "system" and "network" are often used interchangeably in this article.

In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and/or other information.

As shown in Figure 1, an embodiment of the present application provides an audio data transmission method, which is applied to a receiving side device. Optionally, the receiving side device may be a device that supports audio data reception or supports audio data reception and output. Of course, the embodiment of the present application is not limited to the receiving side device only supporting the reception or output of audio data, etc., and may also support the reception or output of video data, or may also support the transmission of audio data and/or video data, etc. The embodiment of the present application is not limited to this.

Specifically, the method comprises the following steps:

Step 11: Receive an RTP data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is AVS encoding method.

It should be noted that AVS includes four main technical standards, namely system, video, audio, and digital rights management, as well as supporting standards such as consistency testing. The AVS encoding method involved in the embodiments of the present application mainly refers to the audio encoding method based on AVS, including but not limited to the audio encoding method of AVS1.0, the audio encoding method of AVS2.0, the audio encoding method of AVS3.0, or other versions of AVS audio encoding methods, etc. The embodiments of the present application are not limited to this.

Optionally, the first indication information may be carried in a header of an RTP data packet.

Step 12: Decode the first audio data according to the first indication information.

Optionally, the receiving device can decapsulate the RTP data packet and read the first indication information and the first audio data carried therein. In this way, when it is known through the first indication information that the encoding method of the first audio data is the AVS encoding method, the first audio data can be decoded using a corresponding decoding method, and then the decoded audio data can be output.

In the above scheme, the first audio data obtained based on the AVS encoding method is carried by the RTP data packet, and the encoding method of the first audio data is indicated as the AVS encoding method in the RTP data packet, so that the receiving side device can decode the first audio data according to the first indication information to realize the real-time transmission of the first audio data obtained based on the AVS encoding method, which solves the problem that there is currently no solution to how to realize real-time transmission of audio data obtained using the AVS encoding method.

Optionally, before receiving the RTP data packet, the method further includes:

The encoding configuration information is determined through a media negotiation process with a sending-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

In this embodiment, the sending device and the receiving device can negotiate and determine the encoding configuration used for the audio data transmitted between the sending device and the receiving device based on the media negotiation process. For example, the media negotiation process can be implemented based on the Session Description Protocol (SDP), which is used to transmit media stream information in a media session. The two parties in a call (the sending device and the receiving device) can negotiate through SDP whether various capabilities such as encoding and decoding match, and allow the recipient of the session description to participate in the session.

For example, when implementing AVS codec negotiation based on SDP, the following methods can be used to carry the codec name information:

In the line description ("m=") of the SDP media description, information such as media type, transmission type, and payload type can be described. The media name field is used to indicate the media type. For example, if medianame corresponds to the media type of Multipurpose Internet Mail Extensions (MIME), which is audio, "m="audio" means that starting from line m, information related to audio data will be described.

In the description of the SDP media format, the value of the payload type (a=rtpmap) corresponds to the payload type (Payload Type, PT) of the RTP packet header, which can be defined by the rtpmap field, and the attribute information can be defined by the format parameters (fmtp) field that follows it. When describing audio data related information, "a=rtpmap" can be used to indicate the AVS encoding method. For example, the encoding name field of "a=rtpmap" indicates that the MIME subname is "AV3A-AATF", which means that the AVS encoding method is used. Specifically, a=rtpmap is described as follows:

a＝rtpmap:<payload type><encoding name>/<clock rate>[/<encodingparameters>];

Among them, "a=rtpmap:111AV3A-AATF/44100/5" means that the encoding name field indicates that the encoding method is AV3A-AATF, that is, the AVS encoding method is adopted.

Optionally, the clock rate field of "a=rtpmap" may be used to indicate the clock frequency of the audio, and the unit may be HZ. For example, "a=rtpmap:111AV3A-AATF/44100/5" indicates that when the clock rate field indicates that the AVS encoding method is used, the clock frequency of the audio is 44100.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information; for example: the encoder identification information is used to indicate the type of encoder when the AVS encoding method is adopted (for example: the encoding category for encoding audio data, whether the encoder includes a neural network model, the category of the neural network model used when the neural network model is included, etc.).

Encoder configuration parameters; for example, the encoder configuration parameters are used to indicate the configuration parameters of the encoder (such as the number of audio channels, etc.) when using the AVS encoding method.

Bit rate: For example, this bit rate is used to indicate the bit rate of the audio stream when the AVS encoding method is used.

In the description of the SDP media format, "a=fmtp" can describe the relevant information of the encoder, such as encoder identification information (codec-nn-id), encoder configuration parameters (config), bitrate, etc. Different parameter configurations can be used for different audio formats.

Optionally, the encoder identification information includes at least one of the following:

The first identification information is used to indicate the encoding category for encoding the audio data; for example, the encoding category includes but is not limited to: general high-bitrate audio encoding data, lossless audio encoding data, general full-bitrate audio encoding data, etc.

The second identification information is used to indicate whether the encoder includes or does not include a neural network model, that is, the second identification information is used to indicate whether the encoder includes a neural network model. For example, a 1-bit value of 0 can be used to indicate that the neural network model is not included, and a 1-bit value of 1 can be used to indicate that the neural network model is included, or a 1-bit value of 1 can be used to indicate that the neural network model is not included, and a 1-bit value of 0 can be used to indicate that the neural network model is included, etc. The embodiments of the present application are not limited to this.

The third identification information is used to indicate the category of the neural network model in the encoder; for example: when the second identification information indicates that the encoder contains a neural network model, the third identification information can be used to indicate the category of the neural network model; or when the encoding configuration information includes the third identification information but does not include the second indication information, it means that the encoder contains a neural network model and indicates the category of the neural network model (for example, the category of the neural network model is a low-complexity neural network model, etc.).

For example, when implementing AVS codec negotiation based on SDP, the above encoding configuration information can be carried in the following manner:

codec-nn-id field: two bytes in hexadecimal can be used to describe the type of encoder.

The first byte (MSB) describes the encoding category of the audio media resource (i.e., the encoding category for encoding the audio data) through the audio coding identifier (audio_codec_id). For example: audio_codec_id is 0, indicating that the media resource is general high-bitrate audio coding data; audio_codec_id is 1, indicating that the media resource is lossless audio coding data; audio_codec_id is 2, indicating that the media resource is general full-bitrate audio coding data; the rest of the audio_codec_id values can be reserved.

Among them, the second byte can be used to describe the information related to the neural network model (nn_type) in the encoder, including whether the encoder contains a neural network model, and/or the category of the neural network model, etc. For example: when the second byte value is 0, it means that the encoder does not contain a neural network model; when the second byte value is 1, it means that the encoder contains a neural network model, and the encoder uses a low-complexity neural network model, etc. The embodiments of the present application are not limited to this.

For example: If a general high-rate encoder with audio_codec_id 0 is used and the encoder does not contain a neural network model, codec-nn-id = 0x0000. If a lossless audio encoder with audio_codec_id 1 is used and the encoder does not contain a neural network model, codec-nn-id = 0x0100. If a general full-rate encoder with audio_codec_id 2 is used and a low-complexity neural network model is used (corresponding to nn_type 1), codec-nn-id = 0x0201.

Config field: A hexadecimal string can be used to describe the configuration parameters of the encoder.

Taking the CA3SpecificBox data structure in AVS Part 6 (AVSP6) as an example, the Config field corresponds to the CA3SpecificBox data structure in AVSP6. The CA3SpecificBox data structure is as follows:

In the above data structure: when audio_codec_id is 2, it corresponds to the AVS3 audio full bit rate (GA) encoding configuration; when audio_codec_id is 0, it corresponds to the AVS3 audio high bit rate (GH) encoding configuration; when audio_codec_id is , it corresponds to the AVS3 audio lossless (LL) encoding configuration.

bitrate field: used to describe the bitrate of the audio stream, indicating the total bitrate in kb/s. Taking the AVS3 audio full bitrate encoding specific configuration (Avs3AudioGASpecificConfig) data structure in Part 6 (AVS3P6) of the AVS3 version as an example, the bitrate field corresponds to the total bitrate (total_bitrate) defined in Avs3AudioGASpecificConfig. Among them, the Avs3AudioGASpecificConfig data structure is as follows:

In the above data structure, "..." represents other configuration parameters, which are not limited in the present embodiment; "unsigned int (16) total_bitrate" is used to describe the total bit rate of the audio stream when the AVS encoding method is adopted. "unsigned int (2) resolution" is used to describe the resolution.

Optionally, when implementing AVS codec negotiation based on SDP, it is also necessary to configure the encoding rate using the AVS encoding method, corresponding to the RTP timestamp scale, which can be the same as the audio sampling rate. If not specified during the negotiation process, the default value (for example, the default value is 90000) can be used.

Optionally, the decoding the first audio data includes:

The first audio data is decoded according to the first indication information and the encoding configuration information.

In this embodiment, when the sending side device and the receiving side device determine the encoding configuration information of the AVS encoding method based on the media negotiation process, if the RTP data packet received by the receiving side device indicates through the first indication information that the encoding method of the first audio data it carries is the AVS encoding method, then the receiving side device can decode the first audio data based on the encoding configuration information of the AVS encoding method, that is, realize real-time transmission of the audio data obtained based on the AVS encoding method.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

For example, when the sending side device sends multiple RTP data packets to the receiving side device, the sequence numbers of these multiple RTP data packets are continuous. For example: when there are multiple first audio data corresponding to the same frame of audio data (that is, one frame of audio data is divided into multiple first audio data), the sequence numbers carried in the RTP data packets corresponding to each first audio data are arranged continuously according to the time sequence of the audio data itself, and the timestamps carried in the RTP data packets corresponding to each first audio data are the same. Correspondingly, the sequence numbers carried in the RTP data packets corresponding to the first audio data of different frames of audio data can also be arranged continuously according to the time sequence of the different frames of audio data themselves (for example, the sequence numbers of the RTP data packets in the same RTP session are continuous), and the timestamps carried in the RTP data packets corresponding to the first audio data of different frames of audio data are different. In this way, the receiving side device can identify the audio data belonging to the same frame based on the timestamp, and determine whether the packet is lost based on the sequence number. In addition, the marker bit carried in the RTP data packet corresponding to the first and/or last first audio data among the multiple first audio data obtained by dividing a frame of audio data is set to the first value, which can ensure that the receiving side device can accurately know the audio data belonging to the same frame.

As shown in Figure 2, a schematic diagram of the RTP header structure is given, which is described in detail as follows:

V field: used to indicate the version of RTP, for example: V=2 is used to indicate RTP version 2.

P field: 1 bit, used to indicate whether to pad. If the P field is set to allow padding, one or more bytes are padded at the end of the RTP packet. These padded bytes are not part of the payload.

X field: 1 bit, used to set the extension bit. If the extension bit is set through the X field, an extension header follows the fixed header (that is, an extension header follows the RTP header).

CC field: 4 bits, used to indicate the contributing source (CSRC) count, that is, the number of CSRC identifiers following the fixed header.

M field: 1 bit, used to mark important events in the bit stream. For example, the marking bit can be carried in the M field.

PT field: 7 bits, used to indicate the payload format carried by the RTP data packet. For example, the first indication information can be carried in the PT field.

Sequence number: 16 bits, used to indicate the sequence of the RTP data packet.

Timestamp: 32 bits, used to indicate the sampling time of the RTP data packet. For example, different RTP data packets of the same frame have the same timestamp.

Synchronization source (SSRC) identifier field: 32 bits, used to identify the synchronization source. This field is a random number, and there is only one synchronization identifier in the same RTP session. For example, two synchronization sources participating in the same video conference cannot have the same SSRC identifier.

Contributing source (CSRC) identifier field: a list of 16 items, such as 0 to 15, each with 32 bits, used to indicate all contributing sources of the payload carried by this RTP packet.

Optionally, the decoding the first audio data according to the first indication information includes:

When the timestamps carried in the received multiple RTP data packets are the same, combining the first audio data carried in the multiple RTP data packets to obtain combined audio data;

The combined audio data is decoded according to the first indication information.

In this embodiment, when the sending side device performs RTP encapsulation for the audio data using the AVS encoding method, the same frame of audio data is divided into multiple first audio data (here, the first audio data can also be understood as an audio segment in a frame of audio data, which can include audio-related information and/or audio data itself), and the timestamp carried in the RTP data packet corresponding to each first audio data is set to the same value, so for the receiving side device, the first audio data belonging to the same frame can be identified according to the timestamp. In this way, the receiving side device can group the multiple first audio data of the same frame into frames, and then perform decoding processing, so that a frame of audio data can be decoded and output.

Optionally, the decoding the first audio data according to the first indication information includes:

In the case where the marker bit carried in the received first RTP data packet is a first value, determining at least one second RTP data packet; wherein each of the second RTP data packets has the same timestamp as that carried in the first RTP data packet;

Combining the first audio data carried in a plurality of RTP data packets to obtain combined audio data; wherein the plurality of RTP data packets include: the first RTP data packet and the at least one second RTP data packet (that is, combining the first audio data carried in the first RTP data packet and the at least one second RTP data packet to obtain the combined audio data);

The combined audio data is decoded according to the first indication information.

In this embodiment, when the sending side device performs RTP encapsulation for the audio data using the AVS encoding method, the marker bit carried in the RTP data packet corresponding to the first and/or last first audio data of the multiple first audio data divided into the same frame of audio data is the first value, so for the receiving side device, it can be determined based on the marker bit whether the multiple first audio data belonging to the same frame are completely received. In this way, the receiving side device performs framing for the multiple first audio data of the same frame and then performs decoding processing, so as to ensure that a complete frame of audio data is obtained through decoding and output.

Optionally, when the marker bit carried in the received first RTP data packet is a first value, determining at least one second RTP data packet includes:

In the case where the marker bit carried in the received first RTP data packet is a first value, determining a target RTP data packet; wherein the marker bit carried in the target RTP data packet is the first value, and the timestamps carried in the target RTP data packet and the first RTP data packet are different;

An RTP data packet that carries a sequence number between a first sequence number and a second sequence number and carries a timestamp that is the same as the timestamp carried in the first RTP data packet is determined as the second RTP data packet; wherein the first sequence number is the sequence number carried in the first RTP data packet, and the second sequence number is the sequence number carried in the target RTP data packet.

For example: when the sending side device performs RTP encapsulation for audio data using the AVS encoding method, among the multiple first audio data divided into the same frame of audio data, the marker bit carried in the RTP data packet corresponding to the last first audio data is the first value, then when the receiving side device receives the first RTP data packet carrying the marker bit of the first value, it is determined that the first audio data carried in the first RTP data packet is the last first audio data (or audio fragment) of the frame in which it is located. At this time, the receiving side device can trace back in the descending order of the sequence number carried in the first RTP data packet until the target RTP data packet carrying the marker bit of the first value is traced back. When the sequence numbers carried by the traced target RTP data packet are continuous and the timestamps are different, the receiving side device can determine these traced RTP data packets (i.e., the RTP data packets carrying the sequence number between the first sequence number and the second sequence number, and the timestamp carried is the same as the timestamp carried in the first RTP data packet) as RTP data packets belonging to the same frame as the first audio data carried in the first RTP data packet.

Optionally, the audio data transmission method further includes:

When the marker bit carried in the received RTP data packet is the second value, continue to receive the RTP data packet until the marker bit carried in the received RTP data packet is the first value or the received RTP data packet is discarded when the timer times out.

In this embodiment, when the sending side device performs RTP encapsulation for audio data using the AVS encoding method, among the multiple first audio data divided into the same frame of audio data, when the marker bit carried in the RTP data packet corresponding to the last first audio data is the first value, if the marker bit carried in the RTP data packet received by the receiving side device is the second value, it is considered that the last first audio data of this frame has not been received, and the RTP data packet is continued to be received until the marker bit carried in the received RTP data packet is the first value, then it is considered that the last first audio data of this frame has been received, so that the framing process can be further performed. Alternatively, if in the process of continuing to receive the RTP data packet, the RTP data packet carrying the marker bit of the first value is still not received when the timer times out, the received RTP data packet is discarded to ensure transmission performance and avoid problems such as downtime.

For another example: when the sending side device performs RTP encapsulation for audio data using the AVS encoding method, among the multiple first audio data divided into the same frame of audio data, the marker bit carried in the RTP data packet corresponding to the first first audio data is the first value, then the receiving side device, when receiving the first RTP data packet carrying the marker bit of the first value, determines that the first audio data carried in the first RTP data packet is the first first audio data (or audio fragment) of the frame in which it is located. At this time, the receiving side device can continue to perform the packet receiving operation in the order of increasing sequence numbers carried in the first RTP data packet until the target RTP data packet carrying the marker bit of the first value is received. When the sequence numbers carried by the received target RTP data packets are continuous and the timestamps are different, the receiving side device can determine the received RTP data packets (i.e., the RTP data packets carrying the sequence numbers between the first sequence number and the second sequence number, and the RTP data packets carrying the same timestamp as the timestamp carried in the first RTP data packet) as RTP data packets belonging to the same frame as the first audio data carried in the first RTP data packet.

For another example: when the sending side device performs RTP encapsulation for audio data using the AVS encoding method, among the multiple first audio data divided into the same frame of audio data, the marker bit carried in the RTP data packet corresponding to the first and last first audio data is the first value, then when the receiving side device receives the first RTP data packet carrying the marker bit of the first value, it can determine that the first audio data carried in the first RTP data packet is the first or last first audio data (or audio fragment) of the frame in which it is located according to the marker bit carried in the RTP data packet with a continuous sequence number as the first value or the second value. For example, taking the determination that the first audio data carried in the first RTP data packet is the first first audio data of the frame in which it is located as an example, at this time, the receiving side device can continue to perform the packet receiving operation in the order of increasing the sequence number carried in the first RTP data packet until the target RTP data packet carrying the marker bit of the first value is received. If the sequence numbers carried by the received target RTP data packet are continuous and the timestamps are the same, it is determined that all the first audio data of this frame are received. In this way, the receiving side device can determine these RTP data packets with continuous sequence numbers received from the first RTP data packet as RTP data packets belonging to the same frame.

Optionally, combining the first audio data carried in the multiple RTP data packets to obtain combined audio data includes:

In the case where the sequence numbers carried in the multiple RTP data packets are continuous, the first audio data carried in the multiple RTP data packets are combined in sequence according to the increasing or decreasing order of the sequence numbers carried in the multiple RTP data packets to obtain combined audio data.

In this embodiment, for multiple RTP data packets carrying the same timestamp, it can be determined that they belong to the same frame of audio data. If the sequence numbers carried in the multiple RTP data packets are continuous, it is determined that the first audio data they carry is continuous in time, so that the first audio data in the multiple RTP data packets can be framed to obtain the framed audio data. In this way, the receiving side device frames the multiple first audio data of the same frame and then performs decoding processing, so that it can ensure that a complete frame of audio data is accurately decoded and output.

Optionally, the audio data transmission method further includes:

In the case that the sequence numbers carried in the multiple RTP data packets are not continuous, the multiple RTP data packets are discarded and the RTP data packets are received again.

In this embodiment, for multiple RTP data packets carrying the same timestamp, if it is determined that the sequence numbers carried in the multiple RTP data packets are discontinuous, it is considered that there may be packet loss in the transmission of this frame of audio data. At this time, the multiple RTP data packets can be discarded and the RTP data packets can be received again to ensure that the audio data of this frame is fully received.

Optionally, receiving a real-time transport protocol RTP data packet includes:

The RTP data packet is received through a web real-time communication WebRTC media channel.

In WebRTC technology, there are two channels, audio and video media channel and data channel (dataChannel), which can transmit audio and video data. For audio and video data obtained by AVS encoding, if the transmission is realized based on the existing WebRTC technology, when the audio and video data obtained by AVS encoding includes video data obtained by encoding based on the highly compressed digital video codec standard (H.264), it is transmitted through the WebRTC audio and video media channel, and the audio data obtained by AVS encoding needs to be transmitted through the WebRTC data channel.

When audio data obtained by AVS encoding is transmitted through the WebRTC data channel, although end-to-end transmission of audio data obtained by AVS encoding can be achieved, since the WebRTC data channel only supports the function of transparent data transmission, it does not support audio processing capabilities such as audio jitter buffering, audio and video synchronization, forward error correction (Forward Error Correction, FEC), echo cancellation, and noise suppression. Therefore, when the audio and video data obtained by AVS encoding include audio data and video data obtained by H264 encoding need to be transmitted at the same time, the above transmission method may cause audio playback to appear stuck, audio and video synchronization, obvious noise, and other problems.

In this embodiment, the sending side device divides the audio data encoded in the AVS mode into multiple first audio data and performs RTP encapsulation separately. The RTP data packet obtained by the encapsulation can be transmitted on the WebRTC media channel. In this way, when the audio and video data obtained in the AVS encoding mode need to be transmitted simultaneously, the audio and video data can be transmitted simultaneously on the WebRTC media channel to ensure the synchronization of audio and video playback and reduce the jamming and noise of audio playback.

As shown in Figure 3, an embodiment of the present application provides an audio data transmission method, which is applied to a sending side device. Optionally, the receiving side device can be a device that supports audio data reception or supports audio data reception and output. Of course, the embodiment of the present application is not limited to the receiving side device only supporting the reception or output of audio data, etc., but can also support the reception or output of video data, or can also support the sending of audio data and/or video data, etc. The embodiment of the present application is not limited to this.

Specifically, the method comprises the following steps:

Step 31: Obtain first audio data; wherein the first audio data is obtained by encoding based on the AVS encoding method;

Step 32: Encapsulate the first audio data according to RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method;

Step 33: Send the RTP data packet.

Optionally, before obtaining the first audio data, the method further includes:

The encoding configuration information is determined through a media negotiation process with a receiving-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information;

Encoder configuration parameters;

Bit rate;

The encoder identification information includes at least one of the following:

First identification information, used to indicate the encoding category for encoding the audio data;

The second identification information is used to indicate whether the encoder includes a neural network model or not;

The third identification information is used to indicate the category of the neural network model in the encoder.

Optionally, obtaining the first audio data includes:

Get initial audio data;

According to the encoding configuration information, the initial audio data is encoded using the AVS encoding method to obtain encoded audio data;

The encoded audio data is divided into a plurality of the first audio data.

Optionally, encapsulating the first audio data according to a real-time transport protocol RTP to obtain an RTP data packet includes:

Each of the first audio data is encapsulated according to RTP to obtain multiple RTP data packets; wherein each RTP data packet carries one of the first audio data and the first indication information.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

Optionally, sending the RTP data packet includes:

The RTP data packet is sent through the WebRTC media channel.

It should be noted that the audio transmission method of the sending side device in the embodiment of the present application and the audio transmission method of the receiving side device mentioned above are based on the same inventive concept, and their embodiments can refer to each other and will not be repeated here.

The following is an explanation of the RTP data packet encapsulation process of the sending device:

For the sending side device, based on the media negotiation process between the sending side device and the receiving side device, when it is determined that the AVS encoding method is needed for encoding, after obtaining the initial audio data (for example; it can be the audio data collected by the sending side device itself, or it can also be collected by the audio acquisition device and transparently transmitted to the sending side device), the AVS encoding method is used to encode the initial audio data to obtain the encoded audio data. Since the data size of the encoded audio data usually exceeds the size of the Maximum Transmission Unit (MTU) when the AVS encoding method is used for encoding, the encoded audio data needs to be split when the encoded audio data is encapsulated based on RTP. For example, starting from the starting position of the encoded audio data, it can be split in sequence according to the preset step length to the end position to obtain multiple first audio data; or, starting from the end position of the encoded audio data, it can be split in sequence according to the preset step length to the starting position to obtain multiple first audio data, etc.; wherein, the preset step length can be set according to the size of the RTP data packet after encapsulation not exceeding the size of the MTU, and the embodiment of the present application does not make specific restrictions.

In this way, the sending side device performs unpacking and encapsulation processing based on RTP for the audio data obtained by AVS encoding, so as to realize the real-time transmission of the audio data obtained by AVS encoding, and to realize the transmission of the audio data obtained by AVS encoding through the WebRTC media channel. In this way, when the audio and video data obtained by AVS encoding need to be transmitted simultaneously, the audio and video data can be transmitted simultaneously through the WebRTC media channel to ensure the synchronization of audio and video playback and reduce the jamming and noise of audio playback.

The following describes the audio transmission method of the embodiment of the present application in combination with the interaction process between the sending side device and the receiving side device:

As shown in FIG4 , an overall flow chart of an audio transmission method is provided, and the specific process includes:

Step 41: The audio streaming end (i.e., the sending side device) collects audio data;

Step 42: The audio streaming end encodes the audio data;

Step 43: the audio streaming end determines whether the encoded audio data is encoded using the AVS encoding method (or it can be understood as determining whether the data format of the encoded audio data is an audio encoding format based on the AVS encoding method, for example, by determining whether it carries the "AV3A-AATF" identifier, which can be seen in the above embodiment and will not be described in detail here)

If yes, execute step 44; if no, audio data transmission can be performed based on WebRTC technology;

Step 44: the audio streaming end unpacks the encoded audio data according to the unpacking rule, splits it into multiple RTP data packets, and stores the multiple RTP data packets in a sending queue after being consecutively marked with sequence numbers. Specifically, unpacking according to the unpacking rule includes:

441. For the audio data obtained by using the AVS encoding method, start executing the unpacking logic;

442. Unpack the encoded audio data according to the total size of each RTP data packet after encapsulation being less than or equal to the MTU size. For example, the MTU of the Internet Protocol (IP) layer is 1500 bytes, that is, the total size of each RTP data packet after encapsulation needs to be less than or equal to 1500 bytes (the specific value can be adjusted as needed according to the routing rules), and the embodiments of the present application are not limited thereto;

443. The sequence numbers of the unpacked RTP data packets increase in sequence, and the timestamps of the RTP data packets are consistent;

444. The M field in the RTP Header of the last RTP data packet is set to 1, indicating that it is the last RTP data packet of the audio frame;

Step 45: the audio streaming end sends the unpacked multiple RTP data packets to the network through the WebRTC media channel; or if unpacking is not performed, audio data transmission is performed based on the WebRTC technology;

Step 46: The audio receiving end (i.e., the receiving device) receives the RTP data packet through the WebRTC media channel;

Step 47: The audio receiving end sorts the received RTP data packets according to the RTP packet sequence numbers;

Step 48: The audio receiving end determines whether the RTP data packet needs to be framed according to the media type (for example, when the PT field in the RTP Header indicates AV3A-AATF, it is determined that framing is required);

If necessary, execute step 49, if not necessary, execute step 410;

Step 49: The audio receiving end performs a corresponding framing operation according to the unpacking rule in step 44, and then transmits the framed audio frame to the decoder; specifically, the framing operation includes:

491. For the audio data obtained by using the AVS encoding method, start executing the framing logic;

492. Determine whether the timestamps of the RTP data packets are consistent; if they are consistent, it is considered that these RTP data packets belong to the same frame; if they are inconsistent, it is considered that these RTP data packets do not belong to the same frame;

493. Determine whether there is an RTP data packet with a value of 1 in the M field in the RTP Header among the RTP data packets with the same timestamp; if not, continue to receive packets until they are discarded due to timeout; if so, it is considered that the last RTP data packet of the audio frame has been received;

494. Start tracing back from the sequence number carried in the RTP data packet whose M field value in the RTP Header is 1. If the RTP data packet traced back is consistent with the timestamp of the RTP data packet, and the sequence number is completely continuous, and can be traced back to the last RTP data packet of the previous audio frame (the timestamp is different, and the value of the M field is 1), then it is considered that all RTP data packets of the audio frame have been successfully received, and framing is performed based on these RTP data packets, that is, the framing operation is completed. Otherwise, restart the packet receiving operation.

Step 410: the audio receiving end decodes the audio data obtained by framing; or, if no framing operation is required, directly decodes the received audio data;

Step 411: The audio receiving end plays the decoded audio data.

In the embodiments of the present application, by extending the WebRTC technology, it is possible to support the transmission of audio encoding obtained by the AVS encoding method, and it is also possible to solve the problems of current WebRTC-based transmission such as stuttering, audio and video asynchrony, and high noise, and significantly improve the transmission quality of audio encoding obtained based on the AVS encoding method.

As shown in FIG. 5 , an embodiment of the present application provides an audio data transmission device 500, which is applied to a receiving-side device, including:

The receiving module 510 is used to receive a real-time transport protocol RTP data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the audio and video codec standard AVS encoding method;

The decoding module 520 is used to decode the first audio data according to the first indication information.

Optionally, the audio data transmission device 500 further includes:

The determination module is used to determine the encoding configuration information through a media negotiation process with a sending side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information;

Encoder configuration parameters;

Bit rate;

The encoder identification information includes at least one of the following:

First identification information, used to indicate the encoding category for encoding the audio data;

The second identification information is used to indicate whether the encoder includes a neural network model or not;

The third identification information is used to indicate the category of the neural network model in the encoder.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

Optionally, the decoding module 520 includes:

A first combining unit is used to combine the first audio data carried in the multiple RTP data packets to obtain combined audio data when the timestamps carried in the multiple RTP data packets are the same;

The first decoding unit is used to decode the combined audio data according to the first indication information.

Optionally, the decoding module 520 includes:

A determination unit, configured to determine at least one second RTP data packet when the marker bit carried in the received first RTP data packet is a first value; wherein each of the second RTP data packet has the same timestamp as the first RTP data packet;

A second combining unit is used to combine the first audio data carried in a plurality of RTP data packets to obtain combined audio data; wherein the plurality of RTP data packets include: the first RTP data packet and the at least one second RTP data packet;

The second decoding unit is used to decode the combined audio data according to the first indication information.

Optionally, the determining unit is further configured to:

In the case where the marker bit carried in the received first RTP data packet is a first value, determining a target RTP data packet; wherein the marker bit carried in the target RTP data packet is the first value, and the timestamps carried in the target RTP data packet and the first RTP data packet are different;

An RTP data packet that carries a sequence number between a first sequence number and a second sequence number and carries a timestamp that is the same as the timestamp carried in the first RTP data packet is determined as the second RTP data packet; wherein the first sequence number is the sequence number carried in the first RTP data packet, and the second sequence number is the sequence number carried in the target RTP data packet.

Optionally, the first combination unit or the second combination unit is further used for:

In the case where the sequence numbers carried in the multiple RTP data packets are continuous, the first audio data carried in the multiple RTP data packets are combined in sequence according to the increasing or decreasing order of the sequence numbers carried in the multiple RTP data packets to obtain combined audio data.

Optionally, the audio data transmission device 500 further includes at least one of the following:

A first processing module, configured to discard the multiple RTP data packets and re-receive the RTP data packets when the sequence numbers carried in the multiple RTP data packets are discontinuous;

The second processing module is used to continue receiving the RTP data packet when the mark bit carried in the received RTP data packet is the second value, until the mark bit carried in the received RTP data packet is the first value or the received RTP data packet is discarded when the timer times out.

Optionally, the receiving module 510 includes:

The receiving unit is used to receive the RTP data packet through the web real-time communication WebRTC media channel.

It should be noted that the device in the embodiment of the present application can implement the various embodiments of the audio transmission method of the above-mentioned receiving-side device and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

As shown in FIG6 , an embodiment of the present application provides an audio data transmission device 600, which is applied to a sending side device, including:

An acquisition module 610 is used to obtain first audio data; wherein the first audio data is obtained by encoding based on the audio and video codec standard AVS encoding method;

The encapsulation module 620 is used to encapsulate the first audio data according to the real-time transport protocol RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method;

The sending module 630 is used to send the RTP data packet.

Optionally, the audio data transmission device 600 further includes:

The determination module is used to determine the encoding configuration information through a media negotiation process with a receiving side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method.

Optionally, the encoding configuration information further includes at least one of the following:

Encoder identification information;

Encoder configuration parameters;

Bit rate;

The encoder identification information includes at least one of the following:

First identification information, used to indicate the encoding category for encoding the audio data;

The second identification information is used to indicate whether the encoder includes a neural network model or not;

The third identification information is used to indicate the category of the neural network model in the encoder.

Optionally, the obtaining module 610 includes:

An acquisition unit, used for acquiring initial audio data;

An encoding unit, configured to encode the initial audio data in the AVS encoding mode according to the encoding configuration information to obtain encoded audio data;

A splitting unit is used to split the encoded audio data into multiple first audio data.

Optionally, the encapsulation module 620 includes:

An encapsulation unit is used to encapsulate each of the first audio data according to RTP to obtain multiple RTP data packets; wherein each RTP data packet carries one of the first audio data and the first indication information.

Optionally, the RTP data packet also carries at least one of the following:

Serial number;

Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same;

Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame.

Optionally, the sending module 630 includes:

The sending unit is used to send the RTP data packet through the web real-time communication WebRTC media channel.

It should be noted that the device in the embodiment of the present application can implement the various embodiments of the audio transmission method of the above-mentioned sending side device and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

As shown in FIG7 , the embodiment of the present application further provides an electronic device, comprising: a processor 71; and a memory 73 connected to the processor 71 through a bus interface 72, wherein the memory 73 is used to store programs and data used by the processor 71 when performing operations, and the processor 71 calls and executes the programs and data stored in the memory 73. Among them, a transceiver 74 is connected to the bus interface 72, and is used to receive and send data under the control of the processor 71; optionally, when the electronic device is used as a receiving side device, the processor 71 is used to read the program in the memory 73 to implement the steps of the audio transmission method of the above-mentioned receiving side device; or, when the electronic device is used as a sending side device, the processor 71 is used to read the program in the memory 73 to implement the steps of the audio transmission method of the above-mentioned sending side device, and can achieve the same technical effect as the above-mentioned audio transmission method embodiment, to avoid repetition, it will not be repeated here.

It should be noted that in FIG. 7 , the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 71 and various circuits of memory represented by memory 73 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 74 may be a plurality of components, namely, a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. For different terminals, the user interface 75 may also be an interface capable of externally connecting or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like. The processor 71 is responsible for managing the bus architecture and general processing, and the memory 73 may store data used by the processor 71 when performing operations.

Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing relevant hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.

In addition, the specific embodiment of the present application also provides a readable storage medium on which a program is stored. When the program is executed by the processor, it implements the steps of the audio transmission method of the above-mentioned receiving side device, or implements the steps of the audio transmission method of the above-mentioned sending side device, and can achieve the same technical effect as the above-mentioned audio transmission method embodiment. To avoid repetition, it will not be repeated here.

In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications are also within the scope of protection of the present application.

Claims (21)

1. An audio data transmission method, characterized in that it is applied to a receiving-side device, the method comprising: Receive a real-time transport protocol RTP data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is an audio and video codec standard AVS encoding method; The first audio data is decoded according to the first indication information. 2. The audio data transmission method according to claim 1, characterized in that before receiving the real-time transport protocol RTP data packet, it also includes: The encoding configuration information is determined through a media negotiation process with a sending-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method. 3. The audio data transmission method according to claim 2, wherein the encoding configuration information further comprises at least one of the following: Encoder identification information; Encoder configuration parameters; Bit rate; The encoder identification information includes at least one of the following: First identification information, used to indicate the encoding category for encoding the audio data; The second identification information is used to indicate whether the encoder includes a neural network model or not; The third identification information is used to indicate the category of the neural network model in the encoder. 4. The audio data transmission method according to claim 1, characterized in that the RTP data packet also carries at least one of the following: Serial number; Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same; Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame. 5. The audio data transmission method according to claim 4, characterized in that the decoding process of the first audio data according to the first indication information comprises: When the timestamps carried in the received multiple RTP data packets are the same, combining the first audio data carried in the multiple RTP data packets to obtain combined audio data; The combined audio data is decoded according to the first indication information. 6. The audio data transmission method according to claim 4, wherein the decoding process of the first audio data according to the first indication information comprises: In the case where the marker bit carried in the received first RTP data packet is a first value, determining at least one second RTP data packet; wherein each of the second RTP data packets has the same timestamp as that carried in the first RTP data packet; Combining the first audio data carried in a plurality of RTP data packets to obtain combined audio data; wherein the plurality of RTP data packets include: the first RTP data packet and the at least one second RTP data packet; The combined audio data is decoded according to the first indication information. 7. The audio data transmission method according to claim 6, characterized in that the step of determining at least one second RTP data packet when the marker bit carried in the received first RTP data packet is a first value comprises: In the case where the marker bit carried in the received first RTP data packet is a first value, determining a target RTP data packet; wherein the marker bit carried in the target RTP data packet is the first value, and the timestamps carried in the target RTP data packet and the first RTP data packet are different; An RTP data packet that carries a sequence number between a first sequence number and a second sequence number and carries a timestamp that is the same as the timestamp carried in the first RTP data packet is determined as the second RTP data packet; wherein the first sequence number is the sequence number carried in the first RTP data packet, and the second sequence number is the sequence number carried in the target RTP data packet. 8. The audio data transmission method according to any one of claims 5 to 7, characterized in that combining the first audio data carried in a plurality of RTP data packets to obtain the combined audio data comprises: In the case where the sequence numbers carried in the multiple RTP data packets are continuous, the first audio data carried in the multiple RTP data packets are combined in sequence according to the increasing or decreasing order of the sequence numbers carried in the multiple RTP data packets to obtain combined audio data. 9. The audio data transmission method according to any one of claims 5 to 7, characterized in that it also includes at least one of the following: In the case where the sequence numbers carried in the multiple RTP data packets are discontinuous, discarding the multiple RTP data packets and receiving the RTP data packets again; When the marker bit carried in the received RTP data packet is the second value, continue to receive the RTP data packet until the marker bit carried in the received RTP data packet is the first value or the received RTP data packet is discarded when the timer times out. 10. The audio data transmission method according to claim 1, wherein the receiving of a real-time transport protocol (RTP) data packet comprises: The RTP data packet is received through a web real-time communication WebRTC media channel. 11. An audio data transmission method, characterized in that it is applied to a sending side device, the method comprising: Obtain first audio data; wherein the first audio data is obtained by encoding based on the audio and video codec standard AVS encoding method; Encapsulating the first audio data according to the real-time transport protocol RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method; The RTP data packet is sent. 12. The audio data transmission method according to claim 11, characterized in that before obtaining the first audio data, it also includes: The encoding configuration information is determined through a media negotiation process with a receiving-side device; wherein the encoding configuration information includes encoding name information, and the encoding name information is used to indicate the AVS encoding method. 13. The audio data transmission method according to claim 12, wherein the encoding configuration information further comprises at least one of the following: Encoder identification information; Encoder configuration parameters; Bit rate; The encoder identification information includes at least one of the following: First identification information, used to indicate the encoding category for encoding the audio data; The second identification information is used to indicate whether the encoder includes a neural network model or not; The third identification information is used to indicate the category of the neural network model in the encoder. 14. The audio data transmission method according to claim 12 or 13, wherein obtaining the first audio data comprises: Get initial audio data; According to the encoding configuration information, the initial audio data is encoded using the AVS encoding method to obtain encoded audio data; The encoded audio data is divided into a plurality of the first audio data. 15. The audio data transmission method according to claim 14, characterized in that encapsulating the first audio data according to the real-time transport protocol RTP to obtain an RTP data packet comprises: Each of the first audio data is encapsulated according to RTP to obtain multiple RTP data packets; wherein each RTP data packet carries one of the first audio data and the first indication information. 16. The audio data transmission method according to claim 15, characterized in that the RTP data packet also carries at least one of the following: Serial number; Timestamp; wherein, in different RTP data packets carrying the first audio data of the same frame, the timestamp is the same; Mark bit; wherein, in the RTP data packet carrying the target audio data of each frame, the mark bit is a first value; and/or, in the RTP data packet carrying the first audio data other than the target audio data in each frame, the mark bit is a second value; the target audio data includes the first and/or last first audio data in each frame. 17. The audio data transmission method according to claim 11, wherein sending the RTP data packet comprises: The RTP data packet is sent through a Web Real-Time Communication (WebRTC) media channel. 18. An audio data transmission device, characterized in that it is applied to a receiving-side device, comprising: A receiving module, configured to receive a real-time transport protocol RTP data packet; wherein the RTP data packet carries first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is an audio and video codec standard AVS encoding method; A decoding module is used to decode the first audio data according to the first indication information. 19. An audio data transmission device, characterized in that it is applied to a sending-side device, comprising: An acquisition module is used to obtain first audio data; wherein the first audio data is obtained by encoding based on the audio and video codec standard AVS encoding method; An encapsulation module, used to encapsulate the first audio data according to the real-time transport protocol RTP to obtain an RTP data packet; wherein the RTP data packet carries the first audio data and first indication information, and the first indication information is used to indicate that the encoding method of the first audio data is the AVS encoding method; A sending module is used to send the RTP data packet. 20. An electronic device, characterized in that it comprises: a processor, a memory, and a computer program stored in the memory and executable on the processor; wherein, when the electronic device is a receiving-side device, the processor implements the steps of the audio data transmission method as described in any one of claims 1 to 10 when executing the computer program; or when the electronic device is a sending-side device, the processor implements the steps of the audio data transmission method as described in any one of claims 11 to 17 when executing the computer program. 21. A readable storage medium, characterized in that a program is stored on the readable storage medium, and when the program is executed by a processor, the steps of the audio data transmission method according to any one of claims 1 to 17 are implemented.