CN105915493A - Audio and video real-time transmission method and device and audio and video real-time playing method and device - Google Patents

Abstract

An embodiment of the present invention provides a method for real-time audio and video transmission, including: storing the grouped video original stream and the grouped audio original stream as program slices after multiplexing; encrypting the program-specific information, and storing it as a program-specific information slice; The program specific information and the program slices are transmitted. Embodiments of the present invention also provide a real-time audio and video transmission method, and a real-time audio and video playback method and device. The embodiment of the present invention can reduce the calculation amount of encryption and decryption in the prior art.

Description

Audio and video real-time transmission method and device, audio and video real-time playback method and device

technical field

Embodiments of the present invention relate to the field of audio and video technologies, and in particular, to a method and device for real-time transmission of audio and video, and a method and device for real-time playback of audio and video.

Background technique

The HLS (HTTP Live Streaming) protocol is an extended HTTP protocol developed by Apple and used in webcast and on-demand scenarios. The protocol uses Transport Stream (TS) to encapsulate data. Each transport stream carries data information such as video (Video), audio (Audio), and stream index information such as program association table (Program Association Table, PAT) and program mapping table (Program Map Table, PMT).

The HLS protocol supports the encryption of the slices of the transport stream. The encryption scheme in the prior art is to encrypt each slice. Therefore, the calculation amount of this encryption method is relatively large, and the calculation amount of the receiving end will be corresponding when decrypting. increase. Therefore, if the encryption scheme in the prior art needs to guarantee the quality of transmission, it either needs the support of a separate hardware encryption and decryption device, or requires a strong processing capability of the CPU.

Contents of the invention

Embodiments of the present invention provide a method and device for real-time transmission of audio and video, and a method and device for real-time playback of audio and video, which are used to solve the technical problem of excessive computational complexity in encryption and decryption in the prior art.

An embodiment of the present invention provides a method for real-time transmission of audio and video, including:

Multiplexing the grouped original video streams and grouped audio original streams and storing them as program slices;

Encrypt program-specific information and store it as program-specific information slices;

The program specific information and the program slices are transmitted.

Further, said encrypting the program-specific information includes encrypting the program association table and the program mapping table.

Further, before the step of multiplexing the packetized video original stream and the packetized audio original stream and storing them as program slices, the step further includes:

A step of packaging the original video stream and the original audio stream into packetized video original streams and packetized audio original streams respectively.

Further, before the step of packaging the original video stream and the original audio stream into packetized video original streams and packetized audio original streams, the step also includes:

A step of encoding the video data and the video data respectively and forming the video original stream and the audio original stream.

The present invention also provides a real-time audio and video playback method, including:

Download encrypted program-specific information slices;

Decrypting the encrypted program-specific information slices to obtain decrypted program-specific information;

Downloading program slices, and parsing the program slices according to the decrypted program specific information.

Further, the encrypted program-specific information slice includes an encrypted program association table and an encrypted program mapping table.

Further, the decrypting the encrypted program-specific information slice includes: decrypting the encrypted program association table and the encrypted program mapping table.

The present invention provides an audio and video real-time transmission device, comprising:

An encryption module, configured to encrypt program-specific information and store it as a program-specific information slice;

a multiplexer, configured to multiplex the packetized original video stream and the packetized audio original stream and store them as program slices;

A transmission module, configured to transmit the program specific information and the program slice.

Further, said encrypting the program-specific information includes encrypting the program association table and the program mapping table.

Further, it also includes video grouper and audio grouper:

The video packetizer is used to pack the video original stream into a grouped video original stream;

The audio packetizer is configured to packetize the audio original stream into grouped audio original streams.

Further, it also includes video encoder and audio encoder:

The video encoder is configured to encode video data to form the original video stream;

The audio encoder is configured to encode audio data to form the audio original stream.

The present invention provides an audio and video real-time playback device, comprising:

A first download module, configured to download encrypted program-specific information slices;

A decryption module, configured to decrypt the encrypted program-specific information slices to obtain decrypted program-specific information;

The second downloading module is configured to download program slices, and analyze the program slices according to the decrypted program specific information.

Further, the encrypted program-specific information slice includes an encrypted program association table and an encrypted program mapping table.

Further, the decryption module is also used to decrypt the encrypted program association table and the encrypted program mapping table.

The audio and video real-time transmission method and device, audio and video real-time playback method and device provided by the embodiments of the present invention do not need to encrypt and decrypt each slice, and can simultaneously reduce the amount of encryption calculations at the transmission end and the decryption calculation amount at the receiving end. .

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are For some embodiments of the present invention, those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of the format of a slice of a transport stream in the prior art;

2 is a schematic diagram of the format of a program slice of a method for real-time audio and video transmission according to an embodiment of the present invention;

3 is a schematic diagram of the format of a program-specific information slice of a method for real-time audio and video transmission according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the composition of sliced packets of an audio and video real-time transmission method according to an embodiment of the present invention;

5 is a flowchart of a method for real-time audio and video transmission according to an embodiment of the present invention;

Fig. 6 is the flow chart of a kind of audio and video real-time playing method of the embodiment of the present invention;

7 is a schematic structural diagram of an audio and video real-time transmission device according to an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of an audio and video real-time playback device according to an embodiment of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

In order to better illustrate the technical solutions of the embodiments of the present invention and the beneficial effects produced by them, the present invention will first introduce the format of slices of transport streams in the prior art.

FIG. 1 is a schematic diagram of the format of a transport stream slice in the prior art.

The transport stream is a hierarchical encapsulation format. It is necessary to find the program association table first to obtain the number of program mapping tables in the transport stream, so as to obtain the number of programs in the transport stream. Each program mapping table corresponds to a program, and then The video packet identifier (Packet Identification, PID) and audio packet identifier of the program are acquired according to the program mapping table.

According to the HLS protocol, a long program is usually divided into multiple 10-second slices. Therefore, in the slice of the transport stream in the prior art, in order to facilitate the user to receive the transport stream at any time and ensure that the audio and video data in the slice are not pirated, the transmission end periodically inserts the program association table and the program mapping table into the transport stream slices, and each slice is encrypted; thus the slice of the transport stream in the prior art includes at least two encrypted program association tables and program mapping tables, and the receiving end also needs to receive the slice after receiving the slice Decrypt all program association tables and program mapping tables in each slice.

For example, in Figure 1, a program is cut into n slices, user A can receive it from program association table 1, and user B can receive it from program association table n to find the program mapping table nearby, so as to facilitate live broadcast; The terminal encrypts the program association tables 1 to n and the program mapping tables 1 to n during the transmission process, so the receiving end also needs to decrypt the encrypted program association tables 1 to n and program mapping tables 1 to n . Therefore, the prior art greatly increases the redundant data in the transport stream.

After a long period of research, the inventors of the present invention found that in each slice, it is not necessary to periodically insert the program association table and the program mapping table, but only need to package and encrypt the program association table and the program mapping table separately, then The effect of not only maintaining compatibility with the transport stream in the prior art, but also reducing redundant data in the transport stream can be achieved.

The format of the slice of the transport stream of the present invention will be described below.

FIG. 2 is a schematic diagram of the format of a program slice in a method for real-time audio and video transmission according to an embodiment of the present invention.

The transport stream in the embodiment of the present invention is a packet-based stream, and each group of transport streams is divided into several slices. It should be noted that only the format of one slice is shown in FIG. 3 , and the formats of other slices are actually the same as The slice format in Figure 2 is similar.

The program slices in the embodiment of the present invention include package 1, package 2...packet n-1, package n, and each program slice does not include a program association table and a program mapping table.

Fig. 3 is a schematic diagram of the format of program-specific information slices in a method for real-time audio and video transmission according to an embodiment of the present invention.

The program-specific information slice in this embodiment only includes two pieces of program-specific information, the encrypted program association table and the program mapping table, but does not include video data and audio data. A program specific information slice may be stored as a file named init.ts.

FIG. 4 is a schematic diagram of the packet composition of a slice in a method for real-time audio and video transmission according to an embodiment of the present invention.

The slices shown in this embodiment can be program-specific information slices or program slices, each slice includes a packet header and data, and each packet is 188 bytes or 204 bytes, of which 204 bytes The packet is formed by adding 16 bytes of CRC check data after the 188-byte packet.

The packet header may also include an extended self-use area, the length of the packet header occupies 4 bytes, and the self-use area and packet data occupy a total of 184 bytes. The packet header consists of 8 parts: synchronization byte, transmission error code indicator, payload unit start indicator, transmission priority, packet identifier, transmission scrambling control, adaptive area control and continuous counter.

Among them, the packet restriction in the data stream can be detected through the automatic correlation characteristic of the bit string of the synchronization byte, and packet synchronization can be established; the transmission error indicator means that when there is an error code that cannot be eliminated, the error correction decoder can be used Indicates a 1-bit error, but the error cannot be corrected; the payload unit start indicator indicates whether the packet has definite start information; the transmission priority is used to assign priority to the packet; the packet identifier value is determined by the user Determined, the decoder distinguishes the packets of different elementary streams (ElementaryStream, ES) according to the packet identifier to reconstruct the original stream; the transmission scrambling control can be used to indicate whether the packet content is scrambled; the adaptive area control is used to indicate whether there is Adaptive area, usually represented by 2 bits, for example, "01" indicates that there is useful information but no adaptive area, "10" indicates that there is no useful information and there is adaptive area, "11" indicates that there is useful information and there is automatic Adaptation area, "00" means no definition; the continuous counter can count the transmission sequence of the packet identifier, and according to the reading of the continuous counter, the receiving end can judge whether there is a packet loss or a packet transmission sequence error. All in all, the packet header has the functions of synchronization, identification, error detection and encryption for the transport stream.

In addition, the packet identifier is a unique identifier in the transport stream, and the data content in the packet is determined by the packet identifier. If the packet identifier in the header of a packet in a transport stream is 0, then the data content of the packet is the program association table, if not 0, it is video data, audio data, program mapping table, or other types of data.

Fig. 5 is a flow chart of an audio and video real-time transmission method according to an embodiment of the present invention.

The real-time transmission of audio and video includes real-time transmission of video data and real-time transmission of audio data, so it is necessary to package the video original stream and audio original stream into packetized video original stream and packetized audio original stream respectively; The original stream and the grouped audio original stream are multiplexed and stored as program slices; in addition, this embodiment also needs to encrypt and package the program association table and program mapping table respectively, as follows:

The specific steps of packaging the original video stream of the packet are as follows:

S101: Encode the video data to form an original video stream (Elementary Stream, ES);

The original video stream is the original basic bit stream output by the encoder, which only contains the information necessary for the decoder and is close to the original image. The encoded original video stream may be in MPEG-2 or MPEG-4 format, and may also be in H.264 format or other formats, which is not limited in the present invention.

S102: Pack the original video stream into a packetized original video stream (Packet Elemental Stream, PES);

The grouped original video stream exists in the form of divided data packets, mainly adding time stamps and other description information to the data frame on the original video stream. The grouped original video stream provides standard packet headers and packaging methods, and provides decoding time sign.

The specific steps of packaging the packetized audio original stream are as follows:

S201: Encode the audio data to form an audio original stream;

If the digital audio signal is directly transmitted without compression, it will occupy a huge bandwidth. Therefore, it is necessary to use audio compression technology to process the audio data in order to effectively transmit the audio data. The audio coding method in this step may be MPEG-1 audio coding, or MPEG-2 AAC (Advanced Audio Coding) coding, which is not limited in the present invention.

S202: Pack the original video stream into packetized audio original streams.

The audio packetizer 202 divides the audio original stream into segments or packs them into groups, and adds corresponding header files to pack into grouped audio original streams. The packets of the audio original stream may be discontinuous.

After packing the original video stream and the original audio stream of the packet, enter step S300;

S300: Multiplex the packetized original video stream and the packetized audio original stream and store them as program slices.

S400 is the packaging of program-specific information slices, specifically: encrypting program-specific information and storing it as program-specific information slices. Encrypting the program specific information includes encrypting the program association table and the program mapping table.

Finally, enter step S500: transmit program specific information and program slices.

FIG. 6 is a flowchart of a method for real-time audio and video playback according to an embodiment of the present invention.

S301: Download encrypted program-specific information slices; the encrypted program-specific information slices include encrypted program association tables and encrypted program mapping tables.

S302: Decrypt the encrypted program-specific information slice to obtain decrypted program-specific information; specifically, decrypting the encrypted program-specific information slice includes: decrypting an encrypted program association table and an encrypted program mapping table.

S303: Download the program slice, and analyze the program slice according to the decrypted program specific information. The analysis process is specifically:

Firstly, the program association table is analyzed to obtain the package identifier of the program mapping table; in this embodiment, the program association table whose package identifier is 0 is first analyzed to obtain the package identifier of the program mapping table.

Then, the program mapping table is searched according to the packet identifier of the mapping table; in this embodiment, the number of program mapping tables is equal to the number of programs.

Secondly, search the packet identifier of the video data and the packet identifier of the audio data according to the program mapping table; in this embodiment, the packet identifier of the video data and the packet identifier of the audio data are searched to obtain the video data and the audio data.

Finally, the video data and the audio data are analyzed according to the packet identifier of the video data and the packet identifier of the audio data.

The above parsing process is completed by the receiving end, which is usually a consumer electronic product, such as a smart TV, a set-top box, a projector, a video player, a personal computer, a smart phone, a tablet computer, etc.

FIG. 7 is a schematic structural diagram of an audio and video real-time transmission device according to an embodiment of the present invention.

The real-time audio and video transmission device 10 of the embodiment of the present invention includes: a video encoder 101 , an audio encoder 201 , a video packetizer 102 , an audio packetizer 202 , a multiplexer 300 , an encryption module 400 , and a transmission module 500 .

Since the collected video data and audio data are all analog data, sampling, quantization and compression encoding are required to form the original video stream and the original audio stream. The original video stream and the original audio stream are unsegmented continuous code streams.

The video encoder 101 is used to encode video data to form an original video stream. In this embodiment, the original video stream includes multiple access units, and each access unit is the encoded data of an image. The video coding method of the present invention may adopt transform coding, entropy coding, motion estimation and motion compensation, or a mixed coding method for coding, which is not limited in the present invention.

The audio encoder 201 is used for respectively encoding audio data to form an audio original stream. In this embodiment, if the digital audio signal is directly transmitted without compression, it will occupy a huge bandwidth. Therefore, it is necessary to use audio compression technology to process the audio data in order to effectively transmit the audio data. Digital audio compression coding compresses the audio data signal as much as possible on the premise of ensuring that the signal does not produce distortion in the auditory sense. Digital audio compression coding is realized by removing redundant components in the sound signal. The audio coding method of the present invention may be waveform coding, parameter coding, or mixed coding, which is not limited in the present invention.

The video packetizer 102 is used for packetizing the video original stream into grouped video original streams. In detail, the video packetizer 102 divides the original video stream into segments or packs it into groups, and adds corresponding header files to pack the original video stream into groups, and the packets of the original video stream may be discontinuous of.

The audio packetizer 202 is used to packetize the audio original stream into grouped audio original streams. In detail, the audio packetizer 202 divides the audio original stream into segments or packs them into groups, and adds corresponding header files to pack into grouped audio original streams. The packets of the audio original stream may be discontinuous of.

The multiplexer 300 is used to multiplex the packetized original video stream and the packetized audio original stream and store them as program slices, and transmit program specific information and program slices. In this embodiment, the multiplexer 300 packs the program association table, program mapping table, packetized video original stream, and packetized audio original stream into slices with a fixed length of 188 bytes during transmission.

The encryption module 400 is used to encrypt program specific information and store it as a program specific information slice. In this embodiment, program-specific information is encrypted, including program association table and program mapping table.

The transmission module 500 is used for transmitting program specific information and program slices.

The audio and video real-time transmission device 100 provided by the embodiment of the present invention can be specifically applied to electronic equipment in practical applications, and related functional modules can be implemented by hardware processors. The electronic equipment is usually various types of video servers.

FIG. 8 is a schematic structural diagram of an audio and video real-time playback device according to an embodiment of the present invention.

An audio and video real-time playback device 400 according to an embodiment of the present invention includes a first download module 401 , a decryption module 402 , and a second download module 403 .

The first download module 401 is used for downloading the encrypted program-specific information slice; the encrypted program-specific information slice includes an encrypted program association table and an encrypted program mapping table.

The decryption module 402 is used to decrypt the encrypted program-specific information slice to obtain the decrypted program-specific information; since the encrypted program-specific information slice includes an encrypted program association table and an encrypted program mapping table, the decryption module 402 also It is used to decrypt the encrypted program association table and the encrypted program mapping table.

The second downloading module 403 is used for downloading the program slice, and analyzing the program slice according to the decrypted program specific information.

Through the above-mentioned technical solution of the embodiment of the present invention, it is not necessary to periodically insert the program association table and the program mapping table, but only need to package and encrypt the program association table and the program mapping table separately, so as to maintain the same level as the prior art The transport streams in the transport stream are compatible, and at the same time reduce the effect of redundant data in the transport stream.

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative efforts.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (14)

1. an audio frequency and video live transmission method, it is characterised in that including:

Program section will be stored as after the video primary flow of packet and the audio frequency primary flow multiplexing of packet；

Program Specific Information is encrypted, and is stored as Program Specific Information section；

Transmit described Program Specific Information and the section of described program.

Method the most according to claim 1, it is characterised in that

Described be encrypted Program Specific Information includes: encrypt Program Association Table and Program Map Table.

Method the most according to claim 2, it is characterised in that the described video primary flow by packet And also include before being stored as the step of program section after the audio frequency primary flow multiplexing of packet:

Video primary flow and audio frequency primary flow are packaged as respectively video primary flow and the audio frequency of packet of packet The step of primary flow.

Method the most according to claim 3, it is characterised in that described by video primary flow and audio frequency Also wrap before the step of the audio frequency primary flow that primary flow is packaged as video primary flow and the packet being grouped respectively Include:

Video data and video data are separately encoded and are formed described video primary flow and described audio frequency is original The step of stream.

5. an audio frequency and video real time playing method, it is characterised in that including:

Download the Program Specific Information section after encryption；

Decipher the section of the Program Specific Information after described encryption, the Program Specific Information after being deciphered；

Program downloading is cut into slices, and resolves the section of described program according to the Program Specific Information after described deciphering.

Method the most according to claim 5, it is characterised in that

Program Specific Information section after described encryption includes the Program Association Table after encryption and the joint after encryption Mesh mapping table.

Method the most according to claim 6, it is characterised in that the joint after the described encryption of described deciphering The section of mesh customizing messages includes:

Decipher the Program Association Table after described encryption and the Program Map Table after encryption.

8. an audio frequency and video real-time Transmission device, it is characterised in that including:

Encrypting module, for being encrypted Program Specific Information, and is stored as Program Specific Information section；

Multiplexer, for being stored as after the video primary flow of packet and the audio frequency primary flow multiplexing of packet Program is cut into slices；

Transport module, is used for transmitting described Program Specific Information and the section of described program.

Device the most according to claim 8, it is characterised in that

Described be encrypted Program Specific Information includes: encrypt Program Association Table and Program Map Table.

Device the most according to claim 9, it is characterised in that also include video packetizer and sound Frequently burster:

Described video packetizer, for being packaged as the video primary flow of packet by video primary flow；

Described audio packet device, for being packaged as the audio frequency primary flow of packet by audio frequency primary flow.

11. devices according to claim 10, it is characterised in that also include video encoder and sound Frequently encoder:

Described video encoder, for forming described video primary flow by video data encoding；

Described audio coder, for forming described audio frequency primary flow by audio data coding.

12. 1 kinds of audio frequency and video real-time play devices, it is characterised in that including:

First download module, the Program Specific Information section after downloading encryption；

Deciphering module, the Program Specific Information section after deciphering described encryption, the joint after being deciphered Mesh customizing messages；

Second download module, cuts into slices for program downloading, and according to the Program Specific Information after described deciphering Resolve the section of described program.

13. devices according to claim 12, it is characterised in that

Program Specific Information section after described encryption includes: after Program Association Table after encryption and encryption Program Map Table.

14. devices according to claim 13, it is characterised in that described deciphering module is additionally operable to solve Program Association Table after close described encryption and the Program Map Table after encryption.