CN105959786A - Audio and video playing method and device - Google Patents

Abstract

The invention relates to the field of information technology, and discloses a method and device for playing audio and video. The audio and video playing method for home devices includes searching for audio playback devices from devices registered in the network; splitting audio and video data into audio data and video data; sending the audio data to the found audio playback device for playing; decoding the video data obtained by splitting, and playing the decoded video data. The present invention can avoid complex wiring between home equipment and audio playback equipment under the premise of maintaining the same playback quality as that of using signal lines, and facilitates the upgrade and expansion of the home theater system composed of home equipment and audio playback equipment; the same The audio playback device can be used by different home devices, and one home device can also use multiple audio playback devices at the same time, which increases the flexibility of use, and the whole process does not require manual plugging and unplugging of signal lines, which is convenient for users.

Description

Method and device for playing audio and video

technical field

The present invention relates to the field of information technology, in particular to an audio and video playing method and device.

Background technique

With the development of various electronic devices, people can use various devices to play multimedia content in a home environment. People usually play video and audio directly in the device. However, household devices such as tablet computers, personal computers (PCs), televisions, etc. are limited by equipment volume, cost, etc., and usually do not have dedicated audio playback equipment such as speakers, so that the audio playback effect is not ideal.

In the prior art, it is possible to use a dedicated audio playback device such as a speaker to play sound by inserting a signal cable of the audio playback device into an audio output socket of the home device. The disadvantage of this method is that the signal line is used to connect the home device and the dedicated audio playback device, which leads to the limitation of the arrangement position of the device, and the connection requires manual operation, which brings inconvenience to the user.

Contents of the invention

The object of the present invention is to provide an audio and video playing method and device to solve the above technical problems, or at least partly solve the above technical problems.

In order to achieve the above object, the present invention provides an audio and video playback method for home devices, the method comprising: searching for an audio playback device from devices registered in the network; splitting the audio and video data into audio data and video data; The audio data is sent to the found audio playback device for playback; the video data obtained by splitting is decoded, and the decoded video data is played.

Preferably, the sending the audio data to the found audio playback device for playback includes: transcoding the audio data into a unified format; sending the audio data in the unified format to the audio playback device for playback.

Preferably, the method further includes: calculating the transmission delay in the network; modifying the time stamp of the audio data according to the calculated transmission delay, and sending the modified time stamp to the audio playback device; according to the calculated transmission delay The time stamp of the video data is modified at any time, and the video data is played according to the modified time stamp.

Preferably, the method further includes: performing clock synchronization with the audio playback device.

Preferably, the method further includes: controlling video playback according to the received command; sending the received command to the audio playback device so that the audio playback device controls audio playback according to the command.

According to another aspect of the present invention, there is provided an audio playback method for an audio playback device, the method comprising: registering the audio playback device in the network; receiving audio data sent by the home device; The data is decoded, and the decoded audio data is played.

Preferably, the method further includes: performing clock synchronization with the home device.

Preferably, the method further includes: receiving a command sent by the home device; and controlling audio playback according to the received command.

According to another aspect of the present invention, there is provided an audio and video playback device for home devices, the device includes: a device search module, used to search for audio playback devices from devices registered in the network; an audio and video distribution module, It is used to split the audio and video data into audio data and video data; the audio sending module is used to send the audio data to the found audio playback device for playback; the video decoding module is used to decode the video data obtained by splitting ; The video playing module is used to play the decoded video data.

Preferably, the device further includes an audio transcoding module, the audio transcoding module is used to transcode audio data into a unified format; the audio sending module is used to send the audio data in a unified format to the audio playback device to play.

Preferably, the device further includes a delay calculation module, the delay calculation module is used to calculate the transmission delay in the network; the audio sending module is used to modify the time stamp of the audio data according to the calculated transmission delay, Send the modified timestamp to the audio playback device; the video playback module is used to modify the timestamp of the video data according to the calculated transmission delay, and play the video data according to the modified timestamp.

Preferably, the device further includes: a time synchronization module, configured to perform clock synchronization with the audio playback device.

Preferably, the device further includes a session module, the video playback module is also used to control video playback according to the received command; the session module is used to send the received command to the audio playback device to make the audio The playback device controls audio playback according to commands.

According to yet another aspect of the present invention, an audio playback device for an audio playback device is provided, the device comprising: a device registration module, configured to register the audio playback device in the network; an audio receiving module, used for The audio data sent by the home device is received; the audio decoding module is used to decode the received audio data; the audio playing module is used to play the decoded audio data.

Preferably, the device further includes: a time synchronization module, configured to perform clock synchronization with the home device.

Preferably, the device further includes: a conversation module, the conversation module is used to receive the command sent by the home device; the audio playback module is also used to control the audio playback according to the received command.

Through the above-mentioned technical scheme, search for audio playback devices from devices registered in the network; split audio and video data into audio data and video data; send audio data to the found audio playback devices for playback; The video data is decoded, and the decoded video data is played. In this way, professional equipment can be used for audio playback when playing audio and video, and the quality of sound playback can be improved; and automatic wireless connection can be made between home equipment and audio playback equipment, so that the placement of each equipment is no longer limited by signal lines. Under the premise of the same playback quality as using signal cables to connect, it avoids the complicated wiring between home equipment and audio playback equipment, and facilitates the upgrade and expansion of the home theater system composed of home equipment and audio playback equipment; the same audio playback equipment can be used Different home devices are used, and one home device can also use multiple audio playback devices at the same time, which increases the flexibility of use; and the whole process does not require manual plugging and unplugging of signal lines, which is convenient for users.

Other features and advantages of the present invention will be described in detail in the detailed description that follows.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, together with the following specific embodiments, are used to explain the present invention, but do not constitute a limitation to the present invention. In the attached picture:

FIG. 1 is a flow chart of an audio and video playback method for a home device according to an embodiment of the present invention;

Fig. 2 is the flow chart of the audio playing method for audio playing device according to the embodiment of the present invention;

Fig. 3 is a schematic diagram of an application scenario according to an embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a system according to an embodiment of the present invention;

5 is a structural diagram of an audio and video playback device for home equipment according to an embodiment of the present invention;

6 is a structural diagram of an audio and video playback device for home equipment according to an embodiment of the present invention;

7 is a structural diagram of an audio playback device for an audio playback device according to an embodiment of the present invention; and

Fig. 8 is a structural diagram of an audio playback device for an audio playback device according to an embodiment of the present invention.

detailed description

Specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

Fig. 1 is a flow chart of an audio and video playing method for a home device according to an embodiment of the present invention, the method can be used for a tablet computer, a personal computer (PC), a mobile phone, a TV and other home devices, and the home device of the present invention is usually A device that can play audio and video data. As shown in Figure 1, the method includes the following steps.

In step S110, an audio playback device is searched from devices registered in the network.

For example, in a home environment, various devices are connected using WiFi. After the audio playback device is connected to the network, it can register its own device ID and device information through the UPnP (Universal Plug and Play) protocol or the mDNS (Multicast DNS) protocol. The home device can discover the audio playback device from the WiFi network by using the UPnP protocol or the mDNS protocol.

In step S120, split the audio and video data into audio data and video data.

The home device splits the audio and video data to be played, and splits out audio data and video data.

In step S130, the audio data is sent to the found audio playback device for playback.

The home device pushes the audio data to the audio playback device. In order to avoid packet loss and reduce the complexity of packet loss processing, the audio data can be sent using the TCP protocol, which can be sent in units of frames.

In an embodiment, the sending the audio data to the found audio playback device for playback may include transcoding the audio data into a unified format, and sending the audio data in the unified format to the audio playback device for playback.

For example, the home device can select a lossy encoding method or a lossless encoding method according to the playback quality requirements, or adaptively select a lossy encoding method or a lossless encoding method according to the real-time network bandwidth, and transcode the audio data into a unified encoding format. In this way, the audio playback device can be decoded according to one encoding format, which reduces the decoding complexity of the audio playback device. After transcoding the audio data, the home device sends the audio data and the corresponding time stamp to the audio playback device.

In step S140, the video data obtained by splitting is decoded, and the decoded video data is played.

The home device decodes the split video stream to obtain a video playback frame and a corresponding time stamp, and the home device renders the video playback frame according to the time stamp and the system clock to realize playback.

By adopting the technical solution in this embodiment, professional equipment can be used for audio playback when playing audio and video, and the quality of sound playback can be improved; and automatic wireless connection can be performed between home equipment and audio playback equipment, so that the placement position of each equipment is no longer affected. Signal cable limitation, under the premise of maintaining the same playback quality as using signal cable connection, it avoids the complicated wiring between home equipment and audio playback equipment, and facilitates the upgrade and expansion of the home theater system composed of home equipment and audio playback equipment; The same audio playback device can be used by different home devices, and one home device can also use multiple audio playback devices at the same time, which increases the flexibility of use; and the whole process does not require manual plugging and unplugging of signal lines, which is convenient for users.

In an embodiment, the method in the present invention may also include calculating the transmission delay in the network; modifying the time stamp of the audio data according to the calculated transmission delay, and sending the modified time stamp to the audio playback device; The calculated transmission delay modifies the time stamp of the video data, and plays the video data according to the modified time stamp.

For example, the home device may use a moving window weighted average algorithm, a polynomial fitting algorithm or any other known algorithm to calculate the transmission delay between the home device and the audio playback device at the next moment. For example, the transmission delay at the next moment is calculated as Δt. The home device delays the time stamp of the video by Δt, and delays the time stamp of the audio by Δt, the home device plays the video data according to the time stamp of the delayed video, and sends the time stamp of the delayed audio to the audio playback device, and the audio plays The device plays the audio data according to the delayed timestamp. In this way, the audio data and the video data are played according to the delayed time stamp, which ensures that the video playback is synchronized with the audio playback.

In an embodiment, the method of the present invention may further include performing clock synchronization with an audio playback device.

For example, clock synchronization is periodically performed between the home device and the audio playback device to ensure that the time of the system clocks of the home device and the audio playback device are consistent. When the clock is synchronized, the home device can be used as the server, and the audio playback device can be used as the client, and when the clock is synchronized, the home device can modify the current time information sent for synchronization according to the estimated transmission delay. For example, the current time is T, and the home device modifies the current time in the synchronization information to T+Δt, so as to further improve the accuracy of clock synchronization.

In an embodiment, the method of the present invention may further include controlling video playback according to the received command; sending the received command to the audio playback device so that the audio playback device controls audio playback according to the command.

For example, the home device can receive commands such as play, pause, fast forward, fast rewind, multiple play, stop, etc. from the remote control or the interface. The home device performs corresponding control on video playback according to the received command. A session is established between the home device and the audio playback device, which is also called signaling interaction, and commands are transmitted to the audio playback device through the session. In addition, information such as device parameters and heartbeat can be exchanged between the home device and the audio playback device through a session.

The audio and video playing method in the present invention is described above from the perspective of the home device, and the audio playing method in the present invention is described below from the perspective of the audio playing device.

FIG. 2 is a flow chart of an audio playback method for an audio playback device according to an embodiment of the present invention. The method can be used in an audio playback device such as a sound box. As shown in FIG. 2 , the method includes the following steps.

In step S210, the audio playback device is registered in the network.

For example, in a home environment, devices can be connected using WiFi. After the audio playback device is connected to the network, it can register its own device ID and device information through the UPnP (Universal Plug and Play) protocol or mDNS (Multicast DNS) protocol, so that the home device can discover the audio playback device.

In step S220, audio data sent by the home device is received.

The audio playback device receives the audio data sent by the home device. For example, audio data is received from a home device over a TCP connection.

In step S230, the received audio data is decoded, and the decoded audio data is played.

The audio playback device decodes the received audio data, for example, decodes to obtain an audio frame and a corresponding time stamp, and then plays the audio frame according to the current time and time stamp of the system clock.

In an embodiment, the method of the present invention may further include performing clock synchronization with the home device.

For example, clock synchronization is periodically performed between the home device and the audio playback device to ensure that the time of the system clocks of the home device and the audio playback device are consistent. When the clock is synchronized, the home device can be used as the server, and the audio playback device can be used as the client, and when the clock is synchronized, the home device can modify the current time information sent for synchronization according to the calculated transmission delay. For example, the current time is T, and the home device modifies the current time in the synchronization information to be T+Δt. The audio playback device adjusts the local clock according to the current time in the received synchronization information, so that the local clock is synchronized with the clock of the home device.

In an embodiment, the method of the present invention may further include receiving a command sent by the home device; and controlling audio playback according to the received command.

For example, establishing a session between an audio playback device and a home device is also referred to as signaling interaction, and receiving commands from the home device through the session. The command may be a play, pause, fast forward, fast rewind, multiple play, stop and other commands received by the home device from the remote controller or the interface. The audio playback device performs corresponding control on audio playback according to the received command. In addition, information such as device parameters and heartbeat can be exchanged between the audio playback device and the home device through a session.

In this way, in the above-mentioned embodiment, the audio playback device can establish a connection with the home device, and play the audio data sent by the home device, so that the placement of the audio playback device is no longer limited by the signal line connection, and different home devices can use the same As for the audio playback device, one home device can also use multiple audio playback devices at the same time, which increases the flexibility of use.

The method for playing audio and video by a TV using multiple speakers in a WiFi network environment is described below as an example. As shown in Figure 3, the TV is connected to multiple speakers through a WiFi network. As shown in Figure 4, the home device includes: a device discovery module, a session module, an audio and video stream module, a video decoding module, an audio transcoding module, a delay calculation module, a video playback module, an audio sending module, and a time synchronization module; the speaker includes Device registration module, session module, audio receiving module, audio decoding module, audio playing module and time synchronization module. The signaling and data interaction between modules are shown in Figure 4.

The device registration module uses UPnP or mDNS protocol to register the device ID and device information of the speaker to the local area network. The device discovery module uses UPnP or mDNS protocol to discover each speaker from the network. A session is established between the session module of the TV and the session modules of each speaker, and the device information and heartbeat are transmitted. After starting to play, the conversation module of the TV transmits the play command to the conversation module of the sound box and its own audio and video distribution module respectively through signaling. The conversation module of the sound box notifies other modules to prepare for playback processing through signaling. The audio and video splitting module splits the audio and video data, and transmits the video stream VES to the video decoding module, and transmits the audio stream AES to the audio transcoding module. The audio transcoding module decodes audio data, encodes it into a unified audio format, selects lossy or lossless encoding methods according to quality requirements, or adaptively selects lossy or lossless encoding methods according to real-time bandwidth conditions, and converts audio data into AES and The time stamp PTS is sent to the audio sending module. The video decoding module decodes the video stream, and transmits the decoded video data YUV and time stamp PTS to the video playing module. The delay calculation module calculates the transmission delay between the TV and the speaker, and can use algorithms such as moving window weighted average and polynomial fitting to estimate the transmission delay Δt at the next moment. The audio transmission module modifies the timestamp of the audio according to the delay Δt calculated by the delay calculation module (for example, delaying the timestamp of the audio by Δt), and pushes the transcoded audio data and timestamp to each speaker, in order to avoid packet loss and Reduce the complexity of packet loss processing, and use TCP protocol to send audio data in units of frames with time stamps. The video playback module modifies the timestamp of the video according to the delay Δt calculated by the delay calculation module (for example, delaying the timestamp of the video by Δt), and renders the video frame data according to the timestamp and the current system time for video playback. The audio data receiving module receives the audio frame data and the time stamp PTS sent by the audio sending module. The audio decoding module decodes the received audio data AES, and transmits the decoded audio data PCM and time stamp PTS to the audio playing module. The audio playback module plays audio frame data according to the timestamp and the current system time. The time synchronization module between the TV and the sound box synchronizes the system time, and the TV can use the transmission delay calculated by the delay calculation module to further improve the synchronization accuracy during time synchronization.

5 is a structural diagram of an audio and video playback device for home equipment according to an embodiment of the present invention. The device can be used for home equipment such as tablet computers, personal computers (PCs), mobile phones, and televisions. It is a device that can play audio and video data. As shown in Figure 5, the device includes the following modules.

A device search module 510, configured to search for an audio playback device from devices registered in the network;

Audio and video splitting module 520, for splitting audio and video data into audio data and video data;

Audio sending module 530, for sending audio data to the found audio playback device to play;

A video decoding module 540, configured to decode the video data obtained by splitting;

The video playing module 550 is used to play the decoded video data.

In one embodiment, as shown in FIG. 6 , the device may further include an audio transcoding module 602, the audio transcoding module 602 is used to transcode audio data into a unified format; the audio sending module 530 is used to convert the audio data into a unified format The audio data is sent to the audio playback device for playback.

In one embodiment, as shown in FIG. 6, the device may further include a delay calculation module 604, which is used to calculate the transmission delay in the network; the audio sending module 530 is used to calculate the transmission delay according to the calculation Modify the timestamp of the audio data, and send the modified timestamp to the audio playback device; the video playback module 540 is used to modify the timestamp of the video data according to the calculated transmission delay, and play the video data according to the modified timestamp.

In an embodiment, as shown in FIG. 6 , the device may further include a time synchronization module 606 for performing clock synchronization with the audio playback device.

In one embodiment, as shown in FIG. 6 , the device may also have a session module 608 . The video playing module 540 is also used to control the video playing according to the received command; the session module 608 is used to send the received command to the audio playing device so that the audio playing device controls the audio playing according to the command.

The above-mentioned method corresponds to the above-mentioned device, and its exemplary description can refer to the detailed description in the method, and details are not repeated here.

Fig. 7 is a structural diagram of an audio playback device for an audio playback device according to an embodiment of the present invention; as shown in Fig. 7, the device may include the following modules.

A device registration module 710, configured to register the audio playback device in the network.

An audio receiving module 720, configured to receive audio data sent by the home device;

An audio decoding module 730, configured to decode received audio data;

The audio playing module 740 is configured to play the decoded audio data.

In an embodiment, as shown in FIG. 8 , the device may further include a time synchronization module 802 configured to perform clock synchronization with the home device.

In an embodiment, as shown in FIG. 8, the device may further include a conversation module 804, the conversation module 804 is used to receive commands sent by home devices; the audio playback module 730 is also used to control audio playback according to the received commands .

The above-mentioned method corresponds to the above-mentioned device, and its exemplary description can refer to the detailed description in the method, and details are not repeated here.

The preferred embodiment of the present invention has been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the specific details of the above embodiment, within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, These simple modifications all belong to the protection scope of the present invention.

In addition, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable way if there is no contradiction. The combination method will not be described separately.

In addition, various combinations of different embodiments of the present invention can also be combined arbitrarily, as long as they do not violate the idea of the present invention, they should also be regarded as the disclosed content of the present invention.

Claims (16)

1., for an audio and video playing method for household equipment, the method includes:

Audio-frequence player device is searched from the equipment registered in a network；

Audio, video data is split into voice data and video data；

The audio-frequence player device being sent to find by voice data plays out；

The described video data obtaining shunting is decoded, and broadcasts the video data of decoding gained Put.

Method the most according to claim 1, it is characterised in that described voice data is sent to The audio-frequence player device found plays out and includes:

It is consolidation form by voice data transcoding；

The voice data of consolidation form is sent to described audio-frequence player device play out.

Method the most according to claim 1, it is characterised in that described method also includes:

Transmission delay in network is calculated；

According to the timestamp of the transmission delay amendment voice data calculated, amended timestamp is sent to Described audio-frequence player device；

According to the timestamp of the transmission delay amendment video data calculated, according to amended timestamp to regarding Frequency is according to playing out.

Method the most according to claim 1, it is characterised in that described method also includes:

Clock is carried out synchronization with described audio-frequence player device.

Method the most according to claim 1, it is characterised in that described method also includes:

Video playback is controlled by the order according to receiving；

The order received is sent to audio-frequence player device makes described audio-frequence player device right according to order Audio frequency is play and is controlled.

6., for an audio frequency playing method for audio-frequence player device, the method includes:

In a network described audio-frequence player device is registered；

Receive the voice data that household equipment sends；

The voice data received is decoded, the voice data of decoding is played out.

Method the most according to claim 6, it is characterised in that described method also includes:

Clock is carried out synchronization with described household equipment.

Method the most according to claim 6, it is characterised in that described method also includes:

Receive the order that household equipment sends；

According to the order received, audio frequency broadcasting is controlled.

9., for an audio and video display device for household equipment, this device includes:

Equipment searches module, for searching audio-frequence player device from the equipment registered in a network；

Audio frequency and video diverter module, for being split into voice data and video data by audio, video data；

Audio frequency sending module, broadcasts for the audio-frequence player device being sent to find by voice data Put；

Video decoding module, is decoded for the described video data obtaining shunting；

Video playback module, for playing out the video data of decoding gained.

Device the most according to claim 9, it is characterised in that described device also includes audio conversion Code module,

It is consolidation form that described audio frequency transcoding module is used for voice data transcoding；

Described audio frequency sending module sets for the voice data of consolidation form is sent to the broadcasting of described audio frequency For playing out.

11. devices according to claim 9, it is characterised in that described device also includes time delay meter Calculate module,

Described time delay computing module is for calculating transmission delay in network；

Described audio frequency sending module, will for the timestamp according to the transmission delay amendment voice data calculated Amended timestamp is sent to described audio-frequence player device；

Described video playback module is for the timestamp according to the transmission delay amendment video data calculated, root According to amended timestamp, video data is played out.

12. devices according to claim 9, it is characterised in that described device also includes:

Time synchronized module, synchronization for carrying out clock with described audio-frequence player device.

13. devices according to claim 9, it is characterised in that described device also includes session mould Block,

Video playback is controlled by the order that described video playback module is additionally operable to according to receiving；

Described conversation module makes described audio frequency broadcast for the order received is sent to audio-frequence player device Audio frequency broadcasting is controlled by equipment of putting according to order.

14. 1 kinds of audio playing apparatus for audio-frequence player device, this device includes:

Facility registration module, for registering described audio-frequence player device in a network；

Audio frequency receiver module, for receiving the voice data that household equipment sends；

Audio decoder module, for being decoded the voice data received；

Audio playing module, for playing out the voice data of decoding.

15. devices according to claim 14, it is characterised in that described device also includes:

Time synchronized module, synchronization for carrying out clock with described household equipment.

16. devices according to claim 14, it is characterised in that described device also includes: meeting Words module,

Described conversation module is for receiving the order that household equipment sends；

Described audio playing module is additionally operable to be controlled audio frequency broadcasting according to the order received.