CN115278323A - Display device, intelligent device and data processing method - Google Patents

Abstract

The present disclosure relates to a display device, an intelligent device, and a data processing method, the display device including: a communicator configured to: communicating with a smart device; a controller configured to: receiving target length data of preset byte number in a target data packet sent by intelligent equipment, wherein the target data packet comprises: the target length data is used for representing the number of target bytes corresponding to the target media data, and the target media data is any one of the following items: the intelligent device comprises video data, audio data and audio-video composite data, wherein target media data are obtained by encoding collected target source data through the intelligent device; receiving data of the number of target bytes sent by the intelligent equipment based on the target length data to obtain target media data; decoding the target media data to obtain target source data; and playing the target source data. The time delay of the mirror image screen projection can be reduced, and the data processing efficiency during the screen projection is improved.

Description

A display device, smart device and data processing method

technical field

The embodiment of the present application relates to mirroring screen projection technology. More specifically, it relates to a display device, a smart device and a data processing method.

Background technique

The existing mirror projection method transmits data to the projected terminal at the projecting terminal, and when the projected terminal receives the start flag bit of the next frame of data, it determines that the current frame of data has been received, and starts from the already received After the current frame data is separated from the data, it is decoded and processed by the decoder and played.

Therefore, it is necessary to wait for one more frame of data to arrive at the projected end each time, resulting in a large processing delay of the projected data.

Contents of the invention

In order to solve the above-mentioned technical problems or at least partially solve the above-mentioned technical problems, the present application provides a display device, a smart device and a data processing method, which can reduce the time delay of mirror projection and improve the data processing efficiency during screen projection.

In the first aspect, the embodiment of the present application provides a display device, which includes: a communicator configured to: communicate with a smart device; a controller configured to: receive the target data packet sent by the smart device Target length data with a preset number of bytes, the target data packet includes: target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is any of the following: Video data, audio data, audio-video composite data, the target media data is obtained by encoding and processing the target source data collected by the smart device; based on the target length data, the data of the target number of bytes sent by the smart device is received to obtain the target Media data; decode the target media data to obtain the target source data; play the target source data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: flag bits, the flag bits are used to indicate the type of target media data, the type is audio type or video type, and the number of target bytes It is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits; the controller is specifically configured to: receive the target length data sent by the smart device through the mixed data channel; based on the target length data , through the mixed data channel, receive the data of the target number of bytes sent by the smart device to obtain the flag bit and the target media data; when the flag bit indicates that the type of the target media data is an audio type, perform audio on the target media data The target source data is obtained through decoding processing; when the flag bit indicates that the type of the target media data is a video type, video decoding processing is performed on the target media data to obtain the target source data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a target timestamp, the target timestamp is used to indicate the collection time of the target source data, and the target byte number is specified by the target media data The sum of the number of bytes occupied by the target time stamp and the number of bytes occupied by the target timestamp; the controller is specifically configured to: receive the target length data sent by the smart device through the target data channel, and when the target media data is audio data In this case, the target data channel is an audio data channel, and when the target media data is video data, the target data channel is a video data channel; based on the target length data, receive the target number of bytes sent by the smart device through the target data channel data to obtain the target timestamp and target media data; based on the target timestamp, the target source data and the first source data are played synchronously, and the target source data and the first source data are different types of media data synchronously collected by smart devices.

In some embodiments of the present application, the target media data is video data, and the target data packet also includes: a target serial number, which is used to indicate the collection sequence of the target source data; the controller is also configured to: When the acquisition sequence indicated by the target serial number sent by the device satisfies the sequence condition, the target length data of the preset number of bytes sent by the smart device is received. The sequence condition is used to indicate that the target source data and the last source data received are The data continuously collected by the smart device; when the collection sequence indicated by the target serial number does not meet the sequence condition, a request message is sent to the smart device, and the request message is used to request that the next source data to be sent be encoded in the form of a key frame And send; receive the next data packet sent by the smart device, and the media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame.

In a second aspect, an embodiment of the present application provides a smart device, which includes: a communicator configured to: communicate with a display device; a controller configured to: send a target data packet to the display device, the target data The package includes: target length data and target media data with a preset number of bytes. The target length data is used to represent the target number of bytes corresponding to the target media data. The target media data is any of the following: video data, audio data, Audio and video composite data, the target media data is obtained by encoding and processing the collected target source data by the smart device; among them, the target data packet is used to display the target length data received by the device, and based on the target length data, receive the target word sent by the smart device The data of the number of sections to obtain the target media data, and decode the target media data to obtain the target source data, and play the target source data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: flag bits, the flag bits are used to indicate the type of target media data, the type is audio type or video type, and the number of target bytes It is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits; the controller is specifically configured to: send the target data packet to the display device through the mixed data channel; wherein, in the target media When the data is audio data, the flag bit is used to indicate that the type of the target media data is an audio type; when the target media data is video data, the flag bit is used to indicate that the type of the target media data is a video type.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a time stamp, the time stamp is used to indicate the collection time of the target source data, and the number of target bytes is occupied by the target media data The sum of the number of bytes and the number of bytes occupied by the target timestamp; the controller is specifically configured to: send the target data packet to the display device through the target data channel, and when the target media data is audio data, The target data channel is an audio data channel. In the case where the target media data is video data, the target data channel is a video data channel; wherein, the target timestamp is used to display the device based on the target timestamp to synchronously play the target source data and the first source The data, the target source data and the first source data are different types of media data synchronously collected by smart devices.

In some embodiments of the present application, the target media data is video data, and the target data packet also includes: a target serial number, which is used to indicate the acquisition sequence of the target source data; the controller is also configured to: After the target data packet, the request message sent by the display device is received, and the request message is used to request that the next source data to be sent be encoded and sent in the form of a key frame; based on the request message, the next data packet is sent to the display device, and the next The media data in a data packet is obtained by encoding the next source data to be sent in the form of a key frame.

In the third aspect, the embodiment of the present application provides a data processing method, which is applied to a display device. The method includes: receiving target length data with a preset number of bytes in a target data packet sent by a smart device, and the target data packet includes : Target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, the target media data is any of the following: video data, audio data, audio-video composite data, the target media data is The smart device encodes the collected target source data; based on the target length data, receives the data of the target number of bytes sent by the smart device to obtain the target media data; decodes the target media data to obtain the target source data; Play target source data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: flag bits, the flag bits are used to indicate the type of target media data, the type is audio type or video type, and the number of target bytes It is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits; receiving the target length data of the preset number of bytes in the target data packet sent by the smart device, including: channel, receiving the target length data sent by the smart device; based on the target length data, receiving the target number of bytes sent by the smart device, including: based on the target length data, receiving the target number of bytes sent by the smart device through a mixed data channel Number of data, to obtain flag bit and target media data; Under the situation that flag bit indicates that the type of target media data is an audio type, carry out audio decoding process to target media data and obtain target source data; Indicate target media data in flag bit When the type is a video type, video decoding processing is performed on the target media data to obtain the target source data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a target timestamp, the target timestamp is used to indicate the collection time of the target source data, and the target byte number is specified by the target media data The sum of the number of bytes occupied by the target time stamp and the number of bytes occupied by the target time stamp; receiving the target length data of the preset number of bytes in the target data packet sent by the smart device, including: receiving the smart device through the target data channel For the target length data sent by the device, if the target media data is audio data, the target data channel is an audio data channel; if the target media data is video data, the target data channel is a video data channel; based on the target length data, Receive the data of the target number of bytes sent by the smart device, including: based on the target length data, receive the data of the target number of bytes sent by the smart device through the target data channel to obtain the target time stamp and target media data; based on the target Timestamp, synchronously play the target source data and the first source data, the target source data and the first source data are different types of media data collected synchronously by the smart device.

In some embodiments of the present application, the target media data is video data, and the target data packet also includes: a target serial number, which is used to indicate the collection sequence of the target source data; the method also includes: When the acquisition order indicated by the target serial number satisfies the sequence condition, the target length data of the preset number of bytes sent by the smart device is received. The sequence condition is used to indicate that the target source data and the last source data received are consecutive to the smart device. Collected data; when the collection order indicated by the target serial number does not meet the order condition, send a request message to the smart device, and the request message is used to request that the next source data to be sent be encoded and sent in the form of a key frame; Receive the next data packet sent by the smart device, and the media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame.

In a fourth aspect, the embodiment of the present application provides a data processing method, which is applied to a smart device. The method includes: sending a target data packet to a display device, and the target data packet includes: target length data with a preset number of bytes and a target Media data, the target length data is used to represent the target number of bytes corresponding to the target media data, the target media data is any of the following: video data, audio data, audio and video composite data, the target media data is the target collected by the smart device The source data is encoded and processed; among them, the target data packet is used to display the target length data received by the device, and based on the target length data, receive the data of the target number of bytes sent by the smart device to obtain the target media data, and the target The media data is decoded to obtain the target source data, and the target source data is played.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: flag bits, the flag bits are used to indicate the type of target media data, the type is audio type or video type, and the number of target bytes It is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits; sending the target data packet to the display device includes: sending the target data packet to the display device through a mixed data channel; wherein, in When the target media data is audio data, the flag bit is used to indicate that the type of the target media data is an audio type; when the target media data is video data, the flag bit is used to indicate that the type of the target media data is a video type.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a time stamp, the time stamp is used to indicate the collection time of the target source data, and the number of target bytes is occupied by the target media data The sum of the number of bytes and the number of bytes occupied by the target timestamp; sending the target data packet to the display device, including: sending the target data packet to the display device through the target data channel, when the target media data is audio data In this case, the target data channel is an audio data channel, and when the target media data is video data, the target data channel is a video data channel; wherein, the target timestamp is used to display that the device synchronously plays the target source data and the first The first source data, the target source data and the first source data are different types of media data collected synchronously by smart devices.

In some embodiments of the present application, the target media data is video data, and the target data packet further includes: a target serial number, which is used to indicate the acquisition order of the target source data; the method also includes: sending the target data packet to the display device After that, the request message sent by the display device is received, and the request message is used to request that the next source data to be sent be encoded and sent in the form of a key frame; based on the request message, the next data packet is sent to the display device, and the next data packet The media data in is obtained by encoding the next source data to be sent in the form of a key frame.

In a fifth aspect, the present application provides a computer-readable storage medium, including: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the data processing methods shown in the third aspect and the fourth aspect are implemented .

In a sixth aspect, the present application provides a computer program product, which is characterized by comprising: when the computer program product is run on a computer, the computer is enabled to implement the data processing methods shown in the third aspect and the fourth aspect.

In the embodiment of the present application, the smart device sends the target data packet to the display device, the target data packet includes: target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is Any of the following: video data, audio data, audio-video composite data, the target media data is obtained by encoding and processing the target source data collected by the smart device; the display device receives the preset bytes in the target data packet sent by the smart device The number of target length data, based on the target length data, receiving the data of the target number of bytes sent by the smart device to obtain the target media data; decoding the target media data to obtain the target source data, and playing the target source data. Thus, the display device can determine the corresponding number of bytes (target number of bytes) of the target media data according to the received target length data, and continue to receive the data of the target number of bytes to determine the target media data ( One frame of data) is received, so that the target media data can be decoded and played immediately without buffering the current frame data. After waiting for the arrival of the next frame data, the current frame data is separated from the buffer area and then decoded, which saves Waiting time and buffer space, thereby reducing the delay of mirroring screen projection; at the same time, the existing technology judges the completion of receiving the current frame data through the start code of the next frame data, so it is necessary to judge each received data to determine whether It is the start code, which increases the pressure on the display device to process data. However, this application determines the number of bytes corresponding to the target media data to be received based on the target length data. The display device does not need to judge each data, which reduces the number of display devices. The pressure of processing data further reduces resource consumption. In summary, the solution of this application can improve the data processing efficiency during screen projection.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the implementation methods in related technologies, the following will briefly introduce the drawings that need to be used in the descriptions of the embodiments or related technologies. Obviously, the drawings in the following description are the For some embodiments of the application, those skilled in the art can also obtain other drawings according to these drawings.

FIG. 1 shows an operation scenario between a control device and a display device according to some embodiments;

Fig. 2 shows a hardware configuration block diagram of the control device 100 according to some embodiments;

FIG. 3 shows a block diagram of a hardware configuration of a display device 200 according to some embodiments;

Fig. 4 shows a scene architecture diagram of a data processing method according to some embodiments;

Fig. 5 shows a schematic diagram of software configuration in a smart device 500 according to some embodiments;

FIG. 6 shows a schematic diagram of software configuration in a display device 200 according to some embodiments;

Fig. 7 shows a schematic diagram of the structure of a target data packet according to some embodiments;

Fig. 8 shows a schematic diagram of the structure of a target data packet according to some embodiments;

Fig. 9 shows a schematic diagram of the structure of a target data packet according to some embodiments;

Figure 10 shows a schematic diagram of the structure of a target packet according to some embodiments;

FIG. 11 shows one of the schematic flowcharts of a data processing method according to some embodiments;

Fig. 12 shows the second schematic flow diagram of a data processing method according to some embodiments;

Fig. 13 shows the third schematic flowchart of the data processing method according to some embodiments;

Fig. 14 shows a fourth schematic flowchart of a data processing method according to some embodiments.

Detailed ways

In order to make the purpose and implementation of the application clearer, the following will clearly and completely describe the exemplary implementation of the application in conjunction with the accompanying drawings in the exemplary embodiment of the application. Obviously, the described exemplary embodiment is only the present application. Claim some of the examples, not all of them.

It should be noted that the brief description of the terms in this application is only for the convenience of understanding the implementations described below, and is not intended to limit the implementations of this application. These terms are to be understood according to their ordinary and usual meaning unless otherwise stated.

The terms "first", "second", and "third" in the description and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean limiting specific sequential or sequential unless otherwise noted. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprising" and "having", as well as any variations thereof, are intended to be inclusive but not exclusive, for example, a product or device comprising a series of components is not necessarily limited to all components expressly listed, but may include not expressly listed other components listed or inherent to these products or equipment.

The display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a TV, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, a mobile phone, Tablet PCs, Notebook PCs, PDAs, Automotive Electronics, etc.

Fig. 1 is a schematic diagram of an operation scene between a display device and a control device according to an embodiment, wherein the control device includes a smart device or a control device. As shown in FIG. 1 , the user can operate the display device 200 through the smart device 300 or the control device 100 .

In some embodiments, the control device 100 may be a remote control, and the communication between the remote control and the display device includes infrared protocol communication, bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled wirelessly or wiredly. The user can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, and the like.

In some embodiments, the smart device 300 (such as a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200 . For example, the display device 200 is controlled using an application program running on the smart device.

In some embodiments, the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gesture.

In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300. For example, the module for obtaining voice commands configured inside the display device 200 can directly receive the user's voice command control , the user's voice command control can also be received through the voice control device installed outside the display device 200.

In some embodiments, the display device 200 also performs data communication with the server 400 . The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200 . The server 400 may be one cluster, or multiple clusters, and may include one or more types of servers.

Fig. 2 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment. As shown in FIG. 2 , the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , an external memory, and a power supply. The control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .

As shown in Figure 3, the display device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a user interface 280, an external memory, and a power supply at least one.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.

The display 260 includes a display screen component for presenting a picture, and a drive component for driving image display, for receiving an image signal output from the controller, for displaying video content, image content and a menu control interface component, and a user control UI interface .

The display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220 .

The user interface 280 can be used to receive a control signal from the control device 100 (such as an infrared remote controller, etc.). It can also be used to directly receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to. At this time, it can be called a user input interface.

The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, the detector 230 includes a light receiver, which is a sensor for collecting ambient light intensity; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or , the detector 230 includes a sound collector, such as a microphone, for receiving external sound.

The external device interface 240 may include but not limited to the following: high-definition multimedia interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. multiple interfaces. It may also be a composite input/output interface formed by the above-mentioned multiple interfaces.

The tuner demodulator 210 receives broadcast TV signals through wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or cable broadcast TV signals.

In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box Wait.

The controller 250 controls the work of the display device and responds to user operations through various software control programs stored in the memory (internal memory or external memory). The controller 250 controls the overall operations of the display device 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In some embodiments, the controller includes a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processor (Graphics Processing Unit, GPU), and a random access memory (Random Access Memory, RAM) , a read-only memory (Read-Only Memory, ROM), at least one of the first interface to the nth interface for input/output, a communication bus (Bus), and the like.

Among them, RAM is also called main memory, which is an internal memory that directly exchanges data with the controller. It can be read and written at any time (except when refreshing), and the speed is very fast, and it is usually used as a temporary data storage medium for operating systems or other running programs. The biggest difference between it and ROM is the volatility of data, that is, the stored data will be lost once the power is turned off. RAM is used in computers and digital systems to temporarily store programs, data, and intermediate results. ROM works in a non-destructive readout mode, and can only read out but not write in information. Once the information is written, it is fixed, even if the power is cut off, the information will not be lost, so it is also called fixed memory.

The user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

"User interface" is a medium interface for interaction and information exchange between application programs or operating systems and users, which realizes the conversion between the internal form of information and the form acceptable to users. A commonly used expression form of a user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the display device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. Visual interface elements.

Fig. 4 is a schematic diagram of a scene in some embodiments. As shown in FIG. 4 , the data processing method provided by the embodiment of the present invention includes: a display device 200 and a smart device 500 . Wherein, a communication connection is established between the display device 200 and the smart device 500 . The smart device 500 sends the target data packet to the display device 200, and the display device 200 first receives the target length data of the preset number of bytes in the target data packet sent by the smart device 500, and then receives the target length data sent by the smart device 500 based on the target length data. The data of the target number of bytes is used to obtain the target media data, and the target media data is decoded to obtain the target source data, and the target source data is played. In Fig. 4, the smart device 500 is shown as an example of a mobile phone, but the embodiment of the present invention is not limited thereto, the smart device 500 in the embodiment of the present invention can also be other types of devices, for example: the smart device 500 can also be a TV phones, smart watches, personal computers, handheld computers, etc.

FIG. 5 is a schematic diagram of software configuration in a smart device 500 according to one or more embodiments of the present application. As shown in FIG. layer"), application framework (Application Framework) layer (referred to as "framework layer"), Android runtime (Android runtime) and system library layer (referred to as "system runtime layer"), and the kernel layer. The kernel layer contains at least one of the following drivers: audio driver, display driver, bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive etc. The application layer includes: a communicator for communicating with the display device 200, a video collector (video Record) for video file collection, an audio collector (Audio Record) for audio file collection, and an audio file collection device (Audio Record). An audio data processor (AudioEffect) for encoding processing, a video data processor (video Effect) for encoding video files, and an encapsulator (MUX) for encapsulating data.

FIG. 6 is a schematic diagram of the software configuration in the display device 200 according to one or more embodiments of the present application. As shown in FIG. 6, the system is divided into four layers. layer"), application framework (Application Framework) layer (referred to as "framework layer"), Android runtime (Android runtime) and system library layer (referred to as "system runtime layer"), and the kernel layer. The kernel layer contains at least one of the following drivers: audio driver, display driver, bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive etc. The application layer includes: a communicator for communicating with the smart device 500 , an audio data parser for decoding audio data, and a video data parser for decoding video data.

An embodiment of the present application provides a display device, which includes: a communicator configured to: communicate with a smart device; a controller configured to: receive preset bytes in the target data packet sent by the smart device The number of target length data, the target data packet includes: target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is any of the following: video data, audio Data, audio and video composite data, the target media data is obtained by encoding and processing the collected target source data by the smart device; based on the target length data, receive the data of the target number of bytes sent by the smart device to obtain the target media data; The target media data is decoded to obtain the target source data; the target source data is played.

It can be understood that when the target source data is played, that is, when the target source data is the screen of the smart device recorded by the smart device (that is, the target source data is collected, and the target source data is data of the video type), it is displayed by the display 260 of the above-mentioned FIG. 3 Target source data; in the case where the target source data is the audio of the smart device recorded by the smart device (i.e. collecting the target source data, the target source data is the data of the audio type), play the target source data by the audio output interface 270 of above-mentioned Fig. 3 .

An embodiment of the present application provides a smart device, the smart device includes: a communicator configured to: communicate with a display device; a controller configured to: send a target data packet to the display device, the target data packet includes: Set the target length data and target media data of the number of bytes, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is any of the following: video data, audio data, audio-video composite data , the target media data is obtained by encoding and processing the collected target source data by the smart device; among them, the target data packet is used to display the target length data received by the device, and based on the target length data, receive the target byte number sent by the smart device data to obtain the target media data, and decode the target media data to obtain the target source data, and play the target source data.

It can be understood that the smart device sends the target data packet to the display device; the display device receives the target length data of the preset number of bytes in the target data packet sent by the smart device, and based on the target length data, the display device receives the target word sent by the smart device The data of the number of sections is obtained to obtain the target media data, and the target media data is decoded to obtain the target source data and the target source data is played.

It can be understood that the smart device records the currently playing media file and obtains the audio file and video file synchronously. The target source data is the audio file or video file recorded by the smart device, and the smart device encodes the target source data (the audio file is encoded by audio Encoder, the video file is encoded by the video encoder), the target media data is obtained, and then the target media data is encapsulated to obtain the target data packet (including target length data and target media data with a preset number of bytes) .

It can be understood that the specific value of the preset number of bytes is determined according to actual needs, and is not limited in this embodiment of the present application.

It can be understood that the target media data is any of the following: video data, audio data, and audio-video composite data; wherein, the smart device performs video encoding processing on the recorded video file to obtain video data; the smart device performs video encoding on the recorded audio file Process the audio data; the smart device performs video encoding processing on the recorded video files to obtain video data, performs video encoding processing on the recorded audio files to obtain audio data, and then composites the audio data and video data to obtain audio and video composite data (for Composite audio files and video files are recorded simultaneously by smart devices).

It can be understood that the target media data is at least one frame of complete data. In this application, the target media data is a complete frame of data for description, that is, the video data is a complete frame of video data, and the audio data is a complete frame of video data. , the audio-video composite data is a complete frame of video data and a complete frame of audio data.

It can be understood that the target length data is used to represent the target number of bytes corresponding to the target media data. The target number of bytes may be the number of bytes occupied by the target media data, or the target number of bytes may be the number of bytes occupied by the target media data. The sum of the number of bytes occupied and the number of bytes occupied by other information in the target data packet is specifically determined according to actual conditions, and is not limited in this embodiment of the present application.

Specifically, in the case where the target media data is audio-video composite data, the target number of bytes may be the number of bytes occupied by the target media data, and in the case where the target media data is video data or audio data, the target number of bytes The number of sections may also be the sum of the number of bytes occupied by the target media data and the number of bytes occupied by other information (such as flag bits, time stamps, etc.) in the target data packet.

It can be understood that the display device first receives target length data with a preset number of bytes, and converts the target length data into an integer, which is the number of bytes occupied by the target media data (ie, the number of target bytes), and displays The device continues to receive data, and after receiving the data of the target number of bytes (that is, the target media data), it can be determined that the target media data has been received (that is, the target data packet has been received), and the target media data is decoded to obtain the target source data. Play the target source data, thus, according to the target number of bytes indicated by the target length data, the length of the media data to be received can be determined, and the media data of the corresponding length can be processed and played after receiving the media data of the corresponding length, without waiting for the next frame of data The arrival of , saves time and reduces delay.

Exemplarily, taking the transmission of a frame of data in a data packet as an example, as shown in FIG. 7, it is a possible structure of a target data packet, and the first four (that is, the preset number of bytes) bytes are the target length Data, starting from the fifth byte is a frame of data to be transmitted (that is, the target media data), the target length data is converted into an integer, and the integer is the number of bytes occupied by the target media data. For example: the smart device determines the number of bytes occupied by the target media data, and encapsulates the target media data to obtain the target data packet. The first four bytes of the target data packet are the target length data, and the fifth byte is the target media data. The display device sends the target data packet. The display device first receives the 4-byte target length data sent by the smart device, converts the target length data into an integer 100, and then determines that the target byte number is 100 (the length of the target media data is 100 bytes), the display device continues to receive 100 bytes of data, that is, it can determine that the target media data has been received, decode and play the 100 bytes of data.

In the embodiment of the present application, the smart device sends the target data packet to the display device, the target data packet includes: target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is Any of the following: video data, audio data, audio-video composite data, the target media data is obtained by encoding and processing the target source data collected by the smart device; the display device receives the preset bytes in the target data packet sent by the smart device The number of target length data, based on the target length data, receiving the data of the target number of bytes sent by the smart device to obtain the target media data; decoding the target media data to obtain the target source data, and playing the target source data. Thus, the display device can determine the corresponding number of bytes (target number of bytes) of the target media data according to the received target length data, and continue to receive the data of the target number of bytes to determine the target media data ( One frame of data) is received, so that the target media data can be decoded and played immediately without buffering the current frame data. After waiting for the arrival of the next frame data, the current frame data is separated from the buffer area and then decoded, which saves Waiting time and buffer space, thereby reducing the delay of mirroring screen projection; at the same time, the existing technology judges the completion of receiving the current frame data through the start code of the next frame data, so it is necessary to judge each received data to determine whether It is the start code, which increases the pressure on the display device to process data. However, this application determines the number of bytes corresponding to the target media data to be received based on the target length data. The display device does not need to judge each data, which reduces the number of display devices. The pressure of processing data further reduces resource consumption. In summary, the solution of this application can improve the data processing efficiency during screen projection.

It can be understood that the target data packets can be transmitted through the same data channel, or can be transmitted through different data channels.

Specifically, when the target media data carried in the target data packet is composite audio and video data, the smart device sends the target data packet to the display device through the same data channel. At this time, the audio and video data are transmitted simultaneously, and the display device receives the composite data. After data, the audio data and video data are separated first, and then the audio data and video data are decoded respectively; when the target media data carried in the target data packet is audio data or video data, the smart device passes the target data packet through the same The data channel is sent to the display device, but the display device cannot distinguish whether the target media data carried in the received target data packet is audio data or video data, so it is not sure whether to perform audio decoding or video encoding processing on the received target media data. At this point, a flag can be added to the target data packet, so that the display device can distinguish whether the target media data carried in the target data packet is audio data or video data according to the flag; the target media data carried in the target data packet is For audio data or video data, the data packets carrying audio data and the data packets carrying video data can also be transmitted through different data channels. At this time, the display device can determine the received target data packets according to the different data channels The target media data is audio data or video data.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: flag bits, the flag bits are used to indicate the type of target media data, the type is audio type or video type, and the number of target bytes It is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits. The smart device sends the target data packet to the display device through the mixed data channel; the display device receives the data packet sent by the smart device through the mixed data channel. The target length data, based on the target length data, the display device receives the data of the target number of bytes sent by the smart device through the mixed data channel to obtain the flag bit and target media data, and the flag bit indicates that the type of the target media data is audio In the case of type, perform audio decoding processing on the target media data to obtain the target source data, and if the flag bit indicates that the type of the target media data is video type, perform video decoding processing on the target media data to obtain the target source data.

It can be understood that the flag bit is used to indicate that the type of the target media data is an audio type or a video type, and the number of bytes occupied by the flag bit and the specific value are determined according to actual needs, and are not limited in this embodiment of the present application.

It can be understood that no matter the type of target media data is audio type or video type, the target data packet carrying the target media data is transmitted through the mixed data channel, that is, after the smart device encapsulates the audio data or video data to be transmitted, it is transmitted through the same data The channel is sent to the display device.

It can be understood that the smart device encodes the collected target source data to obtain the target media data, and encapsulates the target media data to obtain the target data packet. The data in the target data packet are: target length data, flag bits and target media data ; The display device receives the target length data of the target data packet, thus can determine the flag bits to be received and the number of bytes occupied by the target media data, and then determine the type of the target media data according to the flag bits, in the target audio data When the type is an audio type, audio decoding processing is performed, and when the type of the target audio data is a video type, video decoding processing is performed, and the target source data obtained after the processing is played.

Specifically, the display device encapsulates the process of target media data to determine the flag bit: when the target media data is audio data, the flag bit is used to indicate that the type of the target media data is an audio type, and when the target media data is video data Next, the flag bit is used to indicate that the type of the target media data is a video type, and the target length data is determined according to the result of summing the number of bytes occupied by the flag bit and the number of bytes occupied by the target media data; the target length data, flag The bit and the target media data are packed to obtain the target data packet.

Exemplarily, take the transmission of a frame of data in a data packet as an example, as shown in Figure 8, which is a possible target data packet structure, and the first four (that is, the preset number of bytes) bytes are the target length data , the fifth byte is a flag bit (for example, the flag bit is 0 indicating that the type of the target media data is an audio type, and the flag bit is 1 indicating that the type of the target media data is a video type), starting from the sixth byte for the need to transmit A frame of data (that is, the target media data). For example: the smart device determines the number of bytes occupied by the target media data, encapsulates the target media data to obtain the target data packet, the first four bytes of the target data packet are the target length data, the fifth byte is the flag bit, and the sixth The first byte is the target media data, send the target data packet to the display device, the display device first receives the first 4 bytes of target length data sent by the smart device, converts the target length data into an integer 101, and the fifth byte The corresponding data is 1, so it can be seen that the target media data is video data, and the length of the target media data is 100 bytes. The display device will receive 100 bytes of data except the flag bit, and perform video decoding processing and play.

It can be understood that the data packets carrying audio data and the data packets carrying video data are transmitted through the same data channel, and the smart device processes and sends them according to the order in which the source data is collected, so the data packets received by the smart device are also in order, and the data packets are decoded and processed After playing the media data carried in the video, there is no need for video data and audio data to wait for each other. Specifically, because the data packets carrying audio data and the data packets carrying video data are received through the same channel, for the same frame of video data and corresponding audio data, the smart device sends them sequentially, and the display device also receives them sequentially. Then the source data corresponding to the video data and the source data corresponding to the audio data after processing may have a difference in time (also affected by the decoding processing time of the video data and audio data), but the time difference is generally small, and the human eye or The ear cannot distinguish, so it does not affect the playback effect, so the video data and audio data do not need to wait for each other.

In the embodiment of the present application, the target data packet also includes: a flag bit, which is used to indicate the type of the target media data, the type is audio type or video type, and the number of target bytes is the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits, the smart device sends the target data packet to the display device through the mixed data channel; the display device receives the target length data sent by the smart device through the mixed data channel, and the display device based on the target length data , through the mixed data channel, receive the data of the target number of bytes sent by the smart device to obtain the flag bit and the target media data, and perform audio on the target media data when the flag bit indicates that the type of the target media data is an audio type The target source data is obtained through decoding processing, and when the flag bit indicates that the type of the target media data is a video type, video decoding processing is performed on the target media data to obtain the target source data. The smart device processes and sends the data according to the order in which the source data is collected, and the data packets received by the display device are also sequential. Therefore, the display device will play the media data received through the mixed data channel after processing, avoiding the need for video data and audio data to pass through When different data channels are transmitted, they need to wait for each other to play synchronously. Different types of data packets are transmitted through the same data channel. The display device can process and play the media data after receiving the media data, which saves time and reduces delay.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a target timestamp, the target timestamp is used to indicate the collection time of the target source data, and the target byte number is specified by the target media data The sum of the number of bytes occupied and the number of bytes occupied by the target timestamp. The smart device sends the target data packet to the display device through the target data channel; the display device receives the target length data sent by the smart device through the target data channel, and based on the target length data, receives the target bytes sent by the smart device through the target data channel data to obtain the target timestamp and target media data, based on the target timestamp, the target source data and the first source data are played synchronously, and the target source data and the first source data are different types of media data synchronously collected by smart devices; Wherein, when the target media data is audio data, the target data channel is an audio data channel, and when the target media data is video data, the target data channel is a video data channel.

It can be understood that the target time stamp is used to indicate the collection time of the target source data. Specifically, the time indicated by the target time stamp may be the collection time of the target source data, or the time when the target source data is encoded, or The time for encapsulating the target media data, specifically the time indicated by the target timestamp is determined according to actual needs, and is not limited in this embodiment of the present application.

It can be understood that based on the target time stamp, the target source data and the first source data are played synchronously, that is, based on the target time stamp and the first time stamp, the target source data and the first source data are played synchronously, and the first time stamp is used to indicate the first The acquisition time of the media data, the absolute value of the difference between the first timestamp and the target timestamp is less than or equal to the difference threshold, the first media data and the target media data are different types of media data, and the first source data is the first The media data is decoded and processed.

It can be understood that if the absolute values of the differences between the timestamps carried in multiple data packets and the target timestamps are all less than or equal to the difference threshold, the difference between the timestamps carried in multiple data packets and the target timestamps can be The media data carried in the data packet with the smallest absolute value is used as the first media data, or the media data carried in any data packet of multiple data packets may be used as the first media data, which is not specifically limited in this embodiment of the present application.

It can be understood that the target source data and the first source data are different types of media data synchronously collected by smart devices, that is, the type of the target source data is audio type, then the type of the first source data is video type, and the type of target source data is video type, the type of the first source data is audio type.

It can be understood that the audio data channel and the video data channel are different data channels, the media data carried in the data packet received by the display device through the audio data channel, and the source data is obtained through audio decoding processing; the data packet received through the video data channel The media data carried in the source data is obtained through video decoding processing.

Exemplarily, taking the transmission of one frame of data in one data packet as an example, as shown in FIG. 9, it is a possible target data packet structure, and the first four (that is, the preset number of bytes) bytes are the target length data , the 5th to 12th bytes are timestamps (eight bytes), and the 13th byte is a frame of data to be transmitted (that is, target media data).

In the embodiment of this application, the smart device sends the target data packet to the display device through the target data channel; the display device receives the target length data sent by the smart device through the target data channel, and receives the target length data from the smart device through the target data channel based on the target length data. Send the data of the target number of bytes to obtain the target timestamp and target media data. Based on the target timestamp, the target source data and the first source data are played synchronously. The target source data and the first source data are synchronously collected by smart devices Different types of media data. Thus, the data packets carrying video data and audio data packets are sent through different data channels, and the display device synchronously plays the target source data and the first source data based on the target timestamp carried in the received data packets, and displays The audio and video synchronization effect on the device side is better, and the user experience is better.

In some embodiments of the present application, the target media data is video data, and the target data packet also includes: a target serial number, which is used to indicate the collection sequence of the target source data; the display device receives the target serial number sent by the smart device When the indicated collection sequence satisfies the sequence condition, receive the target length data of the preset number of bytes sent by the smart device. The sequence condition is used to indicate that the target source data and the last source data received are the data continuously collected by the smart device , when the acquisition sequence indicated by the target serial number does not meet the sequence condition, the display device sends a request message to the smart device, and the request message is used to request that the next source data to be sent be encoded and sent in the form of a key frame; the smart The device receives the request message sent by the display device, and based on the request message, sends the next data packet to the display device. The media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame; display The device receives the next packet sent by the smart device.

It can be understood that the target sequence number is used to indicate the collection sequence of the target source data, such as: continuously collect source data, the target sequence number in the data packet corresponding to the first frame of source data is 1, and the target sequence number in the data packet corresponding to the second frame of source data The target sequence number of the frame is 2, the target sequence number in the data packet corresponding to the third frame source data is 3, and so on, the number of the sequence number can be a sequential positive integer (such as 1, 2, 3...), or It may be a positive integer (such as 0, 2, 4, .

Exemplarily, the sequence number is numbered according to the sequence of positive integers, indicating that the device received the sequence number of the last data packet as 3, and received the sequence number of the current data packet as 4. This shows that the data packets before the current data packet are not lost; it shows that the device received the sequence number of the previous data packet is 3, and the sequence number of the current data packet is 5, and the sequence number 5 and sequence number 3 do not meet the sequence condition. This shows that the data packet whose sequence number is 4 before the current data packet is lost, and the display device sends a request message to the smart device, requesting that the next source data to be sent be encoded and sent in the form of a key frame.

It can be understood that a key frame means that during the encoding process, all the data of the frame is retained, and the display device can completely decode all the data of the frame after receiving the key frame without relying on other frame data. In order to save space and improve transmission efficiency, the processing of other frame data will depend on key frames. For example, the first frame data is a key frame, and the second frame data stores the area that is different from the first frame data. The same area is not stored. The third frame data stores the area that is different from the second frame data, and the same area is not stored. Therefore, after the previous frame data is lost, the next frame data only stores the area that is different from the previous frame data , then only the area with the difference can be displayed after decoding, and mosaics will be displayed in other areas.

It can be understood that when the display device determines that there is data packet loss, all data packets received by the display device can be discarded before the arrival of the key frame, so as to prevent a large number of mosaics when playing video due to data packet loss.

It can be understood that the source data can be encoded into key frames based on the H.264 protocol, and the source data can be encoded into key frames based on the H.265 protocol. Specifically, it is determined according to actual needs, which is not limited in this embodiment of the present application.

Exemplarily, take the transmission of one frame of data in one data packet as an example, as shown in Figure 10, which is a possible target data packet structure, the first 4 bytes are the sequence number, and the 5th to 8th bytes are the target Length data, starting from the 9th byte is a frame of data to be transmitted (that is, target media data).

It can be understood that the target data packet may also include other information, such as a flag bit, a time stamp, etc., which are not limited in this embodiment of the present application.

It can be understood that the smart device receives the request message sent by the display device, encodes the latest collected source data to be sent in the form of a key frame to obtain media data, and then encapsulates the media data into a data packet and sends it to the display device.

In the embodiment of the present application, the target data packet includes a serial number, and the display device determines whether the data packet before receiving the current data packet is lost according to the serial number, and in the case of data packet loss, the display device sends a request message to the smart device , request to encode and send the source data of the next frame to be sent in the form of a key frame; after packet loss occurs in the mirror projection scene, mosaics will appear when playing the video, which will affect the user experience. Therefore, the smart device will send the next frame The source data to be sent is encoded into a key frame form, and the display device receives the key frame without relying on other frame data. After decoding, a complete frame of source data to be played can be obtained, and mosaics will no longer appear in the video, improving user experience.

In the embodiment of the present application, a data processing method is provided. As shown in FIG. 11 , the method includes the following steps 801 to 805 .

801. The smart device sends a target data packet to the display device.

Wherein, the target data packet includes: target length data and target media data with a preset number of bytes, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is any of the following: video data , audio data, audio-video composite data, and the target media data is obtained by encoding and processing the collected target source data by the smart device.

802. The display device receives target length data with a preset number of bytes in the target data packet sent by the smart device.

803. Based on the target length data, the display device receives data of the target number of bytes sent by the smart device.

Wherein, based on the target length data, the data of the target number of bytes sent by the smart device is received to obtain the target media data.

804. The display device decodes the target media data to obtain target source data.

805. The display device plays the target source data.

In the embodiment of the present application, the smart device sends the target data packet to the display device, the target data packet includes: target length data and target media data, the target length data is used to represent the target number of bytes corresponding to the target media data, and the target media data is Any of the following: video data, audio data, audio-video composite data, the target media data is obtained by encoding and processing the target source data collected by the smart device; the display device receives the preset bytes in the target data packet sent by the smart device The number of target length data, based on the target length data, receiving the data of the target number of bytes sent by the smart device to obtain the target media data; decoding the target media data to obtain the target source data, and playing the target source data. Thus, the display device can determine the corresponding number of bytes (target number of bytes) of the target media data according to the received target length data, and continue to receive the data of the target number of bytes to determine the target media data ( One frame of data) is received, so that the target media data can be decoded and played immediately without buffering the current frame data. After waiting for the arrival of the next frame data, the current frame data is separated from the buffer area and then decoded, which saves Waiting time and buffer space, thereby reducing the delay of mirror projection; at the same time, the existing technology judges the completion of receiving the current frame data through the start code of the next frame data, so it is necessary to judge each received data to determine whether It is the start code, which increases the pressure on the display device to process data. However, this application determines the number of bytes corresponding to the target media data to be received based on the target length data. The display device does not need to judge each data, which reduces the number of display devices. The pressure of processing data further reduces resource consumption. In summary, the solution of this application can improve the data processing efficiency during screen projection.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a flag; in conjunction with FIG. 11, as shown in FIG. Step 803a is implemented, and the above step 804 can be specifically implemented through the following steps 804a and 804b.

801a. The smart device sends the target data packet to the display device through the hybrid data channel.

802a. The display device receives the target length data sent by the smart device through the mixed data channel.

803a. Based on the target length data, the display device receives the data of the target number of bytes sent by the smart device through the mixed data channel.

Among them, the number of target bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits; based on the target length data, the target byte number sent by the smart device is received through the mixed data channel. Number of data to get flag bits and target media data.

804a. When the flag bit indicates that the type of the target media data is an audio type, the display device performs audio decoding processing on the target media data to obtain target source data.

Wherein, the flag bit is used to indicate the type of the target media data, which is an audio type or a video type.

804b. When the flag bit indicates that the type of the target media data is a video type, the display device performs video decoding processing on the target media data to obtain target source data.

In the embodiment of the present application, the target data packet also includes: a flag bit, which is used to indicate the type of the target media data, the type is audio type or video type, and the number of target bytes is the number of bytes occupied by the target media data and the number of bytes occupied by the flag bits, the smart device sends the target data packet to the display device through the mixed data channel; the display device receives the target length data sent by the smart device through the mixed data channel, and the display device based on the target length data , through the mixed data channel, receive the data of the target number of bytes sent by the smart device to obtain the flag bit and the target media data, and perform audio on the target media data when the flag bit indicates that the type of the target media data is an audio type The target source data is obtained through decoding processing, and when the flag bit indicates that the type of the target media data is a video type, video decoding processing is performed on the target media data to obtain the target source data. The smart device processes and sends the data according to the order in which the source data is collected, and the data packets received by the display device are also sequential. Therefore, the display device will play the media data received through the mixed data channel after processing, avoiding the need for video data and audio data to pass through When different data channels are transmitted, they need to wait for each other to play synchronously. Different types of data packets are transmitted through the same data channel. The display device can process and play the media data after receiving the media data, which saves time and reduces delay.

In some embodiments of the present application, the target media data is video data or audio data; the target data packet also includes: a target timestamp; in conjunction with FIG. 11 , as shown in FIG. 13 , the above steps 801 to 803 can be specifically implemented through the following step 801b Step 803b is implemented, and the above step 805 can be specifically implemented through the following step 805a.

801b. The smart device sends the target data packet to the display device through the target data channel.

Wherein, when the target media data is audio data, the target data channel is an audio data channel, and when the target media data is video data, the target data channel is a video data channel.

802b. The display device receives the target length data sent by the smart device through the target data channel.

803b. Based on the target length data, the display device receives data of the target number of bytes sent by the smart device through the target data channel.

Among them, the number of target bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the target timestamp; based on the target length data, the target bytes sent by the smart device are received through the target data channel number of data to get the target timestamp and target media data.

805a. The display device synchronously plays the target source data and the first source data based on the target timestamp.

Wherein, the target timestamp is used to indicate the collection time of the target source data; the target source data and the first source data are different types of media data synchronously collected by the smart device.

In the embodiment of this application, the smart device sends the target data packet to the display device through the target data channel; the display device receives the target length data sent by the smart device through the target data channel, and receives the target length data from the smart device through the target data channel based on the target length data. Send the data of the target number of bytes to obtain the target timestamp and target media data. Based on the target timestamp, the target source data and the first source data are played synchronously. The target source data and the first source data are synchronously collected by smart devices Different types of media data. Thus, the data packets carrying video data and audio data packets are sent through different data channels, and the display device synchronously plays the target source data and the first source data based on the target timestamp carried in the received data packets, and displays The audio and video synchronization effect on the device side is better, and the user experience is better.

In some embodiments of the present application, the target media data is video data, and the target data packet further includes: a target serial number; referring to FIG. 11 , as shown in FIG. 14 , the method further includes the following steps 806 to 810.

806. The display device judges whether the collection sequence indicated by the received target sequence number satisfies the sequence condition.

If yes, perform the above step 802, otherwise perform the following step 807.

Wherein, the target sequence number is used to indicate the collection sequence of the target source data; the sequence condition is used to indicate that the target source data and the last source data received are data continuously collected by the smart device.

807. The display device sends a request message to the smart device.

Wherein, the request message is used to request to encode and send the next source data to be sent in the form of a key frame.

808. The smart device receives the request message sent by the display device.

809. The smart device sends the next data packet to the display device based on the request message.

Wherein, the media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame.

810. The display device receives the next data packet sent by the smart device.

It should be noted that, for the specific process of receiving and processing the next data packet by the display device, reference may be made to the description of the above step 802 to step 805, which will not be repeated here.

In the embodiment of the present application, the target data packet includes a serial number, and the display device determines whether the data packet before receiving the current data packet is lost according to the serial number, and in the case of data packet loss, the display device sends a request message to the smart device , request to encode and send the source data of the next frame to be sent in the form of a key frame; after packet loss occurs in the mirror projection scene, mosaics will appear when playing the video, which will affect the user experience. Therefore, the smart device will send the next frame The source data to be sent is encoded into a key frame form, and the display device receives the key frame without relying on other frame data. After decoding, a complete frame of source data to be played can be obtained, and mosaics will no longer appear in the video, improving user experience.

The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process performed by the above-mentioned data processing method can be achieved, and the same Technical effects, in order to avoid repetition, will not be repeated here.

Wherein, the computer-readable storage medium may be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

The present invention provides a computer program product, including: when the computer program product is run on a computer, the computer is enabled to implement the above data processing method.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit it; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present application. scope.

For convenience of explanation, the above description has been made in conjunction with specific implementation manners. However, the above exemplary discussion is not intended to be exhaustive or to limit the implementations to the precise forms disclosed above. Many modifications and variations are possible in light of the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various modified embodiments suitable for specific use considerations.

Claims (10)

1. A display device, characterized in that the display device comprises: a communicator configured to: communicate with the smart device; The controller is configured to: receive target length data with a preset number of bytes in the target data packet sent by the smart device, the target data packet includes: the target length data and target media data, and the target The length data is used to characterize the number of target bytes corresponding to the target media data, the target media data is any of the following: video data, audio data, audio-video composite data, and the target media data is the smart device Obtained by encoding and processing the collected target source data; Based on the target length data, receive the data of the target number of bytes sent by the smart device to obtain the target media data; Decoding the target media data to obtain the target source data; Play the target source data. 2. The display device according to claim 1, wherein the target media data is video data or audio data; the target data packet further comprises: a flag, and the flag is used to indicate the target media The type of data, the type is an audio type or a video type, and the target number of bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bit; The controller is specifically configured as: receiving the target length data sent by the smart device through a hybrid data channel; Based on the target length data, receive the data of the target number of bytes sent by the smart device through the mixed data channel to obtain the flag bit and the target media data; When the flag bit indicates that the type of the target media data is an audio type, perform audio decoding processing on the target media data to obtain the target source data; In a case where the flag bit indicates that the type of the target media data is a video type, video decoding processing is performed on the target media data to obtain the target source data. 3. The display device according to claim 1, wherein the target media data is video data or audio data; the target data packet further comprises: a target timestamp, and the target timestamp is used to indicate the The acquisition time of the target source data, the target number of bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the target timestamp; The controller is specifically configured as: Receive the target length data sent by the smart device through the target data channel, in the case where the target media data is audio data, the target data channel is an audio data channel, and when the target media data is video data In the case of , the target data channel is a video data channel; Based on the target length data, receive the data of the target number of bytes sent by the smart device through the target data channel to obtain the target timestamp and the target media data; Based on the target timestamp, the target source data and the first source data are synchronously played, and the target source data and the first source data are different types of media data synchronously collected by the smart device. 4. The display device according to any one of claims 1 to 3, wherein the target media data is video data, and the target data packet further includes: a target serial number, the target serial number is used to indicate The collection sequence of the target source data; The controller is further configured to: When the collection order indicated by the target serial number sent by the smart device satisfies the sequence condition, receive the target length data of the preset number of bytes sent by the smart device, and the sequence condition Used to indicate that the target source data and the received last source data are data continuously collected by the smart device; In the case that the collection sequence indicated by the target sequence number does not satisfy the sequence condition, a request message is sent to the smart device, and the request message is used to request that the next source data to be sent be encoded in the form of a key frame process and send; The next data packet sent by the smart device is received, and the media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame. 5. A smart device, characterized in that the smart device comprises: a communicator configured to: communicate with a display device; The controller is configured to: send a target data packet to the display device, the target data packet includes: target length data and target media data with a preset number of bytes, the target length data is used to characterize the target The number of target bytes corresponding to the media data, the target media data is any of the following: video data, audio data, audio-video composite data, the target media data is the target source data collected by the smart device processed; Wherein, the target data packet is used for the display device to receive the target length data, and based on the target length data, receive the data of the target byte number sent by the smart device, so as to obtain the target media data, and decode the target media data to obtain the target source data, and play the target source data. 6. The smart device according to claim 5, wherein the target media data is video data or audio data; the target data packet further comprises: a flag, and the flag is used to indicate the target media The type of data, the type is an audio type or a video type, and the target number of bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the flag bit; The controller is specifically configured as: sending the target data packet to the display device through a mixed data channel; Wherein, when the target media data is audio data, the flag bit is used to indicate that the type of the target media data is an audio type; When the target media data is video data, the flag bit is used to indicate that the type of the target media data is a video type. 7. The smart device according to claim 5, wherein the target media data is video data or audio data; the target data packet further comprises: a time stamp, and the time stamp is used to indicate the target source The collection time of the data, the target number of bytes is the sum of the number of bytes occupied by the target media data and the number of bytes occupied by the target timestamp; The controller is specifically configured as: Send the target data packet to the display device through the target data channel, in the case where the target media data is audio data, the target data channel is an audio data channel, and in the case where the target media data is video data In the case, the target data channel is a video data channel; Wherein, the target time stamp is used for the display device to synchronously play the target source data and the first source data based on the target time stamp, and the target source data and the first source data are the smart device Different types of media data collected simultaneously. 8. The smart device according to any one of claims 5 to 7, wherein the target media data is video data, and the target data packet further includes: a target serial number, which is used to indicate The collection sequence of the target source data; The controller is further configured to: After sending the target data packet to the display device, receiving a request message sent by the display device, the request message is used to request that the next source data to be sent be encoded and sent in the form of a key frame; Based on the request message, a next data packet is sent to the display device, and the media data in the next data packet is obtained by encoding the next source data to be sent in the form of a key frame. 9. A data processing method, characterized in that being applied to a display device, said method comprising: Receive target length data with a preset number of bytes in the target data packet sent by the smart device, the target data packet includes: the target length data and target media data, and the target length data is used to characterize the target media The number of target bytes corresponding to the data, the target media data is any of the following: video data, audio data, audio-video composite data, the target media data is the target source data collected by the smart device for encoding processing owned; Based on the target length data, receive the data of the target number of bytes sent by the smart device to obtain the target media data; Decoding the target media data to obtain the target source data; Play the target source data. 10. A data processing method, characterized in that being applied to an intelligent device, said method comprising: Send a target data packet to the display device, the target data packet includes: target length data and target media data with a preset number of bytes, and the target length data is used to represent the target number of bytes corresponding to the target media data , the target media data is any of the following: video data, audio data, audio-video composite data, and the target media data is obtained by encoding the collected target source data by the smart device; Wherein, the target data packet is used for the display device to receive the target length data, and based on the target length data, receive the data of the target byte number sent by the smart device, so as to obtain the target media data, and decode the target media data to obtain the target source data, and play the target source data.

Images