CN120075532A - Video playing method and related equipment - Google Patents

Abstract

The application provides a video playing method and related equipment, and relates to the technical field of terminal equipment. The video playing method comprises the steps of responding to a first operation, displaying a first interface of a video application, responding to a second operation on the first interface, starting to play a first picture of a first video at a first moment, wherein image data of the first video is not processed by a first processor between the first moment and a second moment, responding to a preset trigger event at the second moment, processing the image data of the first video by the first processor, and playing the second picture of the first video based on the image data of the first video processed by the first processor after the second moment, wherein the power consumption of the first processor between the first moment and the second moment is smaller than the power consumption of the first processor after the second moment. By using the method, the power consumption of playing the video by the terminal equipment can be reduced.

Description

Video playing method and related equipment

The application relates to a video playing method and related equipment, which are divided into Chinese patent applications with the application number of 202410749805.X and the application date of 2024-06-11.

Technical Field

The embodiment of the application relates to the field of terminal equipment, in particular to a video playing method and related equipment.

Background

With the development of terminal technology, the functions of terminal equipment are more and more abundant, and especially, the video playing function gradually becomes a more common function of the terminal equipment. The terminal equipment can play the video through various installed applications.

In order to improve the smoothness and image quality of video playing, video playing applications generally employ a multi-buffer (buffer) technology to buffer image frames, and call a graphics processor (graphics processing unit, GPU) to enhance the buffered image frames. However, in the video playing process, the buffering of the image frames by using multiple buffers and the image frame enhancement processing by using the GPU all result in the increase of the power consumption of the terminal device, and the duration of the terminal device is reduced.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a video playing method and related devices, which can solve the problem that the power consumption of the terminal device is relatively high during the video playing process.

The application provides a video playing method which is applied to terminal equipment and comprises the steps of responding to a first operation, starting a video application, responding to a second operation of playing a first video on the video application, obtaining a video stream of the first video, adopting a first processing mode to process the video stream, wherein the first processing mode comprises the steps of decoding the video stream, storing video image frames obtained by decoding in a display buffer area for sending and displaying, responding to a preset trigger event, adopting a second processing mode to process the video stream, wherein the second processing mode comprises the steps of decoding the video stream, storing video image frames obtained by decoding in the buffer area, and copying the video image frames stored in the buffer area to the display buffer area for sending and displaying.

By adopting the technical scheme, under the condition that the video application plays the video, the first processing mode is used for decoding and playing the video stream by default, as the buffer (buffer) in the buffer area is not used for caching the video image frames and the GPU is not used for processing the video image frames, the video playing power consumption of the terminal equipment can be reduced, under the condition of detecting the preset trigger event, for example, the video image frame playing and clamping event or the video application receives the event of the preset operation, and then the processing mode of the video stream is switched from the first processing mode to the second processing mode, so that the smoothness of video playing is ensured and the preset operation can be correctly executed, for example, the preset operation can include an operation of triggering and calling a Graphics Processor (GPU) to process the video image frame.

In one possible implementation manner, the video playing method further includes obtaining media information of a video stream of the first video, wherein the media information includes a resolution of the first video, processing the video stream in a first processing mode when the resolution of the first video is not a preset resolution, and processing the video stream in a second processing mode when the resolution of the first video is the preset resolution.

By adopting the technical scheme, the preset resolution can be set according to actual requirements, for example, the preset resolution is ultra-high definition resolution, for example, 4K resolution, for video playing with ultra-high definition resolution, the GPU may need to be called to adjust the resolution of the decoded video image frames, in order to ensure the smoothness of video playing and smooth resolution adjustment, the video stream is processed by adopting the second processing mode, the GPU adjusts the resolution of the video image frames stored by the buffer, and then copies the video image frames to the display buffer for display. For the situation that the resolution is not preset, as the GPU is not required to be called to carry out resolution adjustment on the decoded video image frames, the video stream can be processed by adopting the first processing mode, so that the video playing power consumption of the terminal equipment is reduced.

In one possible implementation manner, before the video stream is processed in the first processing mode, the video playing method further comprises the steps of obtaining performance information of the terminal device, wherein the performance information comprises one or more of a processor model, a memory size, a rated battery power and a residual battery power, and determining the processing mode of the video stream based on the performance information.

By adopting the technical scheme, the processing mode used by default for video playing can be determined based on the performance information of the terminal equipment, for example, if one of the conditions of lower performance scores of a processor, smaller memory and smaller battery capacity is met, the residual capacity of the battery is lower, the decoding and playing of the video stream are carried out by default by using the first processing mode, and otherwise, the decoding and playing of the video stream are carried out by default by using the second processing mode.

In one possible implementation manner, before the video stream is processed in the first processing mode, the video playing method further comprises the steps of obtaining historical video playing related information of the terminal device, wherein the historical video playing related information comprises one or more of video playing frequency in unit time, video playing duration in unit time, operation times of controls in a video playing interface in unit time and network speed of a network connected when the terminal device plays the video, and determining the processing mode of the video stream based on the historical video playing related information.

By adopting the technical scheme, the processing mode for default use of video playing can be determined based on the historical video playing related information of the terminal equipment, for example, if one of the conditions of higher video playing frequency in unit time, longer video playing time in unit time and fewer operation times of the control in the video playing interface in unit time is met, the network speed of the network connected with the terminal equipment when the terminal equipment plays video is better, the first processing mode is used for decoding and playing the video stream by default, and otherwise, the second processing mode is used for decoding and playing the video stream by default. The unit time can be set according to actual requirements, for example, the unit time is in a unit of one day.

In one possible implementation, before the video stream is processed in the first processing mode, the video playing method further comprises determining a processing mode of the video stream based on a preset mode selection model, wherein training data of the mode selection model comprises performance information of the terminal device and historical video playing related information of the terminal device.

By adopting the technical scheme, the processing mode for default use of video playing can be determined based on the mode selection model obtained by pre-training, and the training data of the mode selection model can comprise performance information of the terminal equipment and/or historical video playing related information of the terminal equipment.

In one possible implementation, the preset trigger event includes a video image frame play stuck event or an event that the video application receives a preset operation.

By adopting the technical scheme, if the video image frame playing katon event is detected or the video application receives the event of the preset operation, the smoothness of video playing can be ensured and the preset operation can be correctly executed by switching the processing mode of the video stream from the first processing mode to the second processing mode, for example, the preset operation can comprise the operation of triggering and calling a Graphics Processor (GPU) to process the video image frame.

In one possible implementation, the preset operation includes an operation that triggers the invoking of the graphics processor to process the video image frame.

By adopting the technical scheme, the second processing mode is configured with the logic for processing the video image frames by the graphic processor, and the processing mode of the video stream is switched from the first processing mode to the second processing mode under the condition that the event triggering the invoking of the graphic processor to process the video image frames is detected, so that the GPU can smoothly process the video image frames.

In one possible implementation, in the first processing mode, the decoded video image frames are not processed using the graphics processor.

By adopting the technical scheme, in the first processing mode, the GPU is not used for processing the video image frames, so that the video playing power consumption of the terminal equipment can be reduced.

In one possible implementation, the video playing interface provided by the video application includes a plurality of controls, and the preset operation includes a touch operation for a target control in the plurality of controls, where the target control is a control for triggering and calling the GPU.

By adopting the technical scheme, the second processing mode is configured with the execution logic of the task corresponding to the target control, and when the event of clicking operation for the target control in the video playing interface is detected, the processing mode of the video stream is switched from the first processing mode to the second processing mode, so that the clicking operation for the target control can be smoothly responded.

In a second aspect, the application provides a terminal device, which comprises a touch screen, a memory and a processor, wherein the touch screen, the memory and the processor are coupled, the memory is used for storing program instructions, and the processor is used for reading the program instructions stored in the memory so as to realize the video playing method of the first aspect.

In a third aspect, the present application provides a computer readable storage medium having stored therein computer readable instructions which when executed by a processor implement the video playing method of the first aspect.

In a fourth aspect, the present application provides a chip coupled to a memory in a terminal device, where the chip is configured to control the terminal device to implement the video playing method of the first aspect.

In addition, the technical effects of the second aspect to the fourth aspect may be referred to in the description related to the method designed in the method section, and are not repeated here.

Drawings

Fig. 1 is a schematic diagram of a video playing architecture of a terminal device according to an embodiment of the present application;

FIG. 2 is a schematic diagram of an image frame stored in a display buffer according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an image frame stored in a display buffer according to another embodiment of the present application;

Fig. 4 is a schematic diagram of interface display of a terminal device according to an embodiment of the present application;

Fig. 5 is a schematic diagram of interface display of a terminal device according to another embodiment of the present application;

Fig. 6 is a schematic flow chart of decoding and playing an acquired video stream by a terminal device according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an application architecture of a mode selection framework according to an embodiment of the present application;

fig. 8 is a flowchart of a video playing method according to an embodiment of the present application;

Fig. 9 is a hardware architecture diagram of a terminal device according to an embodiment of the present application;

Fig. 10 is a software architecture diagram of a terminal device according to an embodiment of the present application.

Detailed Description

The following description of the technical solutions according to the embodiments of the present application will be given with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In describing embodiments of the present application, words such as "exemplary," "or," "such as," and the like are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "or," "such as," and the like are intended to present related concepts in a concrete fashion.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It is to be understood that, unless otherwise indicated, a "/" means or. For example, A/B may represent A or B. The "and/or" in the present application is merely one association relationship describing the association object, indicating that three relationships may exist. For example, A and/or B may mean that A alone, both A and B, and B alone are present. "at least one" means one or more. "plurality" means two or more than two. For example, at least one of a, b or c may represent seven cases of a, b, c, a and b, a and c, b and c, a, b and c. It will be appreciated that the order of the steps shown in the flowcharts herein may be changed and some may be omitted.

To facilitate understanding of the embodiments of the present application, first, technical terms related to the present application are described:

A User Interface (UI) is a media interface for interaction and information exchange between an application or operating system and a user, which enables conversion between an internal form of information and a form acceptable to the user. The user interface is a source code written in a specific computer language such as java, extensible markup language (extensible markup language, XML) and the like, and the interface source code is analyzed and rendered on the terminal equipment to finally be presented as content which can be identified by a user. A commonly used presentation form of a user interface is a graphical user interface (graphic user interface, GUI), which refers to a graphically displayed user interface that is related to computer operations. It may be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed in the display screen of the terminal device.

An Application (APP) is a software program that is capable of performing some or more specific functions. For example, instant messaging applications, video applications, audio applications, image capture applications, cloud desktop applications, and the like.

Video is a dynamic visual presentation consisting of successive image frames (frames), which are the smallest units that make up a video. Each image frame is a still image, and the human eye perceives dynamic motion and changes when a plurality of image frames are continuously played at a certain rate (e.g., 24FPS, 25FPS, or 30 FPS).

In the related art, in order to ensure the smoothness of video playing and improve the picture quality, video playing applications generally use a multi-buffer (buffer) technology to buffer the image frames obtained by decoding the video stream, and call a graphics processor (graphics processing unit, GPU) to perform rendering enhancement, resolution adjustment and other processes on the buffered image frames. However, with the development of communication technology and video playing technology, the network environment is improved greatly, the smoothness of video playing is guaranteed without buffering the image frames, and the image quality of the decoded image frames is also more likely to meet the viewing requirements of users. In the video playing process, if multiple buffers are always used to buffer image frames, additional operation amount and power consumption are brought (the image frames need to be copied from the buffered buffers to the displayed buffers and occupy more memory space), and the use of the GPU can further cause the power consumption of the terminal device to rise, so that the endurance of the terminal device is reduced.

In view of this, an embodiment of the present application provides a video playing method, in which a first processing mode is preferentially used for decoding and playing a video stream, in the first processing mode, an image frame obtained by decoding the video stream is not buffered by adopting a multi-buffer buffering technology, and a GPU is not invoked to process (e.g., render enhancement, resolution adjustment, etc.) the image frame obtained by decoding the video stream, in the case that an event for triggering a second processing mode is detected, the first processing mode is switched to the second processing mode, the second processing mode is used for decoding and playing the video stream, in the second processing mode, the image frame obtained by decoding the video stream is buffered by adopting the multi-buffer buffering technology, and/or the GPU is invoked to process the image frame obtained by decoding the video stream.

Fig. 1 is a schematic diagram of a video playing architecture of a terminal device in different processing modes according to an embodiment of the present application.

Taking the terminal equipment as a smart phone as an example, the terminal equipment is provided with an application capable of playing video, and the application capable of playing video can be an application capable of playing short video, film and television drama and other types, and the embodiment of the application is not limited to the application. The terminal device may start the video application in response to a first operation by the user. After the video application is started, the video application may play the first video specified by the user in response to a second operation of the user with respect to the application interface.

In some embodiments, the first video may have a different picture definition for selection by the user, which may correspond to a different video uniform resource locator (uniform resource locator, URL). The video application may access a video URL corresponding to a current picture definition of the first video over a network (e.g., a cellular communication network, a wireless local area network, etc.) to obtain a video stream of the first video at the picture definition.

The video stream of the first video may be a data stream in a Transport Stream (TS) format, or may be a data stream in another format, which is not limited in the embodiment of the present application.

As shown in fig. 1, in the first processing mode, the video stream of the first video may be decoded by a video codec (e.g., mediaCodec) to obtain an image frame (hereinafter referred to as a video image frame), and the decoded video image frame may be directly sent to a display buffer, for example, the decoded video image frame may be directly stored in a display screen by a display driver.

In some implementations, the display mechanism of the terminal device may be a double buffer mechanism or a triple buffer mechanism. If the display mechanism of the terminal device is a double buffer mechanism, that is, the display buffer includes two buffers for displaying (hereinafter referred to as display buffers for convenience of distinction), one display buffer is used for displaying the current picture, and the other display buffer is used as a backup buffer, fig. 1 illustrates that the display buffer includes two display buffers. If the display mechanism of the terminal device is a three-buffer mechanism, that is, the display buffer includes three display buffers, one display buffer is used for displaying the current picture, and the other two display buffers are used as backup buffers.

In some implementations, in the first processing mode, the specific process of displaying each decoded video image frame on the display screen may include:

i. one frame of video image frame obtained by decoding is stored in a display buffer.

And ii, a display composition service (SurfaceFlinger) reads image frames to be composed from the display buffer, wherein the image frames to be composed comprise video image frames obtained through decoding.

In some embodiments, during the process of playing the first video by the terminal device, the image frames stored in the display buffer may include only the decoded video image frames, and may include other image frames. I.e. the image frames to be synthesized may comprise only decoded video image frames, and may also comprise decoded video image frames as well as other image frames.

For example, as shown in fig. 2, the image frames stored in the display buffer include a decoded video image frame I1. As shown in fig. 3, the image frames stored in the display buffer include a decoded video image frame I2, an image frame I3 corresponding to a play control (hereinafter referred to as a control image frame), and an image frame I4 corresponding to a bullet screen (hereinafter referred to as a bullet screen image frame). The control image frame I3 and the barrage image frame I4 may be drawn and rendered by a video application, the control image frame I3 may include one or more play controls, for example, a pause control Ac1, a next set control Ac2, a barrage control Ac3, a double speed control Ac4, a screen capture control Ac5, an intelligent recognition control Ac6, and the like, and the barrage image frame I4 may include one or more barrage contents.

The surfaceflinger sends the image frames to be synthesized read from the display buffer to a hardware hybrid renderer (HWC) for image frame synthesis.

The hwc sends the synthesized result to a display driver for display on a display screen.

In some embodiments, if the image frame to be synthesized includes only the decoded video image frame, for example, the playing interface is full-screen playing, in which case the HWC does not need to synthesize the video image frame, the HWC may directly send the video image frame as a synthesis result (i.e., the HWC may not perform any processing on the video image frame) to the display driver, so as to display the video image frame on the display screen, where the display result is shown in fig. 4. The video playing in the situation does not need the participation of the GPU, and the video playing does not cause the GPU to generate power consumption.

In some embodiments, if the image frame to be synthesized includes other image frames besides the decoded video image frame, for example, the playing interface is not full-screen playing, and the image frame to be synthesized further includes the control image frame shown in fig. 3. In this case, the HWC may synthesize the decoded video image frame and the control image frame, and then send the synthesized result to the display driver to display on the display screen, where the display result is shown in fig. 5. Video playback in this scenario, the video application may need to call the GPU to render the control image frames (e.g., render text in the control image frames), but the power consumption of the GPU in this scenario is lower since only the GPU is called to render the control image frames and no GPU is called to process the video image frames.

As shown in fig. 1, in the second processing mode, the video stream of the first video may be decoded by a video codec to obtain video image frames, and the decoded video image frames may be stored in a buffer. The buffer area may include n buffers (hereinafter referred to as buffer buffers for convenience of distinction), where n is a positive integer, and the value of n may be set according to the memory size of the terminal device and the actual buffer requirement, which is not limited in the embodiment of the present application. For example, n is 40, and the buffer may buffer 40 video image frames.

In the second processing mode, the video application may further call the GPU to perform image enhancement, resolution adjustment, format conversion, and other processing on the video image frames stored in the buffer, where the processed video image frames may be transmitted to the display buffer for display. For example, the image enhancement may be a video application invoking a GPU to render each decoded video image frame to adjust one or more of brightness, contrast, color saturation, background, character, etc. of the video image frame. The resolution adjustment may be that the video application invokes the GPU to adjust the resolution of the decoded video image frames to better accommodate the resolution of the display screen. Because the GPU needs to be called to process each frame of video image frame to be played, the power consumption of the GPU in the situation is higher.

In some embodiments, in the second processing mode, the video application may also invoke the GPU to process the intercepted video clip or the intercepted video frame in response to a user clicking operation on the screen capture control Ac 5. For example, the video frames in the intercepted video clips are frames synthesized by the video image frames and the control image frames, or frames synthesized by the video image frames, the control image frames and the barrage image frames, and the finally obtained video clip frames only comprise the video image frames by calling the GPU to process the intercepted video clips. For example, the GPU may process the truncated video clip using recognition algorithms or models in the related art such that the resulting video clip frame includes only video image frames. The power consumption of the GPU in this case is high due to the need to process each video image frame in the video clip.

In some embodiments, in the second processing mode, the video application may also invoke information of each role, player information of each role, etc. in the GPU to identify the current picture in response to a click operation of the intelligent identification control Ac6 by the user. For example, the GPU may identify information for each role, player information for each role, in the current screen by running an existing generative artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) model. The power consumption of the GPU in this case is high due to the need to invoke the GPU to run the generated AI model data.

In some embodiments, the first processing mode is not configured with execution logic for tasks corresponding to the screen capture control Ac5 and the intelligent recognition control Ac6, and the second processing mode is configured with execution logic for tasks corresponding to the screen capture control Ac5 and the intelligent recognition control Ac6. Under the condition that clicking operation aiming at the screen capturing control Ac5 or the intelligent identification control Ac6 is detected, executing a task corresponding to the screen capturing control Ac5 or the intelligent identification control Ac6 in a second processing mode to ensure that the task corresponding to the screen capturing control Ac5 or the intelligent identification control Ac6 is smoothly executed.

The flow of decoding and playing the video stream by the terminal device is described in detail below with reference to fig. 6. Fig. 6 illustrates an example in which a terminal device is provided with a video application, and the video application may be a video playing application for playing short videos, movie shows, and the like, which is not limited by the embodiment of the present application.

S30, responding to the operation of playing the first video on the application interface of the video application by the user, and acquiring the video stream of the first video.

In some embodiments, after the video application is launched, the user may select a video to play, e.g., select to play a first video, at an application interface of the video application. The definition of the first video may be a definition selected by a user, or may be a definition preset by a video application. The video stream of the first video acquired by the terminal equipment is a video stream corresponding to the selected definition. For example, the video streams with different definitions correspond to different video URLs, and the video application in the terminal device can access the corresponding video URLs through the network to obtain the video stream corresponding to the selected definition.

S31, determining a processing mode of a video stream of the first video.

In some embodiments, after obtaining the video stream of the first video, the terminal device may determine a processing mode of the video stream of the first video first, and then process the video stream of the first video based on the determined processing mode to play the first video.

For example, if the score of the processor of the terminal device is low, for example, the running score of the central processing unit (central processing unit, CPU) or the System on Chip (SoC) is lower than a preset score, or the memory is smaller (for example, lower than a first preset capacity), or the battery capacity is smaller (for example, lower than a second preset capacity), or the remaining battery power is lower (for example, lower than a preset percentage), the terminal device determines that the processing mode of the video stream of the first video is the first processing mode. If the score of the processor of the terminal device is higher (the running score of the CPU or the SoC is greater than or equal to the preset score), or the memory is larger (for example, greater than or equal to the first preset capacity), or the battery capacity is larger (for example, greater than or equal to the second preset capacity), or the remaining battery power is higher (for example, greater than or equal to the preset percentage), the terminal device determines that the processing mode of the video stream of the first video is the second processing mode.

For another example, if the frequency/duration of video watching by the user using the terminal device every day is high (for example, greater than or equal to a preset frequency/preset duration), or the number of operations on the video playing interface is small (for example, the number of operations is less than or equal to a preset number of times) when watching the video, or the network speed connected when watching the video is good (for example, the network speed is greater than or equal to a preset speed), the terminal device determines that the processing mode of the video stream of the first video is the first processing mode. If the frequency/duration of watching the video by the user using the terminal device every day is low (for example, less than the preset frequency/preset duration), or the operation number of the video playing interface is more (for example, the operation number is greater than the preset number) when watching the video, or the network speed connected when watching the video is poor (for example, the network speed is less than the preset speed), the terminal device determines that the processing mode of the video stream of the first video is the second processing mode.

S32, if the processing mode of the video stream of the first video is determined to be the first processing mode, decoding the video stream of the first video, and storing the decoded video image frames in a display buffer.

In some embodiments, the display buffer may include two or three buffers (referred to as display buffers). If the processing mode of the video stream of the first video is determined to be the first processing mode, the terminal equipment decodes the video stream of the first video and stores the video image frames obtained by decoding in a display buffer. For example, a video codec in the terminal device decodes a video stream of the first video, and stores the decoded video image frames in a display buffer.

In some embodiments, in the first processing mode, the terminal device may further switch the processing mode of the video stream from the first processing mode to the second processing mode in response to a preset trigger event, for example, a click operation of a user on a designated control in the video playing interface. For example, the specified controls may include a screen capture control Ac5, a smart recognition control Ac6, and the like.

And S33, if the processing mode of the video stream of the first video is determined to be the second processing mode, decoding the video stream of the first video, and storing the decoded video image frames in a buffer.

In some embodiments, the buffer may include a plurality of buffers (referred to as buffer buffers) for buffering video image frames, and if it is determined that the processing mode of the video stream of the first video is the second processing mode, the terminal device decodes the video stream of the first video and stores the decoded video image frames in the buffer including the plurality of buffer buffers. Under the condition that the network is delayed, video image frames stored in a plurality of buffer buffers are copied to a display buffer in sequence for display, so that the fluency of video playing can be improved, and the situation of video blocking is avoided. For example, a video codec in the terminal device decodes the video stream of the first video, and a video image frame obtained by decoding by the video codec is stored in the buffer.

In some embodiments, each buffer in the buffer buffers may be used to store one video image frame. The buffer can also store information for representing the frame playing sequence of the video image frames, such as decoding time stamp, frame number and the like, so that the video image frames buffered by the buffer can be copied to the display buffer conveniently according to the information of the frame playing sequence, and the video image frames can be played sequentially.

And S34, transmitting the video image frames stored by the buffer to the display buffer.

In some embodiments, the terminal device may copy the video image frames stored in the buffer to the display buffer, so as to transmit the video image frames stored in the buffer to the display buffer. For example, a processor in the terminal device may copy a video image frame from a buffer to a display buffer in response to a play request by a video application for the video image frame.

In some embodiments, the terminal device may further perform image enhancement, resolution adjustment, and other processing on the video image frames stored in the buffer, and retransmit the processed video image frames to the display buffer.

S35, surfaceFlinger, reading an image frame to be synthesized from a display buffer, and sending the image frame to be synthesized to the HWC, wherein the image frame to be synthesized comprises a video image frame.

S36, the HWC synthesizes the image frames to be synthesized and sends the synthesis result to the display driver.

S37, the display driver displays the synthesized result on a display screen.

In some embodiments, if the display buffer includes only video image frames, i.e., the image frames to be synthesized include only video image frames, surfaceFlinger reads the video image frames from the display buffer and sends them to the HWC, the HWC may directly send the video image frames to the display driver (i.e., the HWC does not synthesize the video image frames), and the display driver displays the video image frames on the display screen. If the display buffer includes other image frames (e.g., control image frame, bullet screen image frame) in addition to the video image frame, surfaceFlinger reads the image frames (video image frame and other image frames) to be synthesized from the display buffer and sends them to the HWC, the HWC synthesizes the received image frames, and sends the synthesized result to the display driver, and the display driver displays the synthesized result on the display screen.

As shown in fig. 7, in order to implement determination of a processing mode of a video stream, a mode selection frame may be configured in a terminal device, and the terminal device may perform decoding and playing of the video stream in a first processing mode or a second processing mode based on mode selection information output by the mode selection frame. For example, the video application may obtain a video stream of the first video in response to an operation of playing the first video, the first video may call the video codec to decode the video stream of the first video, and the video codec may store the decoded video image frame in a display buffer (corresponding to the first processing mode) or in a buffer (corresponding to the second processing mode) based on mode selection information output by the mode selection framework.

For example, taking the mode selection information as a numerical value as an example, the terminal device adopts the first processing mode to decode and play the video stream when the mode selection information output by the mode selection frame is information 1, and adopts the second processing mode to decode and play the video stream when the mode selection information output by the mode selection frame is information 2. In other embodiments, the mode selection information may be represented by other characters, which is not limited to the above examples in practical application.

In some embodiments, the mode selection framework may output corresponding mode selection information based on the built device performance portraits, device usage portraits, video playback fluency, or particular operations received while the video was being played, etc.

In some embodiments, the device performance portrayal may include a plurality of performance information, which may include parameter information such as a processor, memory, battery, etc. in the terminal device. For example, the mode selection framework may determine a default processing mode for video playback to be the first processing mode or the second processing mode based on the device performance portraits. For example, for the situation that the performance score of the processor is lower, the memory is smaller, the battery capacity is smaller, or the remaining battery power is lower, the mode selection information output by the mode selection framework is information 1, so as to trigger the terminal device to default adopt the first processing mode to decode and play the video stream.

For example, a low processor performance score may be a running score of the processor below a predetermined score, which characterizes the processor (with the GPU integrated) as potentially being time consuming to process video image frames, thus suggesting decoding and playing of the video stream in the first processing mode. The smaller memory may be a memory capacity lower than the first preset capacity, which indicates that there may be insufficient memory space as a buffer, so it is recommended to use the first processing mode to decode and play the video stream. The smaller battery capacity may be that the battery rated capacity is lower than the second preset capacity, and the battery rated capacity is lower than the second preset capacity to indicate that the time for continuously playing the video is shorter, so that the decoding and playing of the video stream are suggested to be performed by adopting the first processing mode. The low battery remaining power may be that the battery remaining power is lower than a preset percentage, and the fact that the battery remaining power is lower than the preset percentage indicates that the time for continuously playing the video is short, so that decoding and playing of the video stream by adopting the first processing mode are recommended. The first preset capacity, the second preset capacity and the preset percentage can be set according to actual requirements, which is not limited in the embodiment of the present application.

In some embodiments, if the mode selection framework determines a default processing mode of video playing based on a plurality of performance information, a weight coefficient may be set for each performance information, for example, a total score is obtained based on a score (pre-constructed score rule) of each performance information and a corresponding weight coefficient, and then the total score is compared with a preset score to determine that the default processing mode is the first processing mode or the second processing mode.

In some embodiments, the device usage representation may include a plurality of historical video playing related information, where the plurality of historical video playing related information may include a frequency/duration of video being watched by a user using the terminal device per unit time (e.g., per day), a number of times the user operates a control in a video playing interface during video playing, a network speed to which the user is connected when the user views the video using the terminal device, and so on. For example, the mode selection framework may determine a default processing mode for video playback as the first processing mode or the second processing mode based on the device using the portraits. For example, for the case that the frequency/duration of video watching per day is higher (greater than the preset frequency or the preset duration), or the number of times of operating the control in the video playing interface is smaller (less than the preset number of times) for video watching per day, or the network speed connected during video watching is better (the network speed is greater than the preset speed), the mode selection information output by the mode selection framework is information 1, so as to trigger the terminal device to perform decoding and playing of the video stream by default in the first processing mode, thereby reducing the power consumption of video playing.

For example, a higher frequency/duration of video viewing per day characterizes a greater amount of device power for video playback, and in order to increase the duration that the device can continue to play video, it is recommended to use the first processing mode for decoding and playing video streams. For another example, watching the video daily less often than not operating the controls in the video playback interface characterizes a less probable click on the designated controls (e.g., the designated controls may include a screen capture control Ac5, a smart identification control Ac6, etc.), thus suggesting that the first processing mode be employed for decoding and playback of the video stream. For another example, the network speed connected during video watching preferably characterizes that video playing is blocked with a small probability under the condition that video image frame buffering is not performed, so that decoding and playing of the video stream are suggested to be performed by adopting the first processing mode.

In some embodiments, if the mode selection framework determines a default processing mode of video playing based on a plurality of historical video playing related information, a weight coefficient may be set for each historical video playing related information, for example, a total score is obtained based on a score (pre-constructed score rule) of each historical video playing related information and a corresponding weight coefficient, and then the total score is compared with a preset score to determine that the default processing mode is the first processing mode or the second processing mode.

In some embodiments, the smoothness of video playing may refer to whether a picture is blocked in the video playing process, and a specific operation received during video playing may refer to whether an operation triggering the invoking of the GPU is received, an operation triggering the invoking of the GPU to process a video image frame, or whether a clicking operation for a specified control is received, etc.

For example, the specific operation may be an operation of clicking the barrage control Ac3 to open a barrage function, clicking the screen capture control Ac5 to capture a video clip, clicking the intelligent recognition control Ac6 to recognize a character in a picture, switching the picture definition to a preset resolution (e.g., 4K resolution), and the like, which may trigger the operation of invoking the GPU. In other embodiments, the specific operation may not include clicking on the barrage control Ac3 to open the barrage function.

For another example, the specific operation may further include clicking the intelligent recognition control Ac6 to recognize a character in a picture, switching the definition of the picture to a preset resolution, and the like, which may trigger the operation of invoking the GPU to process the video image frame.

In some embodiments, when the video playing is blocked, or a specific operation is received during the video playing, the mode selection information output by the mode selection framework is information 2, so as to trigger the terminal device to perform decoding and playing of the video stream by adopting the second processing mode. Assuming that the frame rate of video playback is 25FPS, if it is detected that the display interval between two video image frames exceeds 40ms, it can be considered that picture sticking occurs in video playback. For example, when video playing, the specific operation received needs to call the GPU or other hardware resources to execute the task corresponding to the specific operation, the task processing has certain time consumption, and the equipment resources (hardware resources or network resources) are occupied, and the smoothness of video playing can be guaranteed to the greatest extent by adopting the second processing mode to decode and play the video stream.

In some embodiments, the terminal device may also be configured with a control or interface for setting the processing mode. For example, an interface for setting a processing mode is included in the setting application of the terminal device. The user can set the interface to automatically select the processing mode or manually select the processing mode. The selection of the automatic processing mode may refer to the selection of the processing mode being automatically performed based on a mode selection framework, and the selection of the manual processing mode may refer to the processing mode being set by a user to be used by default by the video application (e.g., to use the first processing mode by default or to use the second processing mode by default).

In some embodiments, the mode selection framework may also determine a processing mode for the video application to default to based on the mode selection model by constructing training data and adding a tag to each training data, which may include performance information and/or historical video playback related information, training based on the tagged training data. For example, any possible algorithm such as decision tree algorithm, support vector machine algorithm, etc. may be used to train to obtain the mode selection model. The selection of the machine learning algorithm can be performed according to actual use requirements, and the embodiment of the application is not limited.

For example, a label 1 (first processing mode) is labeled on the training data a, a label 2 (second processing mode) is labeled on the training data b, the training data a and the training data b are input as a mode selection model, the labels 1 and 2 are set as desired results, and initial values are given to the connection weights between the neurons of the respective layers of the neural network. And according to the connection weight values among the nerve cells of each layer, obtaining an actual output result corresponding to the training data by adopting a nerve network algorithm. After the expected result and the actual output result are processed by adopting the error function, the connection weight between the neurons of each layer is adjusted according to the processing result, the adjusted connection weight between the neurons of each layer is obtained, the training process is repeated, and the trained mode selection model can be obtained after multiple rounds of iterative training.

For example, taking the example that the part of training data includes performance information and the part of training data includes information related to historical video playing, training and deployment of the mode selection model can be completed before the terminal equipment leaves the factory. After the terminal equipment leaves the factory, under the condition that the relevant information of the historical video playing of the preset quantity is not collected, the mode selection framework can input the performance information of the terminal equipment into the mode selection model to obtain a default processing mode of the video stream. Under the condition that the preset quantity of the historical video playing related information is collected, a mode selection model deployed by the terminal equipment can be trained based on the collected historical video playing related information, and the follow-up mode selection frame can input the video playing related information of the terminal equipment into the mode selection model to obtain a default processing mode of the video stream.

For another example, taking the case that the training data includes the historical video playing related information, in the case that the preset amount of the historical video playing related information is not collected (that is, the mode selection model that is effective is not trained yet), the mode selection framework may take the processing mode preset by the developer as the default processing mode of the video stream. For example, the developer may set the first processing mode or the second processing mode to a default processing mode of the video stream in advance based on the performance information of the terminal. Under the condition that the historical video playing related information of the preset quantity is collected, a mode selection model can be obtained through training based on the collected historical video playing related information, and the follow-up mode selection frame can input the video playing related information of the terminal equipment into the mode selection model to obtain a default processing mode of the video stream. For example, the mode selection framework may input information about video playing of the terminal device on the previous day into the mode selection model, so as to obtain a processing mode of the video stream defaulted on the current day.

In some embodiments, if the mode selection framework determines, based on the mode selection model, that the processing mode used by the video application by default is the first processing mode, during video playing, the mode selection framework may also switch the first processing mode to the second processing mode if a screen is stuck or a specific operation is received (e.g., an operation that may trigger a call to the GPU or an operation that may trigger a call to the GPU to process a video image frame).

Referring to fig. 8, a video playing method according to an embodiment of the present application is shown. The video playing method can be applied to terminal equipment, and the terminal equipment comprises a display screen. The video playing method can comprise the following steps:

s51, responding to the first operation, and starting the video application.

For example, the first operation is an operation of clicking an icon of a video application of the desktop, and the terminal device may start the video application in response to the operation of clicking the icon of the video application of the desktop. In other embodiments of the present application, the first operation may also be an operation of calling the video application in other applications, for example, an application running in the foreground of the terminal device is an instant messaging application, and if the user starts the video application through a touch operation of a control, an icon, an option, etc. in the instant messaging application, the video application is started, and an application interface of the video application is entered.

S52, responding to a second operation of playing the first video on the video application, and acquiring a video stream of the first video.

For example, the second operation may refer to an operation of clicking to play the first video on an application interface of the video application, and the terminal device may obtain a video stream of the first video in response to the operation of clicking to play the first video.

S53, processing the video stream by adopting a first processing mode, wherein the first processing mode comprises the steps of decoding the video stream and storing the decoded video image frames in a display buffer area for transmission and display.

In some embodiments, the display buffer may include two or three display buffers, and embodiments of the present application do not limit the number of display buffers included in the display buffer.

In some embodiments, the terminal device may be configured to process the video stream by default using a first processing mode, where in the first processing mode, the GPU is not used to process the decoded video image frame, so as to save power consumption of video playing. The first processing mode may include decoding the video stream and storing the decoded video image frames in a display buffer for display. For example, the image frames (video image frames, or video image frames and other image frames) to be synthesized are read from the display buffer by SurfaceFlinger and sent to the HWC for synthesis and display.

S54, responding to a preset trigger event, processing the video stream by adopting a second processing mode, wherein the second processing mode comprises the steps of decoding the video stream, storing video image frames obtained by decoding in a buffer area, and copying the video image frames stored in the buffer area to a display buffer area for display.

In some embodiments, the buffer area may include a plurality of buffers for buffering video image frames, and the number of buffers included in the buffer area may be set according to actual requirements, which is not limited in the embodiments of the present application.

In some embodiments, the preset trigger event may include a video image frame play stuck event or an event that a video play interface of a video application receives a preset operation. For example, the preset operation includes an operation of triggering the GPU to process the video image frame, or the preset operation may include a touch operation for a target control of a plurality of controls in a video playing interface provided by the video application, where the target control is a control that can trigger the GPU to be invoked. For example, controls that may trigger the invocation of the GPU include a barrage control Ac3, a screen capture control Ac5, and an intelligent recognition control Ac6.

In some embodiments, the processing mode used by the terminal device by default may also be determined based on the resolution of the first video, where the first video is processed by default in the first processing mode when the resolution of the first video is not the preset resolution and the second processing mode when the resolution of the first video is the preset resolution by acquiring the media information of the video stream of the first video (the media information includes the resolution of the first video). The preset resolution may be an ultra-high definition resolution, for example, a 4K resolution, and for video playing with the ultra-high definition resolution, the GPU may need to be called to perform resolution adjustment on the video image frames obtained by decoding, so as to ensure smoothness of video playing and smooth resolution adjustment, and the second processing mode is adopted to process the video stream.

In some embodiments, the processing mode used by default may refer to the processing mode used initially for the video stream.

In some embodiments, the processing mode used by the terminal device by default may also be determined based on the capability information of the terminal device. For example, by obtaining performance information of the terminal device, the performance information may include one or more of a processor model, a memory size, a battery rating, and a battery remaining capacity, and a processing mode for default use is determined based on the performance information.

In some embodiments, the processing mode used by the terminal device by default may also be determined based on historical video play related information of the terminal device. For example, by acquiring historical video playing related information of the terminal device, the historical video playing related information includes one or more of video playing frequency in unit time, video playing duration in unit time, operation times of a control in a video playing interface in unit time, and network speed of a network connected when the terminal device plays video, and a default processing mode is determined based on the historical video playing related information.

In some embodiments, a mode selection model may also be obtained through training, and a processing mode used by the terminal device by default is determined based on the mode selection model. The training data of the mode selection model may comprise performance information of the terminal device and/or historical video play related information of the terminal device.

Referring to fig. 9, a terminal device 100 according to an embodiment of the present application will be described. The terminal device in the embodiment of the present application may be a device including a touch screen, such as a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a large screen, an intelligent television, a netbook, a cellular phone, a Personal Digital Assistant (PDA), an augmented reality (augmented reality, AR) \virtual reality (VR) device, and the like, and the embodiment of the present application does not limit the specific form of the terminal device. Referring to fig. 9, fig. 9 is a schematic structural diagram of a terminal device 100 according to an embodiment of the present application.

The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the application, terminal device 100 may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

In addition, an operating system is run on the components. Such as the iOS operating system developed by apple corporation, the Android open source operating system developed by google corporation, the Windows operating system developed by microsoft corporation, etc.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instructions or data, it may be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an integrated circuit (inter-INTEGRATED CIRCUIT, I2C) interface, an integrated circuit built-in audio (inter-INTEGRATED CIRCUIT SOUND, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others.

The wireless communication function of the terminal device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the terminal device 100 may be used to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas. For example, the antenna 1 may be multiplexed into a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 can amplify the signal modulated by the modem processor, and convert the signal into electromagnetic waves through the antenna 1 to radiate. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be provided in the same device as at least some of the modules of the processor 110.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN) (e.g., wireless fidelity (WIRELESS FIDELITY, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), near field communication (NEAR FIELD communication, NFC), infrared (IR), etc., applied on the terminal device 100. The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signals, filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, frequency modulate it, amplify it, and convert it to electromagnetic waves for radiation via the antenna 2.

In some embodiments, antenna 1 and mobile communication module 150 of terminal device 100 are coupled, and antenna 2 and wireless communication module 160 are coupled, such that terminal device 100 may communicate with a network and other devices via wireless communication techniques. The wireless communication techniques can include the Global System for Mobile communications (global system for mobile communications, GSM), general packet radio service (GENERAL PACKET radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC, FM, and/or IR techniques, among others. The GNSS may include a global satellite positioning system (global positioning system, GPS), a global navigation satellite system (global navigation SATELLITE SYSTEM, GLONASS), a beidou satellite navigation system (beidou navigation SATELLITE SYSTEM, BDS), a quasi zenith satellite system (quasi-zenith SATELLITE SYSTEM, QZSS) and/or a satellite based augmentation system (SATELLITE BASED AUGMENTATION SYSTEMS, SBAS).

The terminal device 100 implements display functions through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos, and the like. The display 194 includes a display panel. The display panel may employ a Liquid Crystal Display (LCD) screen (liquid CRYSTAL DISPLAY), an organic light-emitting diode (OLED), an active-matrix organic LIGHT EMITTING diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microLed, micro-OLED, a quantum dot light-emitting diode (QLED), or the like. In some embodiments, the terminal device 100 may include 1 or N display screens 194, N being a positive integer greater than 1. The display screen 194 in the embodiment of the present application may be a touch screen. I.e. the display 194 has the touch sensor 180K integrated therein.

The internal memory 121 may include one or more random access memories (random access memory, RAM) and one or more non-volatile memories (NVM).

The random access memory may include a static random-access memory (SRAM), a dynamic random-access memory (dynamic random access memory, DRAM), a synchronous dynamic random-access memory (synchronous dynamic random access memory, SDRAM), a double data rate synchronous dynamic random-access memory (doubledata rate synchronous dynamic random access memory, DDR SDRAM, such as fifth generation DDR SDRAM is commonly referred to as DDR5 SDRAM), etc.;

the nonvolatile memory may include a disk storage device, a flash memory (flash memory).

The FLASH memory may include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. divided according to an operation principle, may include single-level memory cells (SLC-LEVEL CELL), multi-level memory cells (multi-LEVEL CELL, MLC), triple-level memory cells (LEVEL CELL, TLC), quad-LEVEL CELL, QLC), etc. divided according to a memory cell potential order, may include general FLASH memory (english: universal FLASH storage, UFS), embedded multimedia memory card (eMMC) MEDIA CARD, eMMC), etc. divided according to a memory specification.

The random access memory may be read directly from and written to by the processor 110, may be used to store executable programs (e.g., machine instructions) for an operating system or other on-the-fly programs, may also be used to store data for users and applications, and the like.

The nonvolatile memory may store executable programs, store data of users and applications, and the like, and may be loaded into the random access memory in advance for the processor 110 to directly read and write.

The external memory interface 120 may be used to connect an external nonvolatile memory to realize expansion of the memory capability of the terminal device 100. The external nonvolatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

The video playing methods in the above embodiments may be implemented in the terminal device 100 having the above hardware structure.

The operating system of the terminal device may employ a layered architecture, an event driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of the application, an Android system with a layered architecture is taken as an example, and the software structure of terminal equipment is illustrated. Fig. 10 is a software configuration block diagram of the terminal device. The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. Taking And roid systems as an example, in some embodiments, the Android system is divided into four layers, namely, an application layer, an application Framework layer (Framework), a Hardware Abstraction Layer (HAL), and a Kernel layer (Kernel) from top to bottom.

The application layer may include a series of application packages. For example, the application package may include applications for cameras, gallery, calendar, phone calls, maps, navigation, WLAN, bluetooth, music, video, short messages, etc.

The application framework layer provides an application programming interface (Application Programming Interface, API) and programming framework for the application of the application layer. The application framework layer includes a number of predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like. The application framework layer may also include a mode selection framework that may be used to determine a processing mode of the video stream.

The hardware abstraction layer may include HWC, which may be used to perform the synthesis of the image frames.

The kernel layer is a layer between hardware and software. The kernel layer may include display drivers, camera drivers, audio drivers, sensor drivers, and the like.

The kernel layer is the core of the operating system of the terminal equipment, is the first layer of software expansion based on hardware, provides the most basic function of the operating system, is the basis of the operation of the operating system, is responsible for managing the processes, the memory, the device driver, the files and the network system of the system, and determines the performance and the stability of the system. For example, the kernel may determine the time an application is operating on a certain portion of hardware.

The kernel layer includes hardware-closely related programs, such as interrupt handlers, device drivers, etc., basic, common, higher frequency of operation modules, such as clock management modules, process scheduling modules, etc., and critical data structures. The kernel layer may be provided in the processor or cured in an internal memory.

The present embodiment also provides a computer-readable storage medium having stored therein computer instructions that, when executed on the terminal device 100, cause the terminal device 100 to perform the above-described related method steps to implement the video playback method in the above-described embodiments.

The present embodiment also provides a computer program product which, when run on a computer, causes the computer to perform the above-mentioned related steps to implement the video playing method in the above-mentioned embodiments.

In addition, the embodiment of the application also provides a device which can be a chip, a component or a module, and the device can comprise a processor and a memory which are connected, wherein the memory is used for storing computer execution instructions, and when the device runs, the processor can execute the computer execution instructions stored in the memory so that the chip can execute the video playing method in the method embodiments.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided by the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated unit may be stored in a readable storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.

The foregoing embodiments are merely for illustrating the technical solution of the present application, but not for limiting the same, and although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that modifications may be made to the technical solution described in the foregoing embodiments or equivalents may be substituted for parts of the technical features thereof, and such modifications or substitutions do not depart from the spirit of the corresponding technical solution from the scope of the technical solution of the embodiments of the present application.

Claims (13)

1. A video playing method applied to a terminal device, the method comprising:

Responsive to a first operation, displaying a first interface of the video application;

In response to a second operation at the first interface, starting to play a first picture of a first video at a first time, wherein image data of the first video is not processed by a first processor between the first time and a second time;

at the second moment, responding to a preset trigger event, and processing the image data of the first video by using the first processor;

And after the second moment, playing a second picture of the first video based on the image data of the first video processed by the first processor, wherein the power consumption of the first processor between the first moment and the second moment is smaller than the power consumption of the first processor after the second moment.

2. The video playing method according to claim 1, wherein the processing, with the first processor, the image data of the first video in response to a preset trigger event includes:

responsive to a click operation for a first control, image data of the first video is processed with the first processor.

3. The video playback method of claim 2, wherein the first control comprises any one of a barrage control, a screen capture control, an intelligent recognition control, and a resolution selection control.

4. The video playing method according to claim 1, wherein the processing, with the first processor, the image data of the first video in response to a preset trigger event includes:

Acquiring a clamping event for playing the first video;

And processing the image data of the first video by using the first processor based on the clamping event.

5. The video playback method of any one of claims 1-4, wherein the display parameters of the second picture are different from the display parameters of the first picture, the display parameters comprising at least one of brightness, contrast, color saturation, and resolution.

6. The video playing method of claim 1, wherein the starting to play the first picture of the first video at the first time in response to the second operation at the first interface comprises:

And responding to the second operation on the first interface, if the resolution of the first video to be played is not the preset resolution, starting to play a first picture of the first video at the first moment, wherein the image data of the first picture is not processed by the first processor.

7. The video playback method of claim 6, wherein the method further comprises:

and responding to the second operation on the first interface, if the resolution of the first video to be played is the preset resolution, starting to play a third picture of the first video at the first moment, wherein the image data of the third picture is processed by the first processor.

8. The video playing method of claim 1, wherein the starting to play the first picture of the first video at the first time in response to the second operation at the first interface comprises:

And responding to the second operation on the first interface, if the performance information of the terminal equipment meets the preset condition, starting to play a first picture of the first video at the first moment, wherein the image data of the first picture is not processed by the first processor, and the performance information comprises one or more of a processor model, a memory size, a rated battery capacity and a residual battery capacity.

9. The video playback method of claim 8, wherein the method further comprises:

And responding to the second operation on the first interface, if the performance information of the terminal equipment does not meet the preset condition, starting to play a third picture of the first video at the first moment, wherein the image data of the third picture is processed by the first processor.

10. The video playing method of claim 1, wherein the starting to play the first picture of the first video at the first time in response to the second operation at the first interface comprises:

Responding to the second operation on the first interface, if the historical video playing related information of the terminal equipment meets a preset condition, starting to play a first picture of the first video at the first moment, wherein the image data of the first picture is not processed by the first processor, and the historical video playing related information comprises one or more of video playing frequency in unit time, video playing duration in unit time, operation times of a control in the video playing interface in unit time and network speed of a network connected when the terminal equipment plays video.

11. The video playback method of claim 10, wherein the method further comprises:

And responding to the second operation on the first interface, if the historical video playing related information of the terminal equipment does not meet the preset condition, starting to play a third picture of the first video at the first moment, wherein the image data of the third picture is processed by the first processor.

12. A terminal device, characterized in that the terminal device comprises a display screen, a memory and a processor;

The display screen, the memory and the processor are coupled;

the memory is used for storing program instructions;

The processor is configured to read the program instructions stored in the memory to implement the video playing method according to any one of claims 1 to 11.

13. A computer readable storage medium having stored therein computer readable instructions which when executed by a processor implement the video playback method of any one of claims 1 to 11.