HK40086735A - A data processing method, device, equipment and readable storage medium - Google Patents

Description

A data processing method, apparatus, device, and readable storage medium

Technical Field

This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and readable storage medium.

Background Technology

With the rapid development of computer and multimedia technologies, an increasing number of intelligent applications are emerging, enriching daily life. These intelligent applications are typically deployed on terminal devices, allowing users to run them.

It should be understood that when smart applications are deployed on terminal devices, the terminal devices can render application-related data to output application visuals associated with the smart applications. Among these, some smart applications that require low latency and high smoothness (such as cloud gaming applications, real-time video communication applications, and other media applications) have higher requirements for the rendering performance of the terminal devices.

In related technologies, terminal devices are pre-configured with a fixed set of rendering parameters to render different smart applications. However, due to factors such as the terminal device's chip, system version, or others, different rendering parameters result in varying rendering performance on different terminal devices. Therefore, pre-configuring a fixed set of rendering parameters lacks specificity and accuracy; the parameters may not be suitable for the specific terminal device, leading to lower rendering performance and an inability to meet the rendering needs of different smart applications.

Summary of the Invention

This application provides a data processing method, apparatus, device, and readable storage medium that can optimize the rendering parameters of a terminal device and improve its rendering performance.

One embodiment of this application provides a data processing method, including:

Obtain K combinations of rendering parameters for the terminal device; K is a positive integer; each of the K combinations of rendering parameters contains one or more rendering parameters that affect the rendering performance of the terminal device; each rendering parameter that affects the rendering performance of the terminal device contains a combination of rendering parameters _Si , where i is a positive integer;

The test stream is input into the test device. Based on the rendering data of the test stream by the test device, the reference value of the rendering effect corresponding to the rendering parameter combination _Si is determined. The test device refers to the terminal device that switches the device rendering parameters of the terminal device to the rendering parameter combination _Si . The reference value of the rendering effect corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si .

Once the reference values for the rendering effects corresponding to each combination of rendering parameters are determined, the optimal combination of rendering parameters for the terminal device is determined from among the K combinations of rendering parameters based on the K reference values for rendering effects.

One embodiment of this application provides a data processing apparatus, including:

The combination acquisition module is used to acquire K combinations of rendering parameters for the terminal device; each of the K combinations of rendering parameters contains one or more influencing rendering parameters; each influencing rendering parameter refers to a rendering parameter that affects the rendering performance of the terminal device; K is a positive integer; the K combinations of rendering parameters contain a combination of rendering parameters _Si , where i is a positive integer;

The bitstream input module is used to input the test bitstream into the test device; the test device refers to the terminal device that switches the device rendering parameters of the terminal device to the rendering parameter combination _Si .

The reference value determination module is used to determine the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test bitstream by the test equipment; the rendering effect reference value corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si .

The optimal combination determination module is used to determine the optimal rendering parameter combination for the terminal device from among the K rendering parameter combinations, based on the K rendering effect reference values corresponding to each rendering parameter combination.

In one embodiment, the combined acquisition module may include:

The feature acquisition unit is used to acquire R device rendering features configured for the terminal device; R is a positive integer;

The parameter set acquisition unit is used to acquire the set of device rendering parameters corresponding to each of the R device rendering features, and thus obtain the R device rendering parameter sets.

The parameter combination unit is used to combine the device rendering parameters contained in the R sets of device rendering parameters based on the parameter combination rules to obtain K rendering parameter combinations.

In one embodiment, the parameter combination unit may include:

The first combination determination subunit is used to determine the combination of the first target device rendering parameter in the target device rendering parameter set and the second target device rendering parameter in the remaining device rendering parameter set as the rendering parameter combination _Si , based on the parameter combination rules; the target device rendering parameter set is any one of the R device rendering parameter sets; the remaining device rendering parameter set is the set of device rendering parameters other than the target device rendering parameter set among the R device rendering parameter sets; the first target device rendering parameter refers to any one of the device rendering parameters in the target device rendering parameter set; the second target device rendering parameter refers to any one of the device rendering parameters in the remaining device rendering parameter set;

The second combination determination subunit is used to determine the combination of the remaining device rendering parameters in the target device rendering parameter set and the second target device rendering parameters as the remaining rendering parameter combination; the remaining device rendering parameters are any device rendering parameters in the target device rendering parameter set other than the first target device rendering parameters.

The parameter combination determination subunit is used to determine K rendering parameter combinations based on the rendering parameter combination _Si and the remaining rendering parameter combinations.

In one embodiment, the R device rendering features include device rendering feature _Hj ; j is a positive integer; the device rendering parameter set corresponding to each device rendering feature includes the device rendering parameter set _Pj corresponding to device rendering feature _Hj ;

The parameter combination determination sub-unit is also specifically used to determine the rendering parameter combination _Si and the remaining rendering parameter combinations as the initial rendering parameter combination, and to determine the set of the initial rendering parameter combinations as the initial rendering parameter combination set.

The parameter combination determines the sub-unit and is also specifically used to obtain the parameter priority of each device rendering parameter in the device rendering parameter set _Pj corresponding to the device rendering feature _Hj ;

The parameter combination determines the sub-unit, and is also specifically used to sort the device rendering parameters in the device rendering parameter set _Pj according to the order of priority of each device rendering parameter in the device rendering parameter set _Pj , so as to obtain the parameter sequence;

The parameter combination determination subunit is also specifically used to prune the initial rendering parameter combination set based on the parameter sequence to obtain the pruned rendering parameter combination set, and to determine K rendering parameter combinations based on the pruned rendering parameter combination set.

In one embodiment, the parameter combination determining subunit is also specifically used to determine the device rendering parameter located at the end of the sequence as the rendering parameter to be pruned.

The parameter combination determination sub-unit is also specifically used to determine the initial rendering parameter combination that contains the rendering parameters to be pruned in the initial rendering parameter combination set as the rendering parameter combination to be pruned.

The parameter combination determination sub-unit is also specifically used to delete the rendering parameter combinations to be pruned from the initial rendering parameter combination set, so as to obtain the pruned rendering parameter combination set.

In one embodiment, the parameter combination determining subunit is also specifically used to update the pruning attribute of the device rendering feature _Hj from an unpruned attribute to a pruned attribute.

The parameter combination determines the sub-unit and is also specifically used to traverse R device rendering features;

The parameter combination determines the sub-unit, and is also specifically used to determine the set of pruned rendering parameter combinations as K rendering parameter combinations if the pruning attributes of each of the R device rendering features are all pruned attributes.

The parameter combination determination sub-unit is also specifically used to determine that if among R device rendering features there is a device rendering feature with an unpruned attribute among the pruned attributes, the device rendering feature with the unpruned attribute is determined as an unpruned rendering feature. Based on the device rendering parameter set corresponding to the unpruned rendering feature, the set of pruned rendering parameter combinations is pruned, and K rendering parameter combinations are determined based on the pruning result obtained from the pruning process.

In one embodiment, the R device rendering features include device input frame rate features; the device rendering parameter set corresponding to each device rendering feature includes the device input frame rate set corresponding to the device input frame rate feature.

The parameter set acquisition unit may include:

The type acquisition subunit is used to acquire the media encoding types for which the terminal device has encoding and decoding permissions;

The prediction value determination subunit is used to obtain the maximum resolution corresponding to the terminal device and determine the maximum decoding frame rate prediction value corresponding to the terminal device based on the media encoding type and the maximum resolution.

The maximum frame rate determination subunit is used to obtain the screen refresh rate corresponding to the terminal device and determine the minimum value between the maximum decoded frame rate prediction value and the screen refresh rate as the maximum input frame rate of the terminal device.

The frame rate set determination subunit is used to determine the device input frame rate set corresponding to the device input frame rate characteristics based on the maximum input frame rate.

In one embodiment, the frame rate set determining subunit is further configured to obtain an initial configuration frame rate set; the initial configuration frame rate set contains one or more initial configuration frame rates.

The frame rate set determination subunit is also specifically used to obtain an initial configuration frame rate that is less than or equal to the maximum input frame rate from one or more initial configuration frame rates;

The frame rate set determination subunit is also specifically used to determine the set of initial configuration frame rates that are less than or equal to the maximum input frame rate as the device input frame rate set corresponding to the device input frame rate feature.

In one embodiment, the test stream consists of M media data frames; the M media data frames include media data frame _Qj ; M and j are both positive integers; the rendering effect reference value corresponding to the rendering parameter combination _Si includes the average decoding rendering latency corresponding to the rendering parameter combination _Si .

The reference value determination module may include:

The input time acquisition unit is used to acquire the frame input time of media data frame _Qj input to the test device;

The rendering frame acquisition unit is used to acquire the rendering frame corresponding to media data frame _Qj from the rendering data of the test bitstream by the test device.

The output timing acquisition unit is used to acquire the rendering frame corresponding to media data frame _Qj and output the frame output timing to the device display interface; the device display interface refers to the display interface of the test device.

The rendering delay determination unit is used to determine the time interval between the frame input time and the frame output time as the decoding rendering delay corresponding to the media data frame _Qj .

The average delay determination unit is used to determine the average decoding and rendering delay corresponding to the rendering parameter combination _Si based on the decoding and rendering delay corresponding to each of the M media data frames.

In one embodiment, the average delay determination unit may include:

The summation processing subunit is used to sum the M decoding and rendering delays to obtain the total value of the decoding and rendering delay;

The quantity statistics subunit is used to count the total number of media data frames contained in M media data frames;

The mean determination subunit is used to determine the mean between the total value of decoding and rendering latency and the total number of times, so as to obtain the average decoding and rendering latency corresponding to the rendering parameter combination _Si .

In one embodiment, the rendering effect reference value corresponding to the rendering parameter combination _Si refers to the average decoding rendering latency corresponding to the rendering parameter combination _Si ; the K rendering effect reference values refer to the K average decoding rendering latencies.

The optimal combination determination module may include:

The minimum latency determination unit is used to obtain the minimum average decoding and rendering latency among K average decoding and rendering latencies;

The optimal combination determination unit is used to determine the rendering parameter combination with the minimum average decoding and rendering latency among K rendering parameter combinations as the optimal rendering parameter combination for the terminal device.

One embodiment of this application provides a computer device, including: a processor and a memory;

The memory stores a computer program, which, when executed by a processor, causes the processor to perform the methods described in the embodiments of this application.

One aspect of this application provides a computer-readable storage medium storing a computer program, which includes program instructions. When executed by a processor, the program instructions perform the methods described in this application.

One aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in one aspect of the embodiments of this application.

In this embodiment, different rendering parameters affecting the rendering performance of a terminal device can be combined to obtain K rendering parameter combinations. Each rendering parameter combination can serve as a test group, and the rendering effect of each test group can be tested using a test stream. This allows us to determine which rendering parameter combination is the optimal rendering parameter for the terminal device. For example, for the rendering parameter combination _Si , a terminal device with rendering parameters _Si can be designated as a test device. A test stream can then be input into the test device, which can then render the test stream. Based on the rendering data of the test stream, a rendering effect reference value ₍ a value representing the rendering effect corresponding to the rendering parameter combination _Si ) can be determined. When the rendering effect reference value corresponding to each rendering parameter combination is determined, an optimal rendering effect reference value can be determined based on the K rendering effect reference values, thereby identifying an optimal rendering parameter combination. It should be understood that the method of dynamically probing the rendering performance of different combinations of rendering parameters on a terminal device based on test bitstreams in this application can determine the optimal rendering parameters based on actual rendering effect reference data (rendering effect reference values). Compared with randomly configuring rendering parameters for a terminal device, probing different combinations of rendering parameters through test bitstreams and determining the optimal combination of rendering parameters based on actual rendering effect reference data can more accurately and specifically determine the optimal combination of rendering parameters suitable for the terminal device, thereby improving the rendering performance of the terminal device. In summary, this application can optimize the rendering parameters of a terminal device and improve its rendering performance.

Attached Figure Description

To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

Figure 1 is a network architecture diagram provided in an embodiment of this application;

Figure 2 is a schematic diagram of a scene for determining optimal rendering parameters according to an embodiment of this application;

Figure 3 is a flowchart illustrating a data processing method provided in an embodiment of this application;

Figure 4 is a schematic diagram of a process for determining a combination of rendering parameters according to an embodiment of this application;

Figure 5 is a system flowchart provided in an embodiment of this application;

Figure 6 is a schematic diagram of a process for pruning a combination of rendering parameters according to an embodiment of this application.

Figure 7 is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

Figure 8 is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

Detailed Implementation

The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

This application also relates to artificial intelligence and related technologies. For ease of understanding, the concepts of artificial intelligence and related technologies will be briefly described below:

Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating/interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning/deep learning, autonomous driving, and intelligent transportation.

With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

The embodiments of this application mainly relate to computer vision technology in artificial intelligence software technology.

Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

Please refer to Figure 1, which is a schematic diagram of a network architecture provided in an embodiment of this application. As shown in Figure 1, the network architecture may include a service server 1000 and a cluster of terminal devices. The cluster of terminal devices may include one or more terminal devices, and the number of terminal devices is not limited here. As shown in Figure 1, the multiple terminal devices may specifically include terminal device 100a, terminal device 100b, terminal device 100c, ..., terminal device 100n. As shown in Figure 1, terminal device 100a, terminal device 100b, terminal device 100c, ..., terminal device 100n can respectively connect to the aforementioned service server 1000 via the network, so that each terminal device can interact with the service server 1000 through the network connection. The network connection method is not limited here; it can be directly or indirectly connected via wired communication, directly or indirectly connected via wireless communication, or other methods, which are not limited here.

Each terminal device can integrate and install a target application. When the target application runs on each terminal device, it can interact with the business server 1000 shown in Figure 1. The target application can include applications with functions to display text, images, audio, and video data. These applications can include social applications, multimedia applications (e.g., video applications), entertainment applications (e.g., cloud gaming applications), educational applications, live streaming applications, and other applications with media data encoding functions (such as video encoding). Of course, the application can also be other applications with data display and video encoding functions, which will not be listed here. The application can be a standalone application or an embedded sub-application integrated into an application (e.g., social applications, educational applications, and multimedia applications), without limitation.

For ease of understanding, this application embodiment can select one terminal device from the multiple terminal devices shown in FIG1 as the target terminal device. For example, this application embodiment can use terminal device 100a shown in FIG1 as the target terminal device, which can integrate a target application with video encoding function. In this case, the target terminal device can realize data interaction with the business server 1000 through the business data platform corresponding to the application client.

It should be understood that the computer devices (e.g., terminal device 100a, business server 1000) with media data encoding functions (e.g., video encoding functions) in the embodiments of this application can use cloud technology to achieve data encoding and data transmission of multimedia data (e.g., video data). Cloud technology refers to a hosting technology that unifies hardware, software, network, and other resources within a wide area network or local area network to achieve data computation, storage, processing, and sharing.

Cloud technology can be a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

For example, the data processing method provided in this application can be applied to scenarios requiring high resolution, high frame rate, and low latency, such as video viewing, video calling, video transmission, cloud conferencing, cloud gaming, and live streaming. Cloud conferencing, based on cloud computing technology, is an efficient, convenient, and low-cost form of meeting. Users only need to perform simple and easy-to-use operations through an internet interface to quickly and efficiently share voice, data files, and video with teams and clients worldwide. The complex technologies involved in data transmission and processing during the meeting are handled by the cloud conferencing service provider. Currently, domestic cloud conferencing mainly focuses on services based on the SaaS (Software as a Service) model, including telephone, internet, and video services. Video conferencing based on cloud computing is called cloud conferencing. In the era of cloud conferencing, data transmission, processing, and storage are all handled by the computer resources of the video conferencing provider. Users no longer need to purchase expensive hardware or install cumbersome software; they only need to open a browser and log in to the corresponding interface to conduct efficient remote meetings. Cloud conferencing systems support dynamic multi-server cluster deployment and provide multiple high-performance servers, greatly improving meeting stability, security, and availability. In recent years, video conferencing has gained popularity among many users due to its ability to significantly improve communication efficiency, continuously reduce communication costs, and upgrade internal management levels. It has been widely applied in various fields such as transportation, logistics, finance, telecommunications, education, and enterprises. Undoubtedly, with the application of cloud computing, video conferencing will be even more attractive in terms of convenience, speed, and ease of use, and will surely usher in a new wave of video conferencing applications.

It should be understood that computer equipment with media data encoding capabilities (e.g., terminal equipment 100a with video encoding capabilities) can encode media data using a media data encoder (e.g., a video encoder) to obtain the corresponding data bitstream (e.g., the video bitstream corresponding to the video data), thereby improving the transmission efficiency of media data. When the media data encoder is a video encoder, it can be an AV1 video encoder, an H.266 video encoder, an AVS3 video encoder, etc., which will not be listed here. The AV1 video encoder uses a video compression standard developed by the Alliance for Open Media (AOM), which is the first-generation video coding standard.

It should be understood that for encoded data streams (such as video streams), terminal devices can obtain the images (such as video images, game images, etc.) for output display by decoding and rendering them. It is also understood that, to optimize the rendering parameters of the terminal device and improve its rendering performance, thereby obtaining higher-quality rendered images, this application can configure different combinations of rendering parameters for the terminal device, and then test these different combinations based on test streams to determine the optimal rendering parameter combination. This allows the terminal device's rendering parameters to be configured with the most adaptable and highest-performing combination, thereby improving the decoding and rendering performance of the terminal device.

Specifically, for terminal devices, there are various characteristics that affect their rendering performance (each of which can be referred to as a rendering feature). For example, features such as rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate all affect the rendering performance of the terminal device. Therefore, rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate can all be referred to as device rendering features. Among them, a device rendering feature can contain different possible values, and each possible value can be understood as a rendering parameter (that is, a parameter that affects the rendering performance of the terminal device, which can also be referred to as a device rendering parameter in this application). For example, taking the rendering window type as an example, it can take the values SurfaceView and TextureView. Each of the possible window types can be used as a device rendering parameter under the device rendering feature of rendering window type (that is, SurfaceView can be used as a device rendering parameter and TextureView can be used as a device rendering parameter). As another example, taking the rendering frame dropping mode as an example, it can take the values of frame dropping and no frame dropping. Each of the possible values can be used as a rendering parameter under the device rendering feature of rendering frame dropping mode (that is, frame dropping can be used as a device rendering parameter and no frame dropping can be used as a device rendering parameter).

In this application, different rendering parameters affecting the rendering characteristics of various devices can be combined to obtain K rendering parameter combinations. This means that a rendering parameter combination contains one or more different rendering parameters affecting the rendering performance. Based on this, this application can dynamically test the rendering effect of each rendering parameter combination on a test bitstream (a bitstream obtained after encoding processing, such as a media data bitstream like a video bitstream). Specifically, for the rendering effect of a certain rendering parameter combination on the test bitstream, this application can determine the effect based on a reference value of that rendering parameter combination on the test bitstream (i.e., using a numerical value to reflect the quality of the rendering effect). Taking the rendering parameter combination _Si as an example, the rendering parameters of the terminal device can be configured to this _combination (the terminal device configured with this rendering parameter combination _Si can be called a test device). Then, the service server 1000 can encode certain media data. After obtaining the test stream, the test stream can be input into the test device. Thus, the test device can decode and render the test stream based on the rendering parameter combination _Si . Furthermore, the service server 1000 can obtain the rendering data of the test stream from the test device. Based on this rendering data, a rendering effect reference value for the rendering parameter combination _Si can be determined. Similarly, for each rendering parameter combination, the service server 1000 can obtain its corresponding rendering effect reference value. Based on these rendering effect reference values, the service server 1000 can determine which rendering parameter combination has the optimal rendering effect, thereby determining the optimal rendering parameter combination. Finally, the device rendering parameters of the terminal device can be configured to this optimal rendering parameter combination.

For details on how to combine different rendering parameters to obtain K rendering parameter combinations and determine the rendering effect reference value for each rendering parameter combination, please refer to the description in the embodiment corresponding to Figure 3 below.

It is understood that the methods provided in this application embodiment can be executed by computer devices, including but not limited to terminal devices or business servers. The business server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

Optionally, and understandably, the aforementioned computer devices (such as the aforementioned business server 1000, terminal device 100a, terminal device 100b, etc.) can be nodes in a distributed system. This distributed system can be a blockchain system, formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer (P2P) network, where the P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In this distributed system, any type of computer device, such as a business server or terminal device, can become a node in the blockchain system by joining this peer-to-peer network. For ease of understanding, the concept of blockchain is explained below: Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. It is mainly used to organize data in chronological order and encrypt it into a ledger, making it tamper-proof and forgery-proof, while also enabling data verification, storage, and updating. When a computer device is a blockchain node, the immutability and anti-counterfeiting characteristics of the blockchain can ensure the authenticity and security of the data in this application (such as the encoded test stream, the rendering effect reference value corresponding to each rendering parameter combination, etc.), thereby making the results obtained after relevant data processing based on these data more reliable.

It should be noted that, in the specific embodiments of this application, data involving user information and user data (such as relevant application data uploaded in the target application) requires user authorization before it can be obtained. In other words, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

This application embodiment can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. For ease of understanding, please refer to Figure 2, which is a schematic diagram of a scenario for determining optimal rendering parameters provided by this application embodiment. The scenario shown in Figure 2 is an example of video transmission between different terminal devices. In this video transmission scenario, terminal device _S1 can be a sending terminal for sending video data (e.g., video data 1), and the user corresponding to terminal device _S1 can be user a. Terminal device _S2 can be a receiving terminal for receiving video data (e.g., video data 1), and the user corresponding to terminal device _S2 can be user b. The service server 200 in the embodiment of Figure 2 can be a server with a network connection to terminal devices _S1 and _S2 , and this service server 200 can be the service server 1000 shown in Figure 1 above.

It should be understood that in a video transmission scenario, terminal device _S1 can acquire video data 1 associated with user a, captured by an image acquisition device (e.g., a camera). Further, terminal device _S1 can encode the video data 1 using a video encoder (e.g., an AV1 video encoder) to generate a video stream 1 associated with the video data 1. At this time, terminal device _S1 can send the video stream 1 to service server 200. Upon receiving the video stream 1, service server 200 can decode it to obtain video data (also called YUV video data, decoded video data, or video data to be encoded) in pixel image format (also known as YUV format). Then, service server 200 can encode the video data to be encoded (e.g., by encoding it using a video encoder), thereby obtaining a new video stream (e.g., video stream 2) encoded by service server 200.

In this context, it can be understood that the business server 200 encodes the video data to be encoded, which can be understood as encoding each video frame (referred to as the video frame to be encoded) of the video data to be encoded. Therefore, the video stream 2 can contain the encoded frame (referred to as the encoded frame) corresponding to each video frame. Furthermore, the business server 200 can send the video stream 2 to the terminal device _S2 , and the terminal device _S2 can decode and render the video stream 2 for output, so that user b can view the video data 1 through the terminal device _S2 .

It should be understood that, in order to improve the user's experience of watching video data, this application can optimize the rendering parameters of the terminal device to improve the rendering performance of the terminal device, thereby improving the quality of the rendered image (such as the rendered image of video stream 2) obtained by the terminal device. Specifically, the embodiments of this application can obtain different rendering parameters (which can be called influencing rendering parameters) that affect the rendering performance of the terminal device _S2 , and can combine different influencing rendering parameters to obtain multiple rendering parameter combinations. Here, assuming that the rendering parameter combinations include rendering parameter combination 1, rendering parameter combination 2, rendering parameter combination 3, and rendering parameter combination 4, the device rendering parameters of the terminal device _S2 can be switched to rendering parameter combination 1. For ease of distinction, the terminal device _S2 whose device rendering parameters are switched to rendering parameter combination 1 is called terminal device 200a. This terminal device 200a can be used as a test device (hereinafter referred to as test device 200a).

Furthermore, the video stream 2 encoded by the service server 200 can be used as a test stream. This test stream can be input into the test device 200a. In the test device 200a, each encoded frame in the test stream (encoded frame 20a as shown in Figure 2) can be decoded and rendered, thereby obtaining the rendering data for each encoded frame. Subsequently, based on this rendering data, a rendering effect reference value for the test stream can be determined by the test device 200a (rendering effect reference value a as shown in Figure 2). This rendering effect reference value a can be used to characterize the rendering effect of rendering parameter combination 1. Similarly, the device rendering parameters of terminal device _S2 can be switched to rendering parameter combination 2, rendering parameter combination 3, or rendering parameter combination 4. For ease of distinction, terminal device _S2 with its rendering parameters switched to rendering parameter combination 2 is referred to as terminal device 200b, which can serve as a test device (hereinafter referred to as test device 200b). Terminal device _S2 with its rendering parameters switched to rendering parameter combination 3 is referred to as terminal device 200c, which can serve as a test device (hereinafter referred to as test device 200c). Terminal device _S2 with its rendering parameters switched to rendering parameter combination 4 is referred to as terminal device 200d, which can serve as a test device (hereinafter referred to as test device 200d). For each test device, a test bitstream can be input into the test device. By decoding and rendering the test bitstream data through each test device, reference values for the rendering effects corresponding to different rendering parameter combinations can be obtained.

For example, as shown in Figure 2, based on the decoding and rendering data of the test bitstream from test device 200a, the rendering effect reference value a corresponding to rendering parameter combination 1 can be obtained; based on the decoding and rendering data of the test bitstream from test device 200b, the rendering effect reference value b corresponding to rendering parameter combination 2 can be obtained; based on the decoding and rendering data of the test bitstream from test device 200c, the rendering effect reference value c corresponding to rendering parameter combination 3 can be obtained; and based on the decoding and rendering data of the test bitstream from test device 200d, the rendering effect reference value d corresponding to rendering parameter combination 4 can be obtained.

Furthermore, an optimal rendering effect reference value can be determined from rendering effect reference value a, rendering effect reference value b, rendering effect reference value c, and rendering effect reference value d. The combination of rendering parameters corresponding to this optimal rendering effect reference value can be used as an optimal rendering parameter combination, and the device rendering parameters of the terminal device _S2 can be configured as this optimal rendering parameter combination.

It should be understood that the method of dynamically probing the rendering performance of different combinations of rendering parameters on the terminal device based on the test bitstream of this application can determine the optimal rendering parameters based on the actual rendering effect reference data (rendering effect reference value). Compared with randomly configuring rendering parameters for the terminal device, the method of probing different combinations of rendering parameters through the test bitstream and determining the optimal combination of rendering parameters based on the actual rendering effect reference data can more accurately and specifically determine the optimal combination of rendering parameters suitable for the terminal device, thereby improving the rendering performance of the terminal device.

Further, please refer to Figure 3, which is a flowchart illustrating a data processing method provided in an embodiment of this application. The method provided in this embodiment can be applied to various scenarios, including but not limited to audio/video, cloud technology, artificial intelligence, smart transportation, and assisted driving scenarios. This method can be executed by a terminal device (e.g., any terminal device in the terminal device cluster of Figure 1 above), by a server (e.g., the business server 1000 in Figure 1 above), or jointly by a terminal device and a server. For ease of understanding, this embodiment uses the execution of the method by the aforementioned server as an example to illustrate the specific process of data processing on the server. The data processing method may include at least the following steps S101-S103:

Step S101: Obtain K rendering parameter combinations for the terminal device; K is a positive integer; each of the K rendering parameter combinations contains one or more rendering parameters that affect the rendering performance of the terminal device; each rendering parameter that affects the rendering performance of the terminal device; the K rendering parameter combinations contain rendering parameter combination S _i, where i is a positive integer.

In this application, when a target application is deployed on a terminal device, the terminal device needs to render the application data of the target application to output the application screen of the target application. Here, the target application can refer to any application with data processing capabilities. For example, the target application can be a social application (such as an instant messaging application), an entertainment application (such as a cloud gaming application or a short video application), an educational application, a multimedia application, etc., and will not be listed here.

It should be understood that for terminal devices, there are various characteristics that affect their rendering performance (each of which can be referred to as a rendering feature). For example, features such as rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate all affect the rendering performance of the terminal device. Therefore, rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate can all be referred to as device rendering features. Among them, a device rendering feature can contain different possible values, and each possible value can be understood as a rendering parameter (that is, a parameter that affects the rendering performance of the terminal device, which can also be referred to as a device rendering parameter in this application). For example, taking the rendering window type as an example, it can take the values SurfaceView and TextureView. Each of the possible window types can be used as a device rendering parameter under the device rendering feature of rendering window type (that is, SurfaceView can be used as a device rendering parameter and TextureView can be used as a device rendering parameter). As another example, taking the rendering frame dropping mode as an example, it can take the values of frame dropping and no frame dropping. Each of the possible values can be used as a rendering parameter under the device rendering feature of rendering frame dropping mode (that is, frame dropping can be used as a device rendering parameter and no frame dropping can be used as a device rendering parameter).

The rendering parameter combination in this application can refer to a combination of rendering parameters under different device rendering characteristics of the terminal device. For example, taking device rendering characteristics including rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate as an example, the possible values under rendering window type include SurfaceView and TextureView, the possible values under rendering frame dropping mode include frame dropping mode and no frame dropping mode, the possible values under rendering timestamp type include startup elapsed time (Monotonic timestamp, which refers to the length of time elapsed since the terminal device system started), real time (Real timestamp, which refers to the time difference between the current time and 00:00 on January 1, 1970), and custom time (Artificial timestamp, which refers to the timestamp made by the sequence number of the media data frame, for example, the timestamp of the first frame is 1, the timestamp of the second frame is 2, ..., the timestamp of the nth frame is n), and the possible values of rendering input frame rate include 30fps, 60fps, 90fps, 120fps, and 150fps. You can combine any rendering parameter (with one possible value) under the rendering window type, a rendering parameter (with one possible value) under the rendering frame dropping mode, a rendering parameter (with one possible value) under the rendering timestamp type, and a rendering parameter (with one possible value) under the rendering input frame rate into a combination, and this combination can be used as a rendering parameter combination.

It should be noted that the various rendering parameter combinations are different, meaning that the rendering parameters included in each combination are different. For example, in the example above, since the number of possible values under the rendering window type is 2, the number of possible values under the rendering frame dropping mode is 2, the number of possible values under the rendering timestamp type is 3, and the number of possible values under the rendering input frame rate is 5, the total number of functional parameter combinations obtained can be 2×2×3×5＝60.

It is understandable that the different rendering parameters mentioned above determine the rendering effect of the terminal device on the media data. Therefore, the rendering parameters are also the key to determining the rendering performance of the terminal device. In order to optimize the rendering performance of the terminal device and make the rendered image of the terminal device of higher quality, this application can conduct test experiments on different combinations of rendering parameters to find the most beneficial combination of rendering parameters to improve the rendering performance of the terminal device (which can be called the optimal rendering parameter combination). Then, the various device rendering parameters of the terminal device are set to the optimal rendering parameter combination. The optimal rendering parameter combination is determined through testing and is most suitable for the terminal device. Therefore, the rendering performance of the terminal device can be improved based on the optimal rendering parameter combination.

In other words, the K rendering parameter combinations in this application can be determined based on the number of influencing rendering parameters included in each of the aforementioned device rendering features. Typically, K can be the product of the number of influencing rendering parameters under each device rendering feature (e.g., K could be 60 in the example above). Each rendering parameter combination is composed of different influencing rendering parameters, and the influencing rendering parameters included in each combination are not entirely the same (at least one influencing rendering parameter is different). Here, the rendering parameter combination _Si can refer to any one of the K rendering parameter combinations. This is a brief introduction to the principle of determining the K rendering parameter combinations. For a detailed description of the specific implementation of determining the K rendering parameter combinations, please refer to the embodiment corresponding to Figure 4 below.

Step S102: Input the test bitstream into the test device, and determine the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test bitstream by the test device; the test device refers to the device obtained by switching the device rendering parameters of the terminal device to the rendering parameter combination _Si ; the rendering effect reference value corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si .

In this application, to test which of K rendering parameter combinations best improves the rendering performance of the terminal device, this application can test each rendering parameter combination based on a test bitstream to obtain the rendering effect of each rendering parameter combination on the test bitstream. Here, the test bitstream can refer to the media bitstream obtained after encoding certain media data (such as video data). The terminal device can decode and render this test bitstream to output the media data. It should be noted that the media data here can be data composed of different media data frames (such as video frames). Encoding the media data can refer to encoding each media data frame (after encoding, each media data frame corresponds to a separate encoded frame). Therefore, the test bitstream can refer to a bitstream composed of different encoded frames, i.e., the test bitstream contains different encoded frames. For the terminal device, decoding and rendering the test bitstream can actually refer to decoding and rendering each encoded frame one by one.

Taking the rendering parameter combination _Si as an example, this embodiment of the application can set the device rendering parameters of the terminal device to the rendering parameter combination _Si , and define the terminal device with the device rendering parameters set to the rendering parameter combination _Si as a test device for testing. In this application, the principle of A/B testing can be adopted, treating each rendering parameter combination as an experimental group, and assigning identical test data (such as a test bitstream) to each experimental group. That is, the test data input to each experimental group is the same. With identical test data, the rendering effect of each experimental group (or test group) on the test bitstream can be observed. For example, for the rendering parameter combination _Si , after inputting the test bitstream into the test device, the test device can perform decoding and rendering processing on the test bitstream based on the rendering parameter combination _Si , thereby obtaining rendering data. Based on this rendering data, the rendering effect corresponding to the rendering parameter combination _Si can be determined. To more intuitively and accurately reflect the rendering effect corresponding to each rendering parameter combination, this application can pre-configure rendering effect reference values to characterize the rendering effect. Based on these rendering effect reference values, it can be determined which rendering parameter combination has a better rendering effect. Here, the rendering effect reference value can refer to any feature value that can characterize the decoding and rendering effect of the terminal device. For example, for the terminal device, the rendering effect can usually be determined based on the feature values such as the decoding output frame rate, decoding rendering latency, rendering output frame rate, and rendering time. By comparing these feature values of each rendering parameter combination, the optimal rendering parameter combination can be determined.

Specifically, based on the above, the test stream can consist of M media data frames. Taking the M media data frames including media data frame _Qj (M and j are both positive integers) as an example, assuming the rendering effect reference value corresponding to the rendering parameter combination _Si includes the average decoding rendering latency corresponding to the rendering parameter combination _Si , the specific implementation of determining the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test stream by the test device can be as follows: the frame input time of the media data frame _Qj input to the test device can be obtained; subsequently, the rendering frame corresponding to the media data frame _Qj can be obtained from the rendering data of the test stream by the test device, and the frame output time of the rendering frame corresponding to the media data frame _Qj output to the device display interface can be obtained; where the device display interface refers to the display interface of the test device; furthermore, the time period between the frame input time and the frame output time can be determined as the decoding rendering latency corresponding to the media data frame _Qj ; when the decoding rendering latency corresponding to each of the M media data frames is determined, the average decoding rendering latency corresponding to the rendering parameter combination _Si can be determined based on the decoding rendering latency corresponding to each media data frame.

The specific implementation method for determining the average decoding and rendering latency corresponding to the rendering parameter combination _Si based on the decoding and rendering latency corresponding to each media data frame can be as follows: the M decoding and rendering latencies can be summed to obtain the total value of the decoding and rendering latency; then, the total number of media data frames contained in the M media data frames can be counted; the mean between the total value of the decoding and rendering latency and the total number can be determined, thereby obtaining the average decoding and rendering latency corresponding to the rendering parameter combination _Si .

It should be understood that since the test stream consists of M media data frames, during the decoding and rendering process of the test stream by the test device, this application can statistically determine the time from the input of each media data frame to the rendering output to the device display interface (i.e., the screen of the terminal device) (i.e., the time interval between the frame input time to the test device and the frame output time to the device display interface). This time interval can be used as the decoding and rendering delay of a certain media data frame. When the decoding and rendering delay corresponding to each of the M media data frames is determined, an average decoding and rendering delay (e.g., total decoding and rendering delay / total number) can be obtained based on the total number of M media data frames and the sum of the decoding and rendering delays of all media data frames (i.e., the result value obtained after adding the decoding and rendering delays of all media data frames). This average decoding and rendering delay can be used as a reference value for the rendering effect corresponding to the rendering parameter combination _Si .

It should be noted that the above method for determining the rendering effect reference value corresponding to the rendering parameter combination _Si is illustrated using the average decoding and rendering latency as an example. Determining the rendering effect reference value corresponding to the rendering parameter combination _Si is not limited to determining the average decoding and rendering latency. The rendering effect reference value for a given rendering parameter combination can also be any other feature value that can characterize the rendering effect, such as the decoding and rendering time. When using the decoding and rendering time as the rendering effect reference value, the specific implementation for determining the rendering effect reference value corresponding to the rendering parameter combination _Si can also be as follows: the decoding and rendering time corresponding to each media data frame can be counted, and then the total decoding and rendering time corresponding to all media data frames can be counted (the decoding and rendering times corresponding to all media data frames are added together, and the sum is taken as the total decoding and rendering time). This total decoding and rendering time can then be used as a rendering effect reference value. In other words, the reference value for the rendering effect of a combination of rendering parameters is not limited to the average decoding and rendering latency. It can be determined based on the actual scene. However, once the reference value for the rendering effect is determined, the reference value for the rendering effect of each combination of rendering parameters should be the same feature value. This makes it easier to compare and determine the optimal combination of rendering parameters.

Step S103: When the rendering effect reference value corresponding to each rendering parameter combination is determined, the optimal rendering parameter combination for the terminal device is determined from the K rendering effect reference values.

In this application, once the rendering effect reference value corresponding to each rendering parameter combination is determined, K rendering effect reference values can be obtained. Based on the K rendering effect reference values, the optimal rendering parameter combination for the terminal device can be determined from the K rendering parameter combinations. Taking the average decoding rendering latency as an example, the rendering effect reference value corresponding to the rendering parameter combination _Si can refer to the average decoding rendering latency corresponding to the rendering parameter combination _Si , while the K rendering effect reference values refer to the K average decoding rendering latencies. In this case, the specific implementation method for determining the optimal rendering parameter combination for the terminal device from the K rendering parameter combinations based on the K rendering effect reference values can be as follows: the minimum average decoding rendering latency can be obtained from the K average decoding rendering latencies; subsequently, the rendering parameter combination corresponding to the minimum average decoding rendering latency from the K rendering parameter combinations can be determined as the optimal rendering parameter combination for the terminal device.

In other words, among K combinations of rendering parameters, the lower the average decoding and rendering latency, the higher the decoding and rendering efficiency. Therefore, the combination of rendering parameters with the lowest average decoding and rendering latency can be determined as the optimal combination of rendering parameters.

In this embodiment, different rendering parameters affecting the rendering performance of a terminal device can be combined to obtain K rendering parameter combinations. Each rendering parameter combination can serve as a test group, and the rendering effect of each test group can be tested using a test stream. This allows us to determine which rendering parameter combination is the optimal rendering parameter for the terminal device. For example, for the rendering parameter combination _Si , a terminal device with rendering parameters _Si can be designated as a test device. A test stream can then be input into the test device, which can then render the test stream. Based on the rendering data of the test stream, a rendering effect reference value ₍ a value representing the rendering effect corresponding to the rendering parameter combination _Si ) can be determined. When the rendering effect reference value corresponding to each rendering parameter combination is determined, an optimal rendering effect reference value can be determined based on the K rendering effect reference values, thereby identifying an optimal rendering parameter combination. It should be understood that the method of dynamically probing the rendering performance of different combinations of rendering parameters on a terminal device based on test bitstreams in this application can determine the optimal rendering parameters based on actual rendering effect reference data (rendering effect reference values). Compared with randomly configuring rendering parameters for a terminal device, probing different combinations of rendering parameters through test bitstreams and determining the optimal combination of rendering parameters based on actual rendering effect reference data can more accurately and specifically determine the optimal combination of rendering parameters suitable for the terminal device, thereby improving the rendering performance of the terminal device. In summary, this application can optimize the rendering parameters of a terminal device and improve its rendering performance.

Further, please refer to Figure 4, which is a schematic flowchart of determining a combination of rendering parameters according to an embodiment of this application. This process can correspond to the process for obtaining K combinations of rendering parameters for a terminal device in the embodiment corresponding to Figure 3 above. As shown in Figure 4, this process can include at least the following steps S401-S403:

Step S401: Obtain R device rendering features configured for the terminal device; R is a positive integer.

Specifically, for terminal devices, there are different features that affect their rendering performance (each of which can be called a rendering feature). For example, features such as rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate will all affect the rendering performance of the terminal device. Therefore, rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate can all be called device rendering features.

In other words, device rendering features refer to the features that affect the rendering performance of a terminal device. These features can be configured based on the actual scenario. Typically, device rendering features may include, but are not limited to, rendering window type, rendering frame dropping mode, rendering timestamp type, rendering input frame rate, etc.

Step S402: Obtain the set of device rendering parameters corresponding to each of the R device rendering features to obtain the set of R device rendering parameters.

Specifically, a device rendering feature can contain different possible values, each of which can be understood as a rendering parameter (i.e., a parameter affecting the rendering performance of the terminal device, which may also be referred to as a device rendering parameter in this application). For example, taking the rendering window type as an example, it can take the values SurfaceView and TextureView. Each possible window type can then be used as a device rendering parameter under the device rendering feature of rendering window type (i.e., SurfaceView can be used as a device rendering parameter, and TextureView can be used as a device rendering parameter). Similarly, taking the rendering frame dropping mode as an example, it can take the values of frame dropping and no frame dropping. Each possible value can then be used as a rendering parameter under the device rendering feature of rendering frame dropping mode (i.e., frame dropping can be used as a device rendering parameter, and no frame dropping can be used as a device rendering parameter). The possible values contained in a certain device rendering feature in this application can form the set of device rendering parameters corresponding to that feature. For example, for the device rendering feature of rendering window type, its set of device rendering parameters can be {SurfaceView, TextureView}.

It should be noted that the hardware of a terminal device includes a chip, and the decoding speed of the chip will affect the rendering performance of the terminal device. Therefore, in this application, the decoding speed of the terminal device can be estimated in advance, and the rendering performance of the terminal device can be estimated based on the decoding speed, so as to determine the maximum input frame rate that the terminal device can support during the test. Based on the maximum input frame rate, the device input frame rates that the terminal device can support can be determined.

In other words, as can be seen from the above, the R device rendering features may include the rendering input frame rate (which may be referred to as the device input frame rate feature in this application). In this application, the maximum input frame rate that the terminal device can support can be determined based on some hardware parameters of the terminal device, so that the set of device rendering parameters corresponding to the device input frame rate feature can be determined based on the maximum input frame rate.

Specifically, based on the above, the R device rendering features include device input frame rate features, and the device rendering parameter set corresponding to each device rendering feature includes the device input frame rate set corresponding to the device input frame rate feature. Therefore, the specific implementation for obtaining the device rendering parameter set corresponding to each of the R device rendering features can be as follows: First, obtain the media encoding type for which the terminal device has encoding and decoding permissions. Then, obtain the maximum resolution corresponding to the terminal device. Based on the media encoding type and the maximum resolution, determine the predicted maximum decoding frame rate for the terminal device. Simultaneously, obtain the screen refresh rate corresponding to the terminal device. The minimum value between the predicted maximum decoding frame rate and the screen refresh rate can be determined as the maximum input frame rate of the terminal device. Based on the maximum input frame rate, the device input frame rate set corresponding to the device input frame rate feature can be determined.

Specifically, the implementation method for determining the set of device input frame rates corresponding to the device input frame rate feature based on the maximum input frame rate can be as follows: an initial configuration frame rate set can be obtained; wherein the initial configuration frame rate set contains one or more initial configuration frame rates; subsequently, an initial configuration frame rate less than or equal to the maximum input frame rate can be obtained from the one or more initial configuration frame rates; the set of initial configuration frame rates less than or equal to the maximum input frame rate can be determined as the set of device input frame rates corresponding to the device input frame rate feature.

It should be understood that this application can make a preliminary judgment on the rendering performance of the terminal device through different hardware parameters of the terminal device, so as to determine the device input frame rate of the terminal device during the test experiment. For example, since the decoding speed of the terminal device chip will have a certain impact on the terminal device, this application can determine the codec type supported by the chip (such as H264 type/H265 type/VP9 type/AVS type) by reading the profile and level parameters supported by the terminal device chip. The codec types supported by the chip can be identified as the media encoding types for which the terminal device has the authority to encode and decode. The maximum resolution supported by the chip can also be identified as the maximum resolution corresponding to the terminal device. Based on the media encoding types and maximum resolution for which the terminal device has the authority to encode and decode, the decoding capability of the terminal device can be preliminarily estimated. Based on the decoding capability of the terminal device, its rendering performance can be preliminarily categorized into a certain range, which can refer to a frame rate range (e.g., below 30fps, 60fps, 90fps, 120fps, etc.). This estimated frame rate range can also be understood as the predicted maximum decoding frame rate of the terminal device. For example, if the estimated frame rate range of the terminal device is below 60fps, then the predicted maximum decoding frame rate of the terminal device can be determined to be 60fps.

Furthermore, the input frame rate of the terminal device can be limited by combining the screen refresh rate and the maximum predicted decoding frame rate. For example, if the screen refresh rate only supports 60Hz, then even if the maximum predicted decoding frame rate is 90fps, it is not necessary to detect input frame rates above 60fps; only input frame rates of 60fps and below need to be detected. In other words, the maximum input frame rate of the terminal device can refer to the minimum value between the maximum predicted decoding frame rate and the screen refresh rate. Once the maximum predicted decoding frame rate is determined, the set of device input frame rates corresponding to the device's input frame rate characteristics can be determined.

For example, taking an initial configuration frame rate set of {30fps, 60fps, 90fps, 120fps, 144fps} as an example, assuming the maximum input frame rate of the terminal device is 120fps, since 30fps, 60fps, and 90fps are all less than 120fps, the terminal device can support all four input frame rates of 30fps, 60fps, 90fps, and 120fps. Based on this, the set of device rendering parameters corresponding to the device input frame rate characteristics can be {30fps, 60fps, 90fps, 120fps}.

In other words, for the device input frame rate of a terminal device, the device rendering parameters included in its corresponding device rendering parameter set can be a portion of the initial configuration frame rates in the initial configuration frame rate set, not all of them.

Step S403: Based on the parameter combination rules, combine the device rendering parameters contained in the R sets of device rendering parameters to obtain K rendering parameter combinations.

Specifically, once the set of rendering parameters for each device is determined, the device rendering parameters contained in the R sets of device rendering parameters can be combined based on the parameter combination rules, thereby obtaining K rendering parameter combinations.

Here, the parameter combination rule can refer to the rule for combining any one of the device rendering parameters in each device rendering feature. The specific implementation of combining device rendering parameters from R device rendering parameter sets to obtain K rendering parameter combinations based on parameter combination rules can be as follows: Based on parameter combination rules, the combination formed by the first target device rendering parameter in the target device rendering parameter set and the second target device rendering parameter in the remaining device rendering parameter sets is determined as rendering parameter combination S _{_i}; the target device rendering parameter set is any device rendering parameter set among the R device rendering parameter sets; the remaining device rendering parameter set is the set of device rendering parameters other than the target device rendering parameter set among the R device rendering parameter sets; the first target device rendering parameter refers to any device rendering parameter in the target device rendering parameter set; the second target device rendering parameter refers to any device rendering parameter in the remaining device rendering parameter set; the combination formed by the remaining device rendering parameters in the target device rendering parameter set and the second target device rendering parameter is determined as the remaining rendering parameter combination; the remaining device rendering parameter is any device rendering parameter other than the first target device rendering parameter in the target device rendering parameter set; K rendering parameter combinations are determined based on rendering parameter combination S _{_i} and the remaining rendering parameter combination.

Specifically, the target device rendering parameter set here can refer to any one of the R device rendering parameter sets. This application can combine any one device rendering parameter from the target device rendering parameter set with one device rendering parameter from each of the other device rendering parameter sets to obtain a rendering parameter combination. For ease of understanding, taking device rendering features including rendering window type, rendering frame dropping mode, and rendering input frame rate as an example, assuming the device rendering parameter set corresponding to the rendering window type is {SurfaceView, TextureView}, the device rendering parameter set corresponding to the rendering frame dropping mode is {frame dropping, no frame dropping}, and the device rendering parameter set corresponding to the rendering input frame rate is {90fps, 60fps, 30fps} (i.e., the maximum input frame rate is 90fps), then after parameter combination, the resulting rendering parameter combinations can include rendering parameter combination 1 to rendering parameter combination 12:

Rendering parameter combination 1: [90fps, SurfaceView, frame drops];

Rendering parameter combination 2: [60fps, SurfaceView, frame drops];

Rendering parameter combination 3: [30fps, SurfaceView, frame drops];

Rendering parameter combination 4: [90fps, SurfaceView, no frame drops];

Rendering parameter combination 5: [60fps, SurfaceView, no frame drops];

Rendering parameter combination 6: [30fps, SurfaceView, no frame drops];

Rendering parameter combination 7: [90fps, TextureView, frame dropping];

Rendering parameter combination 8: [60fps, TextureView, frame dropping];

Rendering parameter combination 9: [30fps, TextureView, frame dropping];

Rendering parameter combination 10: [90fps, TextureView, no frame drops];

Rendering parameter combination 11: [60fps, TextureView, no frame drops];

Rendering parameter combination 12: [30fps, TextureView, no frame dropping].

In other words, through the above combination process, K rendering parameter combinations can be obtained. Each rendering parameter combination will contain one device rendering parameter under all device rendering features. Between any two rendering parameter combinations, at least one device rendering parameter is different.

Optionally, and understandably, based on the above, a rendering parameter combination is composed of a single device rendering parameter under all device rendering features. Therefore, if the number of device rendering features is large, or the number of device rendering parameters under a single device rendering feature is large, the number of rendering parameter combinations will be significant. With a large number of rendering parameter combinations, it will undoubtedly take a considerable amount of time to test and determine the optimal rendering parameter combination. To improve testing efficiency, this application can prune the rendering parameter combinations based on the parameter priority among the device rendering parameters in the device rendering features. Pruning, in this application, involves deleting some rendering parameter combinations to reduce the total number of rendering parameter combinations. This application can first determine the initial rendering parameter combinations obtained after parameter combining as the initial rendering parameter combinations. After pruning the initial rendering parameter combinations, the remaining rendering parameter combinations can be determined as K rendering parameter combinations.

Specifically, taking R device rendering features including device rendering feature _Hj (j is a positive integer) as an example, where each device rendering feature corresponds to a set of device rendering parameters, including the set of device rendering parameters _{Pj corresponding to device rendering feature Hj ,} the specific implementation for determining K rendering parameter combinations based on rendering parameter combination _Si and the remaining rendering parameter combinations can be as follows: Both rendering parameter combination _Si and the remaining rendering parameter combinations can be determined as initial rendering parameter combinations. Subsequently, the set of initial rendering parameter combinations can be determined as the initial rendering parameter combination set. Further, the parameter priority of each device rendering parameter in the set of device rendering parameters _Pj corresponding to device rendering feature _Hj can be obtained. According to the order of the parameter priorities of each device rendering parameter in the set of device rendering parameters _Pj , the device rendering parameters in the set of device rendering parameters _Pj can be sorted, thus obtaining a parameter sequence. Based on the parameter sequence, the initial rendering parameter combination set can be pruned to obtain a pruned rendering parameter combination set. The K rendering parameter combinations can then be determined based on the pruned rendering parameter combination set.

Specifically, the implementation of pruning the initial rendering parameter combination set based on the parameter sequence to obtain the pruned rendering parameter combination set can be as follows: the device rendering parameters located at the end of the sequence can be identified as the rendering parameters to be pruned; subsequently, the initial rendering parameter combination that contains the rendering parameters to be pruned can be identified as the rendering parameter combination to be pruned; the rendering parameter combination to be pruned in the initial rendering parameter combination set can be deleted, thereby obtaining the pruned rendering parameter combination set.

For ease of understanding, let's take the device rendering feature as the rendering window type as an example. The set of device rendering parameters corresponding to the rendering window type is {SurfaceView, TextureView}. Since SurfaceView performs better than TextureView under the same configuration, the priority of SurfaceView can be 1, while the priority of TextureView can be 2 (the smaller the parameter priority value, the higher the parameter priority). After sorting the device rendering parameters corresponding to the rendering window type according to parameter priority, the parameter sequence we get is {SurfaceView, TextureView}. Furthermore, we can find that the device rendering parameter at the end of the sequence is TextureView. This TextureView can then be used as the rendering parameter to be pruned. We can delete the rendering parameter combinations that contain TextureView from the initial set of rendering parameter combinations. Taking the above 12 rendering parameter combinations as an example, we can see that rendering parameter combinations 7 to 12 all contain TextureView. Therefore, we can delete these rendering parameter combinations, and the resulting set of pruned rendering parameter combinations will contain rendering parameter combinations 1 to 6.

It should be understood that pruning can be performed on each device rendering feature. After pruning each device rendering feature, the remaining rendering parameter combinations can be determined as the final K rendering parameter combinations. In other words, the specific implementation of determining the K rendering parameter combinations based on the pruned rendering parameter combination set can be as follows: the pruning attributes of the device rendering feature _Hj can be updated from unpruned to pruned; then, R device rendering features can be traversed; if the pruning attributes of each of the R device rendering features are all pruned, the pruned rendering parameter combination set can be directly determined as the K rendering parameter combinations; if there are device rendering features among the R device rendering features whose pruning attributes are unpruned, these unpruned features can be determined as unpruned rendering features, and the pruned rendering parameter combination set can be pruned again based on the device rendering parameter set corresponding to the unpruned features. Finally, the K rendering parameter combinations can be determined based on the pruning result.

In other words, each time a device rendering feature is pruned, its pruning attribute is changed from unpruned to pruned. When all device rendering features are pruned, it can be determined that all device rendering features have been pruned. Then, the remaining combinations of rendering parameters can be identified as K combinations for testing. If a device rendering feature has an unpruned attribute, then the current set of rendering parameter combinations can be pruned based on the parameter priority of each device rendering parameter in that feature. This process is repeated for each device rendering feature until all device rendering features have a pruned attribute.

It should be noted that in practical applications, pruning is not required for all device rendering features. Pruning can be performed on only one or a portion of the device rendering features. The pruning process can be defined based on the actual scenario requirements, and this application does not impose any restrictions on it.

Typically, device rendering characteristics for a terminal device include rendering window type, rendering frame dropping mode, rendering timestamp type, and rendering input frame rate. For the rendering window type, possible values include SurfaceView and TextureView, with SurfaceView having higher priority than TextureView. For the rendering frame dropping mode, possible values include dropping frames and not dropping frames, with dropping frames having higher priority than not dropping frames. For the rendering timestamp type, possible values include Monotonic timestamp, Real timestamp, and Artificial timestamp, with Monotonic timestamp having higher priority than Real timestamp, and Real timestamp having higher priority than Artificial timestamp. For the rendering input frame rate, a higher frame rate is preferred.

By prioritizing the rendering features of different devices, the combination of rendering parameters can be pruned, thereby reducing the number of rendering parameter combinations used in the final test experiment (i.e., reducing the K value). This allows the optimal combination of rendering parameters for the terminal device to be determined in less time.

In this application embodiment, a method is provided to dynamically probe the rendering performance of different rendering parameter combinations on a terminal device based on a test bitstream. This method determines the optimal rendering parameters based on actual rendering effect reference data (rendering effect reference values). Compared to randomly configuring rendering parameters for the terminal device, probing different rendering parameter combinations through a test bitstream and determining the optimal combination based on actual rendering effect reference data can more accurately and specifically identify the optimal rendering parameter combination suitable for the terminal device, thereby improving its rendering performance. Furthermore, during the testing process, this application provides a probe pruning method that allows for a limited number of tests based on the default configuration combination, determining the optimal rendering parameter combination in less time and significantly improving testing efficiency.

Further, please refer to Figure 5, which is a system flowchart provided in an embodiment of this application. As shown in Figure 5, the process may include at least the following steps S51-S53:

Step S51: Read the parameter information of the terminal device's chip and determine the rendering parameter combination of the terminal device.

Specifically, for terminal devices, their hardware includes chips, and the decoding speed of the terminal device's chips will also affect the rendering performance of the terminal device. Therefore, in this application, the decoding speed of the terminal device can be estimated in advance, and then the rendering performance of the terminal device can be estimated based on the decoding speed of the terminal device, thereby determining the maximum input frame rate that the terminal device can support during the test. Based on the maximum input frame rate, it can be determined which device input frame rates the terminal device can support.

By reading the profile and level parameters supported by the terminal device chip, the supported codec types (e.g., H.264/H.265/VP9/AVS) can be determined. The supported codec types identify the media encoding types that the terminal device has the authority to encode and decode. The maximum resolution supported by the chip also determines the maximum resolution of the terminal device. Based on the media encoding types and maximum resolution that the terminal device has the authority to encode and decode, a preliminary estimate of the terminal device's decoding capability can be made. Based on this decoding capability, the terminal device's rendering performance can be preliminarily categorized into a certain range. Furthermore, the terminal device's screen refresh rate and the predicted maximum decoding frame rate can be used to limit the input frame rate, thus obtaining the terminal device's maximum input frame rate. Subsequently, based on the current terminal device's supported rendering window types, rendering frame dropping strategies, rendering timestamp types, and other device rendering characteristics, combined with the maximum input frame rate, the device's rendering configuration can be obtained—that is, the combination of various rendering parameters for the terminal device.

Step S52: By dynamically inputting the test bitstream, the combination of rendering parameters is tested to determine the decoding and rendering latency of the terminal device.

Specifically, in this embodiment, the rendering parameter combinations obtained in step S51 can be probed using a real-time test stream. Specifically, a test stream (such as a video stream) needs to be input in real-time to test the terminal device with the corresponding rendering parameter combinations, in order to determine which set of rendering parameter combinations has the best rendering performance. Rendering performance can be determined by decoding rendering latency. The time from input to rendering of each data frame (such as a video frame) to rendering on the terminal device's screen can be used as the decoding rendering latency of that frame. By calculating the average decoding rendering latency of all frames, the rendering parameter combination with the lowest average decoding rendering latency can be obtained from the above rendering parameter combinations. This rendering parameter combination with the lowest average decoding rendering latency is considered the optimal rendering parameter combination.

Step S53: The combination with the lowest rendering/decoding latency is selected as the optimal rendering parameter combination.

Specifically, by statistically analyzing the average decoding and rendering latency of all frames, the rendering parameter combination with the lowest average decoding and rendering latency can be obtained from the above rendering parameter combinations. This rendering parameter combination with the lowest average decoding and rendering latency is used as the optimal rendering parameter combination.

In this embodiment of the application, a method is provided to dynamically detect the rendering performance of different combinations of rendering parameters on a terminal device based on a test bitstream. The optimal rendering parameters can be determined based on actual rendering effect reference data (rendering effect reference value). Compared with randomly configuring rendering parameters for the terminal device, the method of detecting different combinations of rendering parameters through a test bitstream and determining the optimal combination of rendering parameters based on actual rendering effect reference data can more accurately and specifically determine the optimal combination of rendering parameters suitable for the terminal device, thereby improving the rendering performance of the terminal device.

Further, please refer to Figure 6, which is a schematic flowchart of a rendering parameter combination pruning process provided by an embodiment of this application. As shown in Figure 6, the process may include at least the following steps S61-S63:

Step S61: Obtain the set of rendering parameter combinations.

Specifically, this application can combine a device rendering parameter under each device rendering feature to obtain a rendering parameter combination. Different device rendering parameters can be combined to obtain different rendering parameter combinations. For the method of determining how to combine device rendering parameters, please refer to the relevant description in the embodiments corresponding to Figures 3-4 above, which will not be repeated here.

Step S62: Modify the rendering parameter combinations in sequence according to parameter priority.

Specifically, different device rendering parameters under a device rendering feature can have parameter priorities. This application can prune the combined rendering parameters based on the parameter priorities of different device rendering parameters under a device rendering feature, thereby reducing the total number of rendering parameter combinations and reducing the testing time.

It should be understood that the parameter priority of different device rendering parameters under a certain device rendering characteristic can be determined through prior detection. For example, for the device rendering characteristic of rendering window type, all hardware parameters of the terminal device except for the rendering window type can be set to the same parameters, and then the rendering performance of different rendering windows can be detected. The rendering window with the smallest decoding and rendering latency is selected as the rendering window with higher parameter priority. Generally, the parameter priority of SurfaceView is higher than that of TextureView.

For example, regarding the device rendering feature of frame dropping mode, all hardware parameters of the terminal device except for the frame dropping mode can be set to the same value. Then, the rendering performance of different frame dropping modes can be tested. The frame dropping mode with the lowest decoding and rendering latency can be selected as the frame dropping mode with higher parameter priority. Generally, the parameter priority of frame dropping is higher than that of no frame dropping.

For example, regarding the device rendering feature of rendering timestamp type, all hardware parameters of the terminal device except for the rendering timestamp type can be set to the same value. Then, the rendering performance of different rendering timestamp types can be tested. The rendering timestamp type with the lowest decoding and rendering latency can be selected as the rendering timestamp type with higher parameter priority. Generally, the parameter priority of Monotonic timestamp is higher than that of Real timestamp, and the parameter priority of Real timestamp is higher than that of Artificial timestamp.

For example, regarding the rendering input frame rate, a device rendering characteristic, all hardware parameters of the terminal device except for the rendering input frame rate can be set to the same value. Then, the rendering performance of different rendering input frame rates can be tested. The rendering input frame rate with the lowest decoding and rendering latency can be selected as the rendering input frame rate with higher parameter priority. Generally, input frame rates with higher values have higher priority.

Specifically, the process of pruning the combination of rendering parameters based on the parameter priority of different device rendering parameters under different device rendering characteristics can be referred to the relevant description in the embodiment corresponding to Figure 4 above, and will not be repeated here.

Step S63: Detect the rendering performance of different rendering parameters and obtain the optimal parameter configuration.

Specifically, after pruning, the rendering performance of different combinations of rendering parameters can be tested through real-time dynamic test streams, ultimately yielding the optimal parameter configuration for the terminal device.

In the embodiments of this application, during the testing of rendering performance of rendering parameter combinations, this application provides a probe pruning method, which can perform a limited number of tests based on the default configuration combination, spend less time to determine the optimal rendering parameter combination, and can greatly improve testing efficiency.

Further, please refer to Figure 7, which is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. The data processing device can be a computer program (including program code) running on a computer device, for example, the data processing device is an application software; the data processing device can be used to execute the method shown in Figure 3. As shown in Figure 7, the data processing device 1 may include: a combination acquisition module 11, a code stream input module 12, a reference value determination module 13, and an optimal combination determination module 14.

The combination acquisition module 11 is used to acquire K combinations of rendering parameters for the terminal device; K is a positive integer; each of the K combinations of rendering parameters contains one or more rendering parameters that affect the rendering performance of the terminal device; each rendering parameter that affects the rendering performance of the terminal device contains a combination of rendering parameters _Si , where i is a positive integer.

The bitstream input module 12 is used to input the test bitstream into the test device; the test device refers to the terminal device that switches the device rendering parameters of the terminal device to the rendering parameter combination _Si .

The reference value determination module 13 is used to determine the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test bitstream by the test equipment; the rendering effect reference value corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si .

The optimal combination determination module 14 is used to determine the optimal rendering parameter combination for the terminal device from the K rendering parameter combinations based on the K rendering effect reference values when the rendering effect reference values corresponding to each rendering parameter combination are determined.

The specific implementation methods of the combination acquisition module 11, the code stream input module 12, the reference value determination module 13 and the optimal combination determination module 14 can be found in the description of steps S101-S103 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

In one embodiment, the combination acquisition module 11 may include: a feature acquisition unit 111, a parameter set acquisition unit 112, and a parameter combination unit 113.

The feature acquisition unit 111 is used to acquire R device rendering features configured for the terminal device; R is a positive integer;

The parameter set acquisition unit 112 is used to acquire the set of device rendering parameters corresponding to each of the R device rendering features, and thus obtain the R device rendering parameter sets.

The parameter combination unit 113 is used to combine the device rendering parameters contained in the R sets of device rendering parameters based on the parameter combination rules to obtain K rendering parameter combinations.

The specific implementation methods of the feature acquisition unit 111, the parameter set acquisition unit 112, and the parameter combination unit 113 can be found in the description of steps S401-S403 in the embodiment corresponding to Figure 4 above, and will not be repeated here.

In one embodiment, the parameter combination unit 113 may include: a first combination determination subunit 1131, a second combination determination subunit 1132, and a parameter combination determination subunit 1133.

The first combination determination subunit 1131 is used to determine, based on parameter combination rules, the combination formed by the first target device rendering parameter in the target device rendering parameter set and the second target device rendering parameter in the remaining device rendering parameter set as the rendering parameter combination _Si ; the target device rendering parameter set is any one of the R device rendering parameter sets; the remaining device rendering parameter set is the set of device rendering parameters other than the target device rendering parameter set among the R device rendering parameter sets; the first target device rendering parameter refers to any one of the device rendering parameters in the target device rendering parameter set; the second target device rendering parameter refers to any one of the device rendering parameters in the remaining device rendering parameter set;

The second combination determining subunit 1132 is used to determine the combination of the remaining device rendering parameters in the target device rendering parameter set and the second target device rendering parameters as the remaining rendering parameter combination; the remaining device rendering parameters are any device rendering parameters in the target device rendering parameter set other than the first target device rendering parameters.

The parameter combination determination subunit 1133 is used to determine K rendering parameter combinations based on the rendering parameter combination _Si and the remaining rendering parameter combinations.

The specific implementation of the first combination determining subunit 1131, the second combination determining subunit 1132, and the parameter combination determining subunit 1133 can be found in the description in S403 of the embodiment corresponding to Figure 4 above, and will not be repeated here.

In one embodiment, the R device rendering features include device rendering feature _Hj ; j is a positive integer; the device rendering parameter set corresponding to each device rendering feature includes the device rendering parameter set _Pj corresponding to device rendering feature _Hj ;

The parameter combination determination subunit 1133 is also specifically used to determine the rendering parameter combination _Si and the remaining rendering parameter combinations as the initial rendering parameter combination, and to determine the set of the initial rendering parameter combinations as the initial rendering parameter combination set.

The parameter combination determination subunit 1133 is also specifically used to obtain the parameter priority of each device rendering parameter in the device rendering parameter set _Pj corresponding to the device rendering feature _Hj ;

The parameter combination determination subunit 1133 is also specifically used to sort the device rendering parameters in the device rendering parameter set _Pj according to the order of priority of each device rendering parameter in the device rendering parameter set _Pj , so as to obtain the parameter sequence;

The parameter combination determination subunit 1133 is also specifically used to perform pruning processing on the initial rendering parameter combination set based on the parameter sequence to obtain the pruned rendering parameter combination set, and to determine K rendering parameter combinations based on the pruned rendering parameter combination set.

In one embodiment, the parameter combination determining subunit 1133 is also specifically used to determine the device rendering parameter located at the end of the sequence as the rendering parameter to be pruned.

The parameter combination determination subunit 1133 is also specifically used to determine the initial rendering parameter combination that contains the rendering parameters to be pruned in the initial rendering parameter combination set as the rendering parameter combination to be pruned.

The parameter combination determination subunit 1133 is also specifically used to delete the rendering parameter combinations to be pruned in the initial rendering parameter combination set to obtain the pruned rendering parameter combination set.

In one embodiment, the parameter combination determining subunit 1133 is also specifically used to update the pruning attribute of the device rendering feature _Hj from an unpruned attribute to a pruned attribute.

The parameter combination determines subunit 1133, which is also specifically used to traverse R device rendering features;

The parameter combination determination subunit 1133 is also specifically used to determine the set of pruned rendering parameter combinations as K rendering parameter combinations if, among R device rendering features, the pruning attribute of each device rendering feature is a pruned attribute.

The parameter combination determination subunit 1133 is also specifically used to determine the unpruned rendering feature as an unpruned rendering feature if there is a device rendering feature with an unpruned attribute among the R device rendering features. Based on the device rendering parameter set corresponding to the unpruned rendering feature, the set of pruned rendering parameter combinations is pruned, and K rendering parameter combinations are determined based on the pruning result obtained from the pruning process.

In one embodiment, the R device rendering features include device input frame rate features; the device rendering parameter set corresponding to each device rendering feature includes the device input frame rate set corresponding to the device input frame rate feature.

The parameter set acquisition unit 112 may include: a type acquisition subunit 1121, a prediction value determination subunit 1122, a maximum frame rate determination subunit 1123, and a frame rate set determination subunit 1124.

The type acquisition subunit 1121 is used to acquire the media encoding types for which the terminal device has encoding and decoding permissions;

The prediction value determination subunit 1122 is used to obtain the maximum resolution corresponding to the terminal device and determine the maximum decoding frame rate prediction value corresponding to the terminal device based on the media encoding type and the maximum resolution.

The maximum frame rate determination subunit 1123 is used to obtain the screen refresh rate corresponding to the terminal device and determine the minimum value between the maximum decoded frame rate prediction value and the screen refresh rate as the maximum input frame rate of the terminal device.

The frame rate set determination subunit 1124 is used to determine the device input frame rate set corresponding to the device input frame rate feature based on the maximum input frame rate.

The specific implementation methods of the type acquisition subunit 1121, the prediction value determination subunit 1122, the maximum frame rate determination subunit 1123, and the frame rate set determination subunit 1124 can be found in the description of step S402 in the embodiment corresponding to Figure 4 above, and will not be repeated here.

In one embodiment, the frame rate set determining subunit 1124 is further configured to obtain an initial configuration frame rate set; the initial configuration frame rate set contains one or more initial configuration frame rates.

The frame rate set determination subunit 1124 is also specifically used to obtain an initial configuration frame rate that is less than or equal to the maximum input frame rate from one or more initial configuration frame rates;

The frame rate set determination subunit 1124 is also specifically used to determine the set of initial configuration frame rates that are less than or equal to the maximum input frame rate as the device input frame rate set corresponding to the device input frame rate feature.

In one embodiment, the test stream consists of M media data frames; the M media data frames include media data frame _Qj ; M and j are both positive integers; the rendering effect reference value corresponding to the rendering parameter combination _Si includes the average decoding rendering latency corresponding to the rendering parameter combination _Si .

The reference value determination module 13 may include: an input time acquisition unit 131, a rendering frame acquisition unit 132, an output time acquisition unit 133, a rendering delay determination unit 134, and an average delay determination unit 135.

The input timing acquisition unit 131 is used to acquire the frame input timing of media data frame _Qj input to the test device.

The rendering frame acquisition unit 132 is used to acquire the rendering frame corresponding to the media data frame _Qj from the rendering data of the test bitstream by the test device;

The output timing acquisition unit 133 is used to acquire the rendering frame corresponding to media data frame _Qj and output the frame output timing to the device display interface; the device display interface refers to the display interface of the test device.

The rendering delay determination unit 134 is used to determine the time period between the frame input time and the frame output time as the decoding rendering delay corresponding to the media data frame _Qj .

The average delay determination unit 135 is used to determine the average decoding and rendering delay corresponding to the rendering parameter combination _Si based on the decoding and rendering delay corresponding to each of the M media data frames.

The specific implementation methods of the input time acquisition unit 131, the rendering frame acquisition unit 132, the output time acquisition unit 133, the rendering delay determination unit 134, and the average delay determination unit 135 can be found in the description of step S102 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

In one embodiment, the average delay determination unit 135 may include: a summation processing subunit 1351, a quantity statistics subunit 1352, and a mean determination subunit 1353.

The summation processing subunit 1351 is used to perform summation operations on M decoding and rendering delays to obtain the total value of the decoding and rendering delays.

The quantity statistics subunit 1352 is used to count the total number of media data frames contained in M media data frames;

The mean determination subunit 1353 is used to determine the mean between the total value of decoding and rendering latency and the total number, so as to obtain the average decoding and rendering latency corresponding to the rendering parameter combination _Si .

The specific implementation methods of the summation processing subunit 1351, the quantity statistics subunit 1352, and the mean determination subunit 1353 can be found in the description of step S102 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

In one embodiment, the rendering effect reference value corresponding to the rendering parameter combination _Si refers to the average decoding rendering latency corresponding to the rendering parameter combination _Si ; the K rendering effect reference values refer to the K average decoding rendering latencies.

The optimal combination determination module 14 may include: minimum delay determination unit 141 and optimal combination determination unit 142.

The minimum delay determination unit 141 is used to obtain the minimum average decoding and rendering delay among K average decoding and rendering delays;

The optimal combination determination unit 142 is used to determine the rendering parameter combination with the minimum average decoding and rendering latency among the K rendering parameter combinations as the optimal rendering parameter combination for the terminal device.

The specific implementation of the minimum delay determination unit 141 and the optimal combination determination unit 142 can be found in the description of step S103 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

In this application embodiment, a method is provided to dynamically probe the rendering performance of different rendering parameter combinations on a terminal device based on a test bitstream. This method determines the optimal rendering parameters based on actual rendering effect reference data (rendering effect reference values). Compared to randomly configuring rendering parameters for the terminal device, probing different rendering parameter combinations through a test bitstream and determining the optimal combination based on actual rendering effect reference data can more accurately and specifically identify the optimal rendering parameter combination suitable for the terminal device, thereby improving its rendering performance. Furthermore, during the testing process, this application provides a probe pruning method that allows for a limited number of tests based on the default configuration combination, determining the optimal rendering parameter combination in less time and significantly improving testing efficiency.

Further, please refer to Figure 8, which is a schematic diagram of the structure of a computer device provided in an embodiment of this application. As shown in Figure 8, the data processing device 1 in the embodiment corresponding to Figure 7 can be applied to the computer device 8000. The computer device 8000 may include: a processor 8001, a network interface 8004, and a memory 8005. In addition, the computer device 8000 also includes: a user interface 8003, and at least one communication bus 8002. The communication bus 8002 is used to realize the connection and communication between these components. The user interface 8003 may include a display screen and a keyboard. Optionally, the user interface 8003 may also include a standard wired interface and a wireless interface. The network interface 8004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 8005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device. Optionally, the memory 8005 may also be at least one storage device located away from the aforementioned processor 8001. As shown in Figure 8, the memory 8005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

In the computer device 8000 shown in Figure 8, the network interface 8004 provides network communication functions; the user interface 8003 is mainly used to provide an input interface for the user; and the processor 8001 can be used to call the device control application stored in the memory 8005 to achieve:

Obtain K combinations of rendering parameters for the terminal device; K is a positive integer; each of the K combinations of rendering parameters contains one or more rendering parameters that affect the rendering performance of the terminal device; each rendering parameter that affects the rendering performance of the terminal device contains a combination of rendering parameters _Si , where i is a positive integer;

The test stream is input into the test device. Based on the rendering data of the test stream by the test device, the reference value of the rendering effect corresponding to the rendering parameter combination _Si is determined. The test device refers to the device obtained by switching the device rendering parameters of the terminal device to the rendering parameter combination _Si . The terminal device has the same device hardware parameters as the test device. The reference value of the rendering effect corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si .

Once the reference values for the rendering effects corresponding to each combination of rendering parameters are determined, the optimal combination of rendering parameters for the terminal device is determined from among the K combinations of rendering parameters based on the K reference values for rendering effects.

It should be understood that the computer device 8000 described in the embodiments of this application can execute the data processing method described in the embodiments corresponding to Figures 3 and 4 above, and can also execute the data processing device 1 described in the embodiment corresponding to Figure 7 above, which will not be repeated here. In addition, the beneficial effects of using the same method will not be repeated here either.

Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned data processing computer device 8000. This computer program includes program instructions, which, when executed by the processor, enable the execution of the data processing method described in the embodiments corresponding to Figures 3 and 4. Therefore, these descriptions will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

The aforementioned computer-readable storage medium can be an internal storage unit of the data processing apparatus or computer device provided in any of the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

One aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in one aspect of the embodiments of this application.

The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

The methods and related apparatus provided in this application are described with reference to the method flowcharts and/or structural diagrams provided in this application. Specifically, each block of the method flowcharts and/or structural diagrams, as well as combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, create means for implementing the functions specified in one or more blocks of the flowcharts and/or one or more blocks of the structural diagrams. These computer program instructions can also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more blocks of the flowcharts and/or one or more blocks of the structural diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and/or one or more blocks in the structural diagram.

The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims (13)

1. A data processing method, characterized in that it includes: Obtain R device rendering features configured for the terminal device; R is a positive integer; obtain the device rendering parameter set corresponding to each of the R device rendering features to obtain R device rendering parameter sets; based on parameter combination rules, determine the combination of the first target device rendering parameter in the target device rendering parameter set and the second target device rendering parameter in the remaining device rendering parameter set as the rendering parameter combination S _{_i}; i is a positive integer; the target device rendering parameter set is any device rendering parameter set among the R device rendering parameter sets; the remaining device rendering parameter set is the set of device rendering parameters among the R device rendering parameter sets other than the target device rendering parameter set; the first target device rendering parameter refers to any device rendering parameter in the target device rendering parameter set; the second target device rendering parameter refers to any device rendering parameter in the remaining device rendering parameter set; determine the combination of the remaining device rendering parameter in the target device rendering parameter set and the second target device rendering parameter as the remaining rendering parameter combination; the remaining device rendering parameter is any device rendering parameter in the target device rendering parameter set other than the first target device rendering parameter; according to the rendering parameter combination S_i _i is combined with the remaining rendering parameters to determine K rendering parameter combinations, where K is a positive integer; each of the K rendering parameter combinations contains one or more influencing rendering parameters; each influencing rendering parameter refers to a rendering parameter that affects the rendering performance of the terminal device. The test stream is input into the test device, and the rendering effect reference value corresponding to the rendering parameter combination _Si is determined based on the rendering data of the test stream by the test device. The test device refers to the device obtained by switching the device rendering parameters of the terminal device to the rendering parameter combination _Si . The rendering effect reference value corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si . Once the rendering effect reference value corresponding to each rendering parameter combination is determined, the optimal rendering parameter combination for the terminal device is determined from the K rendering effect reference values. 2. The method according to claim 1, wherein the R device rendering features include device rendering feature _Hj ; j is a positive integer; and the device rendering parameter set corresponding to each device rendering feature includes the device rendering parameter set _Pj corresponding to the device rendering feature _Hj ; The step of determining the K rendering parameter combinations based on the rendering parameter combination _Si and the remaining rendering parameter combinations includes: The rendering parameter combination _Si and the remaining rendering parameter combinations are both determined as initial rendering parameter combinations, and the set of the initial rendering parameter combinations is determined as the initial rendering parameter combination set. Obtain the parameter priority of each device rendering parameter in the device rendering parameter set _Pj corresponding to the device rendering feature _Hj ; According to the order of priority of each device rendering parameter in the device rendering parameter set _Pj , the device rendering parameters in the device rendering parameter set _Pj are sorted to obtain the parameter sequence; The initial set of rendering parameter combinations is pruned based on the parameter sequence to obtain a pruned set of rendering parameter combinations, and the K rendering parameter combinations are determined based on the pruned set of rendering parameter combinations. 3. The method according to claim 2, characterized in that, the step of pruning the initial rendering parameter combination set based on the parameter sequence to obtain a pruned rendering parameter combination set includes: The device rendering parameters located at the end of the sequence are determined as the rendering parameters to be pruned. The initial rendering parameter combination that includes the rendering parameters to be pruned in the initial rendering parameter combination set is determined as the rendering parameter combination to be pruned. The pruning rendering parameter combination set is obtained by deleting the pruning rendering parameter combination set from the initial rendering parameter combination set. 4. The method according to claim 2, wherein determining the K rendering parameter combinations based on the set of pruned rendering parameter combinations includes: Update the pruning attribute of the device rendering feature _Hj to the pruned attribute; Iterate through the R device rendering features; If the pruning attribute of each of the R device rendering features is a pruned attribute, then the set of pruned rendering parameter combinations is determined as the K rendering parameter combinations. If among the R device rendering features, there is a device rendering feature with an unpruned attribute, then the device rendering feature with an unpruned attribute is determined as an unpruned rendering feature. Based on the device rendering parameter set corresponding to the unpruned rendering feature, the set of pruned rendering parameter combinations is pruned. Based on the pruning result obtained from the pruning process, the K rendering parameter combinations are determined. 5. The method according to claim 1, wherein the R device rendering features include device input frame rate features; and the device rendering parameter set corresponding to each device rendering feature includes the device input frame rate set corresponding to the device input frame rate feature. The step of obtaining the set of device rendering parameters corresponding to each of the R device rendering features includes: Obtain the media encoding type for which the terminal device has encoding and decoding permissions; Obtain the maximum resolution corresponding to the terminal device, and determine the maximum decoding frame rate prediction value corresponding to the terminal device based on the media encoding type and the maximum resolution; Obtain the screen refresh rate corresponding to the terminal device, and determine the minimum value between the maximum decoded frame rate prediction value and the screen refresh rate as the maximum input frame rate of the terminal device; Based on the maximum input frame rate, determine the set of device input frame rates corresponding to the device input frame rate feature. 6. The method according to claim 5, wherein determining the device input frame rate set corresponding to the device input frame rate feature based on the maximum input frame rate includes: Obtain an initial configuration frame rate set; the initial configuration frame rate set contains one or more initial configuration frame rates; Among the one or more initial configuration frame rates, an initial configuration frame rate that is less than or equal to the maximum input frame rate is obtained; The set of initial configuration frame rates that are less than or equal to the maximum input frame rate is determined as the device input frame rate set corresponding to the device input frame rate feature. 7. The method according to claim 1, wherein the test bitstream consists of M media data frames; the M media data frames include media data frame _Qj ; M and j are both positive integers; the rendering effect reference value corresponding to the rendering parameter combination _Si includes the average decoding rendering latency corresponding to the rendering parameter combination _Si ; The step of determining the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test bitstream from the testing equipment includes: Obtain the frame input time of the media data frame _Qj input to the test device; In the rendering data of the test bitstream by the test device, the rendering frame corresponding to the media data frame _Qj is obtained; The frame output time of the rendered frame corresponding to the media data frame _Qj is obtained and output to the device display interface; the device display interface refers to the display interface of the test device. The time interval between the frame input time and the frame output time is determined as the decoding and rendering delay corresponding to the media data frame _Qj ; When the decoding and rendering latency corresponding to each of the M media data frames is determined, the average decoding and rendering latency corresponding to the rendering parameter combination _Si is determined based on the decoding and rendering latency corresponding to each media data frame. 8. The method according to claim 7, wherein determining the average decoding and rendering latency corresponding to the rendering parameter combination _Si based on the decoding and rendering latency corresponding to each media data frame includes: The total decoding and rendering delay is obtained by summing the M decoding and rendering delays. Count the total number of media data frames contained in the M media data frames; The mean between the total decoding and rendering latency and the total number is determined to obtain the average decoding and rendering latency corresponding to the rendering parameter combination _Si . 9. The method according to any one of claims 1-8, wherein the rendering effect reference value corresponding to the rendering parameter combination _Si refers to the average decoding rendering latency corresponding to the rendering parameter combination _Si ; and the K rendering effect reference values refer to the K average decoding rendering latencies. The step of determining the optimal combination of rendering parameters for the terminal device from among the K combinations of rendering parameters based on K rendering effect reference values includes: Among the K average decoding and rendering latencies, the minimum average decoding and rendering latency is obtained; The rendering parameter combination corresponding to the minimum average decoding and rendering latency among the K rendering parameter combinations is determined as the optimal rendering parameter combination for the terminal device. 10. A data processing apparatus, characterized in that it comprises: The combination acquisition module is used to acquire R device rendering features configured for the terminal device; R is a positive integer; acquire the set of device rendering parameters corresponding to each of the R device rendering features, resulting in R device rendering parameter sets; and, based on parameter combination rules, determine the combination of the first target device rendering parameter in the target device rendering parameter set and the second target device rendering parameter in the remaining device rendering parameter sets as the rendering parameter combination Si _. ; i is a positive integer; the target device rendering parameter set is any one of the R device rendering parameter sets; the remaining device rendering parameter set is the set of device rendering parameters other than the target device rendering parameter set among the R device rendering parameter sets; the first target device rendering parameter refers to any one of the device rendering parameters in the target device rendering parameter set; the second target device rendering parameter refers to any one of the device rendering parameters in the remaining device rendering parameter set; the combination formed by the remaining device rendering parameter in the target device rendering parameter set and the second target device rendering parameter is determined as the remaining rendering parameter combination; the remaining device rendering parameter is any one of the device rendering parameters other than the first target device rendering parameter in the target device rendering parameter set; K rendering parameter combinations are determined according to the rendering parameter combination _Si and the remaining rendering parameter combination, where K is a positive integer; each of the K rendering parameter combinations contains one or more influencing rendering parameters; each influencing rendering parameter refers to a rendering parameter that will affect the rendering performance of the terminal device; The bitstream input module is used to input the test bitstream to the test device; the test device refers to a terminal device whose device rendering parameters are the rendering parameter combination _Si ; The reference value determination module is used to determine the rendering effect reference value corresponding to the rendering parameter combination _Si based on the rendering data of the test bitstream by the test device; the rendering effect reference value corresponding to the rendering parameter combination _Si is used to characterize the rendering effect corresponding to the rendering parameter combination _Si . The optimal combination determination module is used to determine the optimal rendering parameter combination for the terminal device from the K rendering parameter combinations based on the K rendering effect reference values when the rendering effect reference values corresponding to each rendering parameter combination are determined. 11. A computer device, characterized in that it comprises: a processor, a memory, and a network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store computer programs, and the processor is used to call the computer programs to cause the computer device to execute the method according to any one of claims 1-9. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method according to any one of claims 1-9. 13. A computer program product, characterized in that the computer program product comprises a computer program stored in a computer-readable storage medium, the computer program being adapted to be read and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-9.