CN118625962A - Brightness control method and related device - Google Patents

Abstract

The application provides a brightness control method and related equipment, relating to the technical field of terminals, wherein the method comprises the following steps: the display screen displays a third interface; receiving a second operation; responding to the second operation, and judging whether the screen direction of the display screen rotates or not by the WMS; when the screen direction rotates, the WMS sends a second mark for indicating the rotation of the screen direction to the SF service module; and the SF service module is combined with the second identifier, and does not carry out the dimming treatment on the first screenshot of which the layer property is the SDR layer. According to the application, the second mark is added, so that the first screenshot can be determined for rotation by combining with the second mark, and therefore, the first screenshot for rotation can be subjected to no dimming treatment, and flicker in the rotation process is avoided.

Description

Brightness control method and related device

This application is a divisional application of the Chinese patent application submitted to the State Intellectual Property Office on February 24, 2023, with application number 202310207668.2 and application name "Brightness Control Method and Related Equipment".

Technical Field

The present application relates to the field of terminal technology, and in particular to a brightness control method and related equipment.

Background Art

At present, with the development of terminal technology, there are more and more electronic devices with image display functions, such as mobile phones or tablets. However, when electronic devices are displayed, for example, the maximum screen brightness of the display is about 500nit, and in most cases it will be set to only support the display content in the brightness range of about 0 to 200nit. However, the production end of pictures, videos, etc. can already support the production of HDR pictures and HDR videos of 0 to 1000nit. When HDR pictures and videos of 0 to 1000nit are displayed using existing 0 to 200nit displays, due to the brightness set by the display, HDR pictures and videos that should express wider details of bright and dark parts will not be well displayed; users can only see pictures and videos with a maximum brightness of 200nit when watching, which is a very bad experience.

In addition, the display interface of the electronic device is usually composed of multiple layers, that is, multiple layers are superimposed to form the display interface of the electronic device. In this way, when the screen brightness of the display interface is adjusted, some layers may display better after the corresponding brightness adjustment, while some layers may display poorly after the corresponding brightness adjustment.

In this regard, how to adjust the brightness to simultaneously meet the display effects of HDR layers and SDR layers has become a problem that needs to be solved urgently.

Summary of the invention

The present application provides a brightness control method and related equipment, which adjust the HDR layer and the SDR layer respectively to ensure that the overall display effect meets the visual requirements.

In a first aspect, a brightness control method is provided, which is applied to an electronic device including a display screen, a WMS and an SF service module, and the method includes:

The display screen displays a third interface;

receiving a second operation;

In response to a second operation, the WMS determines whether the screen direction of the display screen is rotated;

When the screen direction is rotated, the WMS sends a second identifier for indicating the screen direction rotation to the SF service module;

The SF service module combines the second identifier and does not perform darkening processing on the first screenshot whose layer property is an SDR layer.

In the native brightness control method for rotating scenes in the Android system, only tone mapping is performed on the HDR layer without dimming, and the SDR layer is dimmed. Furthermore, since the first screenshot obtained is an ordinary SDR layer, it will be dimmed once as a whole. In this way, it is equivalent to dimming the brightness of the area corresponding to the SDR layer in the screenshot twice, and dimming the area corresponding to the HDR layer in the screenshot once. This will cause visual flickering to the human eye during rotation, which does not meet viewing requirements. However, this application no longer dims the first screenshot, which is equivalent to retaining the effect of the previous layer-level brightness control, avoiding the phenomenon of rotating flashing black in the related technology.

In combination with the first aspect, in some implementations of the first aspect, the method further includes:

In response to the second operation, when the screen direction is rotated, the target application is configured with a first identifier for indicating a rotation scene, and the target application corresponds to the third interface;

Based on the first identifier, the WMS identifies that the scene type is a rotating scene;

The WMS sends the first identifier for indicating the rotating scene to the SF service module through the first interface.

In the implementation method, when the screen direction rotates, it can be determined that the current scene is a rotating scene. For the rotating scene, the WMS needs to send a first identifier for indicating the rotating scene to the SF service module, so that the SF service module can determine to process the layer according to the rotating scene based on the first identifier.

In combination with the first aspect, in certain implementations of the first aspect, before the SF service module combines the second identifier and does not perform darkening processing on the first screenshot whose layer property is an SDR layer, the method further includes:

In response to the second operation, a virtual screen is created, on which a fourth interface is displayed, the fourth interface being the same as the third interface;

In combination with the received first identifier, the SF service module performs tone mapping processing on the HDR layer included in the fourth interface and performs darkening processing on the included SDR layer according to the maximum screen brightness of the display screen;

Based on the processed HDR layer and the processed SDR layer, the first screenshot is obtained.

The dimming process refers to determining the dimming rate corresponding to the layer and processing the layer according to the determined dimming rate.

In the implementation, for the rotating scene type, in order to avoid the process of obtaining screenshots interfering with the normal display process on the display screen, in the present application, in response to the second operation, a virtual screen can be created separately, and the fourth interface displayed on the virtual screen is the same as the third interface displayed on the display screen. In this way, the present application can process the fourth interface on the virtual screen to obtain a screenshot that meets the display effect. For example, the HDR layer included in the fourth interface on the virtual screen can be tone mapped based on the preset screen brightness, and the SDR layer can be darkened. After the processed HDR layer and the SDR layer are synthesized, a first screenshot with the layer property of the SDR layer is obtained. Then, the first screenshot is no longer darkened.

In combination with the first aspect, in some implementations of the first aspect, the electronic device further includes a rotation animation module, and in response to the second operation, the WMS determines whether the screen direction of the display screen is rotated, including:

In response to the second operation, the target application determines whether to start the rotation animation module;

If so, the WMS determines whether the screen direction of the display screen is rotated.

The rotation animation module is used to control the rotation of the screen, so that the screen presents a vertical or horizontal layout.

In the implementation, by judging whether the rotation animation module is started, the screenshot scene can be distinguished when it is not started; when it is started, it is necessary to continue to judge whether the screen direction is rotated to distinguish the split screen scene from the rotation scene. If the screen direction is not rotated, it is a split screen scene, and if the screen direction is rotated, it is a rotation scene. Usually, the scene type is distinguished, and different brightness controls can be performed for different scenes later.

In combination with the first aspect, in some implementations of the first aspect, the WMS sending a second identifier for indicating screen direction rotation to the SF service module includes:

The WMS sends the second identifier to the SF service module through a second interface, where the second interface is different from the first interface.

In the implementation method, the second identifier is sent through the second interface, and the SF service module can determine the next frame display layer for the split-screen scenario by identifying the second identifier, thereby avoiding incorrect processing flow selection when processing the next frame display layer.

In combination with the first aspect, in some implementations of the first aspect, the electronic device further includes a brightness module, and the method further includes:

In response to a first operation on a first application, the display screen displays a first interface;

When the first application recognizes that the first interface includes an HDR layer, sending a first instruction to the brightness module;

After receiving the first instruction, the brightness module determines a target strategy and transmits it to the SF service module, where the target strategy includes brightness control parameters corresponding to the HDR layer and the SDR layer respectively;

The SF service module adjusts the brightness of the display layer based on the layer property of the display layer and the target strategy;

After adjustment, the data are sent to the display screen for display;

Among them, the layer properties of the display layer include HDR layer and/or SDR layer, and the brightness of the display layer whose property is HDR layer after brightness adjustment is greater than the brightness of the SDR layer whose property is SDR layer after brightness adjustment.

In the implementation method, for the first interface that includes both HDR layers and SDR layers, by identifying the nature of the layers, that is, by identifying whether the layer nature is an HDR layer or an SDR layer, different target strategies are determined for layers of different natures and the layer-level brightness is adjusted, thereby not only improving the brightness of the HDR layer, but also suppressing the brightness of the SDR layer, ensuring that the display effect of the HDR layer is improved and the display effect of the SDR layer is not dazzling, thereby making the overall display effect of the display interface better and more in line with visual needs.

In combination with the first aspect, in some implementations of the first aspect, the electronic device further includes a hardware synthesizer HWC, and the method further includes:

The SF service module obtains the maximum screen brightness of the display screen through the HWC;

The SF service module adjusts the brightness of the display layer based on the layer property of the display layer and the target strategy, including:

The SF service module performs tone mapping processing on the display layer whose layer property is an HDR layer based on the target strategy and the maximum screen brightness.

In the implementation method, by combining the maximum screen brightness, the maximum capacity of the display can be brought into play to improve the display effect of the HDR layer, so that the bright parts are brighter and the dark parts are darker when the HDR layer is displayed.

In a second aspect, an electronic device is provided, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes any one of the brightness control methods in the first aspect.

It should be understood that the expansion, limitation, explanation and description of the relevant contents in the above-mentioned first aspect also apply to the same contents in the second aspect.

In a third aspect, a brightness control device is provided, comprising a unit for executing any one of the brightness control methods in the first aspect.

In one possible implementation, when the brightness control device is an electronic device, the processing unit may be a processor, and the input unit may be a communication interface; the electronic device may also include a memory, which is used to store computer program code, and when the processor executes the computer program code stored in the memory, the electronic device executes any one of the methods in the first aspect.

In a fourth aspect, a chip system is provided, comprising: a processor, configured to call and run a computer program from a memory, so that a device equipped with the chip system executes any one of the brightness control methods in the first aspect or the second aspect.

In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes any one of the brightness control methods in the first aspect or any one of the brightness control methods in the second aspect.

In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes any one of the brightness control methods in the first aspect or any one of the brightness control methods in the second aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a scenario applicable to an embodiment of the present application;

FIG2 is a hardware system diagram of an electronic device provided in an embodiment of the present application;

FIG3 is a software system diagram of an electronic device provided in an embodiment of the present application;

FIG4 is a schematic diagram of a display process provided in an embodiment of the present application;

FIG5 is a schematic flow chart of a brightness control method provided in an embodiment of the present application;

FIG6 is a schematic diagram of the processing logic in the software system corresponding to FIG5;

FIG7 is a schematic diagram of a rotating scene provided in an embodiment of the present application;

FIG8 is a schematic diagram of a related interface provided in an embodiment of the present application;

FIG9 is a brightness control method in a rotating scene provided by an embodiment of the present application;

FIG10 is a schematic diagram of a data structure provided in an embodiment of the present application;

FIG11 is a schematic diagram of obtaining the maximum screen brightness provided by an embodiment of the present application;

FIG12 is a software system diagram of another electronic device provided in an embodiment of the present application;

FIG13 is a schematic diagram of a screen projection scenario provided in an embodiment of the present application;

FIG14 is a schematic diagram of a related interface for enabling the screen projection function provided in an embodiment of the present application;

FIG15 is a schematic diagram of an interface related to enabling the screen recording function provided in an embodiment of the present application;

FIG16 is a brightness control method in a screen projection scenario and a screen recording scenario provided by an embodiment of the present application;

FIG17 is a schematic diagram of the structure of an electronic device provided in the present application.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

1. Brightness: the luminous intensity per unit projection area, quantified in nit or cd/ ^m2 .

2. Layers (surfaces), such as films containing text or graphics, need to be stacked together in a certain order to form the final effect of the page. Layers can accurately position elements on the page.

3. Surface flinger (SF): A system service process that controls the synthesis, where the layers involved in the display, visible areas, etc. are calculated.

4. System applications (applications, APPs) and third-party APPs. APPs in electronic devices can generally be divided into system APPs and third-party APPs. System APPs may be APPs provided or developed by manufacturers of electronic devices. Manufacturers of electronic devices may include manufacturers, suppliers, providers or operators of electronic devices. For example, system APPs in electronic devices may include calls, messages, contacts, cameras, recordings, and so on. Third-party APPs refer to APPs provided or developed by software application developers other than the manufacturers of electronic devices. For example, Youku, Douyin, etc. Third-party APPs can provide users with corresponding functions by calling one or more system functions in electronic devices. For example, Youku and Douyin can provide users with the function of watching film and television works, etc.

5. Application interface (UI): UI refers to the media interface for interaction and information exchange between applications or operating systems and users, which can realize the conversion between the internal form of information and the form that users can receive. The application interface is the source code written in specific computer languages such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the electronic device and finally presented as content that users can recognize. The common form of application interface is the graphical application interface (GUI), which refers to the application interface related to computer operations displayed in a graphical way. It can be visual interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of electronic devices.

6. Foreground running, background running and closed, the APP corresponds to three running states: foreground running, background running and closed. Among them, when the APP runs in the foreground of an electronic device, the electronic device can display the application interface of the APP on the display screen, and the user can interact with the APP through the controls in the application interface of the APP. The APP runs in the electronic device, but it does not belong to the above-mentioned situation of running in the foreground, that is, the APP runs in the background of the electronic device. The situation where the APP runs in the background of the electronic device may include the existence of the APP process in the electronic device, but the application interface of the APP is not displayed on the display screen. Since the application interface of the APP running in the background is not displayed on the display screen, the user usually cannot directly interact with the APP running in the background. Closed means that the APP is not running on the electronic device.

It should be understood that there may be one or more APPs running in the foreground of the electronic device, and there may also be one or more APPs running in the background of the electronic device. Among them, when there are one or more APPs running in the foreground of the electronic device, there may also be one or more APPs running in the background of the electronic device.

7. Process: A process is an application's running activity on a certain data set. It is the basic unit for resource allocation and scheduling in an operating system (such as the Android system). Each process occupies a piece of memory space. An application runs on the operating system in the form of one or more processes to achieve corresponding functions.

The above is a brief introduction to the terms involved in the embodiments of the present application, which will not be repeated below.

At present, with the development of terminal technology, there are more and more electronic devices with image display functions, such as mobile phones or tablets. However, when electronic devices are displayed, for example, the maximum screen brightness of the display is about 500nit, and in most cases it will be set to only support the display content in the brightness range of about 0 to 200nit. However, in order to approach the brightness that the human eye can perceive, the production end of pictures, videos, etc. can already support the production of 0-1000nit HDR pictures and HDR videos. When 0-1000nit HDR pictures and videos are displayed using existing 0-200nit displays, due to the brightness set by the display, HDR pictures and videos that should express wider bright and dark details will not be well displayed; users can only see pictures and videos with a maximum brightness of 200nit when watching, which is a very bad experience.

In addition, the display interface of the electronic device is usually composed of multiple layers, that is, multiple layers are superimposed to form the display interface of the electronic device. In this way, when the screen brightness of the display interface is adjusted, some layers may display better after the corresponding brightness adjustment, while some layers may display poorly after the corresponding brightness adjustment.

The display interface displayed by the electronic device is composed of multiple layers. For example, in the scenario of non-full-screen display of HDR pictures or videos, the display interface synthesized in the electronic device may include an HDR layer and an SDR layer. The HDR layer is used to indicate the layer of display content that displays the HDR picture or video. The SDR layer usually includes other layers in the display interface, such as the status bar layer, the navigation bar layer, etc. Compared with the SDR layer, the HDR layer can use more types of colors and a wider range of color representation, which can support richer image color expression and more vivid image detail expression, especially the dark and bright parts of the picture can present more details.

It should be understood that the above description of the layer synthesis display interface takes non-full screen display as an example. When in full screen display, the layers of the display interface can also be synthesized by multiple layers, and can also include HDR layers and SDR layers. In this regard, the embodiments of the present application do not impose any restrictions on this. Usually, full screen means that the area occupied by the displayed video or picture on the screen is the entire physical display area, and non-full screen means that the area occupied by the displayed video or picture on the screen is only a part of the entire physical display area. Non-full screen and full screen are relative.

Exemplarily, FIG1 shows a schematic diagram of a composite display interface of HDR layers and SDR layers. Referring to FIG1 , an electronic device displays a non-full-screen video playback interface in a gallery APP. The interface can be composed of multiple layers such as the status bar layer shown in FIG1 (a), the UI interface of the gallery, the layer corresponding to the HDR video, and the volume adjustment layer. The composite display effect is shown in FIG1 (b). In the schematic diagram shown in FIG1 (a), except for the layer corresponding to the HDR video, which is an HDR layer, all other layers are SDR layers.

Assume that when the current electronic device displays pictures or videos in a non-full screen, due to hardware limitations, the screen brightness corresponding to all layers is pre-set to 150nit. In order to improve the display effect of HDR video, when the electronic device supports a maximum of 500nit, the user wants to increase the screen brightness to the maximum screen brightness of 500nit. At this time, after the SDR layer and the HDR layer shown in Figure 1 are superimposed with the maximum screen brightness, the overall brightness will be increased to 500nit. In this way, although the display effect of HDR video is improved, the area where the HDR video is not displayed, or the area where the SDR layer is displayed, becomes very dazzling, and the display effect corresponding to the SDR layer will bring negative effects such as visual discomfort and fatigue to the user.

Therefore, in the case where the display interface includes both HDR layers and SDR layers, how to adjust the brightness to simultaneously meet the display effects of both HDR layers and SDR layers has become an urgent problem that needs to be solved.

In view of this, an embodiment of the present application provides a brightness control method. For a display interface that includes both HDR layers and SDR layers, the properties of the layers are identified, that is, by identifying whether the layer properties are HDR layers or SDR layers, different target strategies are determined for layers of different properties and layer-level brightness is adjusted, thereby not only improving the brightness of the HDR layer, but also suppressing the brightness of the SDR layer, ensuring that the display effect of the HDR layer is improved and the display effect of the SDR layer is not dazzling, thereby making the overall display effect of the display interface better and more in line with visual needs.

In addition to mobile phones, the electronic devices provided in the embodiments of the present application can also be tablet computers, personal computers (PCs), personal digital assistants (PDAs), smart watches, netbooks, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, vehicle-mounted devices, smart cars, robots, smart glasses, smart televisions, and other devices with displays that can realize display functions.

FIG. 2 shows a hardware system of an electronic device suitable for the present application.

As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

It should be noted that the structure shown in FIG2 does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in FIG1, or the electronic device 100 may include a combination of some of the components shown in FIG2, or the electronic device 100 may include sub-components of some of the components shown in FIG2. The components shown in FIG2 may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated devices. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a SIM interface, and a USB interface.

Exemplarily, the processor 110 provided in the embodiment of the present application can execute the following method: the display screen displays a third interface; a second operation is received; in response to the second operation, the WMS determines whether the screen direction of the display screen is rotated; when the screen direction is rotated, the WMS sends a second identifier for indicating the rotation of the screen direction to the SF service module; the SF service module combines the second identifier and does not perform darkening processing on the first screenshot whose layer property is an SDR layer.

The connection relationship between the modules shown in Fig. 2 is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of multiple connection modes in the above embodiments.

The wireless communication function of the electronic device 100 can be implemented through components such as the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The electronic device 100 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

Display screen 194 may be used to display images or videos.

Optionally, the display screen 194 can be used to display images or videos. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED) or a quantum dot light emitting diode (QLED). In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted to the camera photosensitive element through the camera. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene and other parameters. In some embodiments, the ISP can be set in the camera 193.

The camera 193 (also referred to as a lens) is used to capture static images or videos. It can be triggered to start by application instructions to realize the photo function, such as taking pictures of any scene. The camera may include components such as an imaging lens, a filter, and an image sensor. The light emitted or reflected by the object enters the imaging lens, passes through the filter, and finally converges on the image sensor. The imaging lens is mainly used to focus the light emitted or reflected by all objects in the camera angle (also referred to as the scene to be photographed, the target scene, or the scene image that the user expects to shoot) to form an image; the filter is mainly used to filter out excess light waves in the light (for example, light waves other than visible light, such as infrared); the image sensor can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor is mainly used to perform photoelectric conversion on the received light signal, convert it into an electrical signal, and then transmit the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV, etc. format.

Exemplarily, the digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

Exemplarily, a video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

Exemplarily, the gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, the y-axis, and the z-axis) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used in scenarios such as navigation and somatosensory games.

For example, the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally the x-axis, y-axis, and z-axis). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. The acceleration sensor 180E can also be used to identify the posture of the electronic device 100 as an input parameter for applications such as horizontal and vertical screen switching and pedometers.

Exemplarily, the distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, for example, in a shooting scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Exemplarily, the ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch.

For example, the fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement functions such as unlocking, accessing application locks, taking photos, and answering calls.

Exemplarily, the touch sensor 180K is also referred to as a touch control device. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also referred to as a touch control screen. The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor 180K may pass the detected touch operation to the application processor to determine the type of touch event. A visual output related to the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, and may be disposed at a different position from the display screen 194.

The hardware system of the electronic device 100 is described in detail above. An operating system is run on the above hardware. The operating system of the operating system layer can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system or Windows operating system, etc. Applications can be installed and run on the operating system.

FIG3 is a software system diagram of an electronic device provided in an embodiment of the present application.

The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer (applications), application framework layer (application framework), Android runtime (AndroidRuntime) and system library, and kernel layer (kernel).

The application layer may include a series of application packages. For example, the application layer may include desktop APP, system user interface (systemUI), game APP, gallery APP, camera APP, browser APP and other applications (applications may be referred to as applications for short), and the present application embodiment does not impose any restrictions on this.

Applications in the application layer can be divided into system applications and non-system applications. System applications can specifically include a desktop launcher (launcher), a system user interface (systemUI), a gallery APP, a camera APP, and a browser APP, etc. Non-system applications can include a game APP, etc.

These apps are generally written in Java, and each app includes one or more class files. Each app can run its own class file in the application as a process. When a user operates in the app, the app can interact with the system library or kernel layer by calling the relevant API or service in the application framework to achieve the corresponding kinetic energy.

The application layer includes a launcher that is used to start the application activity.

Exemplarily, in response to a user clicking an icon corresponding to a camera application, the launcher sends a request to start the application activity to an activity manager service (AMS).

The gallery APP is used to provide users with picture display and video playback functions.

Exemplarily, in response to a user clicking on a video to play, the gallery APP calls a codec to decode the video, and sends a request to start a video playback activity to the activity manager.

The application framework layer provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer includes some predefined functions. As shown in FIG3 , the application framework layer may include an activity manager, a window manager service (WMS), a view system service (VSS), etc., and the embodiments of the present application do not impose any restrictions on this.

The activity manager is used to manage the life cycle of each application and the navigation fallback function. It is responsible for creating the Android main thread and maintaining the life cycle of each application.

The window manager is used to manage window programs. The window manager can obtain the display screen size and the size and shape of the volume bar, and determine whether the size and shape of the volume bar needs to be adjusted.

In the embodiment of the present application, the window manager can, for example, take screenshots, split the screen, etc. The screenshot indicates capturing the content displayed on the screen and correspondingly generating a frame of image; the split screen indicates dividing the display screen into multiple display areas and displaying different content at the same time.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, in an embodiment of the present application, the view system can be used to display a non-full-screen video playback interface.

In an embodiment of the present application, the application framework layer may further include a brightness module, which is used to adjust the brightness strategy, for example, selecting different brightness strategies for the HDR layer and SDR layer included in the display interface.

Exemplarily, the embodiment of the present application will identify different layer properties, determine whether the layer corresponds to an HDR layer or an SDR layer, and then suppress the brightness of the SDR layer and only increase the brightness of the HDR layer.

Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

The system library can include multiple functional modules, such as SF service module (surface flinger service), media libraries, 3D graphics processing library (such as openGL ES), 2D graphics engine (such as SGL), etc. The SF service module is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

Exemplarily, as shown in FIG3 , the SF service module may include SF, Libtonemap, Libsharder, etc. SF is used to control the system service process of synthesis, and in the embodiment of the present application, participates in calculating the displayed layers, visible areas, etc. Libtonemap is a module for controlling the HDR tonemap algorithm. Libsharder is responsible for generating the Sksl language during the GPU synthesis process, and calling the GPU for corresponding processing according to the generated sharder.

Exemplarily, the system library may also include a CPU distributed processing unit (data processing unit, DPU), Skia, and Sksl. Skia indicates a 2D rendering engine, providing a general API applicable to various hardware and software platforms. Sksl indicates that the Skia 2D rendering engine uses a shader language, which is converted into opengl by Skia, and then compiled and linked to generate a program that is a language that can be directly recognized by the GPU; in the embodiment of the present application, Sksl can be directly regarded as a program language that can be directly recognized by the GPU.

The kernel layer is the layer between hardware and software. The kernel layer includes at least the hardware synthesizer (hwcomposer, HWC) driver, graphics processing unit (graphics processing unit, GPU) driver, display driver, graphics buffer manager (gralloc) driver, camera driver, audio driver, sensor driver, etc.

GPU is a general-purpose image processing device used for synthesis. HWC is a special-purpose image processing device, which is used to synthesize layers that cannot be synthesized by GPU in the embodiment of the present application, or to synthesize layers synthesized based on GPU with other layers. HWC can also perform post-processing. For example, post-processing may include: video stabilization, color enhancement (CE), 3D lookup table, i.e., style transformation (3D look up table, 3DLUT), super-resolution reconstruction, etc. Gralloc can be understood as a graphics memory allocator, which is used to allocate and manage memory requested by image producers.

It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

The following takes the electronic device as a mobile phone as an example, and the operating system as the above-mentioned Android system as an example, and briefly describes the display process of the electronic device.

FIG4 shows a schematic diagram of a display process. As shown in FIG4, for the non-full-screen video playback interface shown in FIG1, the display process corresponding to all SDR layers is generally divided into four stages: drawing, rendering, synthesis, and hardware refresh display; the display process corresponding to the played HDR video (i.e., HDR layer) is generally divided into four stages: video file parsing, video decoding, synthesis, and hardware refresh display. Hardware refresh display is used to instruct the display screen to refresh the display.

By comparing the two, it can be seen that the display processes corresponding to the SDR layer and the HDR layer need to be processed in the synthesis stage, and the synthesis stage is usually implemented by the SF service module in the Android system shown in Figure 3. Therefore, in order to adjust and optimize both the HDR layer and the SDR layer, the brightness control method provided in the embodiment of the present application can be applied in the synthesis stage, that is, applied to the SF service module. When the brightness control method provided in the embodiment of the present application is applied in the SF service module, the purpose of layer-level brightness control of the SDR layer and the HDR layer can be achieved.

In combination with Figures 1 to 4, Figure 5 shows a brightness control method provided by an embodiment of the present application, and Figure 6 is a schematic diagram of the processing logic in the software system corresponding to Figure 5. As shown in Figure 5, the brightness control method 10 is applied to the electronic device described above, and the brightness control method 10 may include the following S11 to S15, and steps S11 to S15 are introduced respectively below.

S11. In response to a first operation on a first application, a display screen displays a first interface.

It should be understood that the first application can indicate system applications such as gallery and video, and can also indicate non-system applications such as WeChat, Youku, and Douyin. The first operation can include a click operation, a double-click operation, a slide operation, and can also be other operations such as voice operation, and this application does not make any limitation on this.

It should be understood that the first interface may include a video playback interface and/or a picture display interface. The picture may be an HDR picture or an SDR picture, and the video may be an HDR video or an SDR video. The first interface may be displayed in full screen or in non-full screen on the display screen of the electronic device, and the display screen is a physical screen.

For example, in response to a user clicking a video icon in the gallery APP for indicating a video, the electronic device may display a first interface as shown in (b) of FIG1 , wherein the first interface includes a non-full-screen video playback interface. Here, the non-full-screen video playback interface is composed of three SDR interfaces and one HDR interface superimposed on each other.

S12: When the first application recognizes that the first interface includes an HDR layer, a first instruction is sent to the brightness module.

The first instruction is used to instruct selection of a corresponding brightness strategy based on properties of a layer included in the first interface, wherein, when the first application recognizes that the first interface includes an HDR layer, the first instruction may carry a first identifier indicating that the first interface includes an HDR layer.

It should be understood that the identification method used by the first application to identify whether the first interface includes an HDR layer and the brightness strategy can be pre-set, and the specific content can be set and adjusted as needed. The embodiments of the present application do not impose any limitations on this.

S13. After receiving the first instruction, the brightness module determines the target strategy and transmits it to the SF service module.

Optionally, after receiving the first instruction, the brightness module determines the target policy, and then transmits the target policy to the SF service module through the setDisplayBrightness interface of the Window Manager Service (WMS). Further, the brightness module can transmit the target policy to the SF in the SF service module.

It should be understood that the target policy may include multiple brightness control parameters. For example, the target policy may include a brightness control parameter for an HDR layer in the first interface and a brightness control parameter for an SDR layer in the first interface, or may include a brightness control parameter for each layer in the first interface, wherein the target policy may include a brightness control parameter for each HDR layer in the first interface and a brightness control parameter for an SDR layer.

That is to say, the target strategy can be divided based on the layer properties of the HDR layer and the SDR layer, and different brightness control parameters can be set for different layer properties, so that the brightness of the displayed HDR layer and the SDR layer is different during subsequent adjustments. Alternatively, the target strategy can also set different brightness control parameters for each layer, so that if the first interface includes multiple HDR layers and multiple SDR layers, the multiple HDR layers can be adjusted to different brightness based on the different brightness control parameters that are set, and the multiple SDR layers can also be adjusted to different brightness based on the different brightness control parameters that are set.

The brightness control parameters of the HDR layer and the brightness control parameters of the SDR layer include the SDR white point brightness and the current screen brightness, and the unit is nit.

For example, assuming that the preset brightness strategy includes a first strategy and a second strategy, when the first interface includes an HDR layer, the first strategy needs to be used for adjustment. After the first application sends a first instruction to the brightness module, the first instruction can carry an identifier for indicating that the first interface includes an HDR layer. In this way, after receiving the first instruction, the brightness module can determine that the first interface includes an HDR layer based on the first instruction, and then determine that the target strategy corresponding to the HDR layer is the first strategy.

When the first interface does not include an HDR layer or includes an SDR layer, the second strategy needs to be used for adjustment. The first application can send a second instruction to the brightness module. The second instruction can carry an identifier for indicating that the first interface does not include an HDR layer, or the second instruction can not carry an identifier for indicating that the first interface includes an HDR layer. In this way, after the first application sends the second instruction to the brightness module, the brightness module can determine that the first interface does not include an HDR layer based on the second instruction, and then determine that the target strategy corresponding to not including the HDR layer is the second strategy.

S14. The SF service module adjusts the brightness of the display layer based on the properties of the display layer and the target strategy. After the brightness adjustment, the brightness of the display layer whose properties are HDR layers is greater than the brightness of the display layer whose properties are SDR layers.

It should be understood that the display layer may indicate the layer that needs to be synthesized in the synthesis stage for the next display interface that the electronic device is about to display after the first interface. The multiple display layers used for synthesis may include HDR layers and SDR layers, and the number of HDR layers and SDR layers may include one or more, and the embodiments of the present application do not impose any limitation on this.

Exemplarily, as shown in FIG6 , when the nature of the display layer is an SDR layer, SF can send the brightness control parameters corresponding to the received SDR layer to skiaGlenderEngine, and then use skiaGlenderEngine to determine the dimming ratio of the SDR layer based on the brightness control parameters; then, the SDR layer is processed in Libsharder based on the determined dimming ratio, and the processed SDR layer is then transmitted to the GPU or HWC for synthesis. It should be understood that skiaGlenderEngine is responsible for GPU synthesis.

When the nature of the display layer is an HDR layer, SF can send the brightness control parameters and preset screen brightness corresponding to the received HDR layer to Libsharder. Then, Libsharder can process the HDR layer based on the brightness control parameters, preset screen brightness, and tone mapping parameters provided by Libtonemap. The processed HDR layer is then transmitted to the GPU or HWC for synthesis.

Optionally, as shown in Figure 6, the SF in the SF service module can also obtain the maximum brightness of the display screen (or maximum screen brightness) through HWC, and then the SF service module performs tone mapping on the HDR layer based on the received brightness control parameters, the received maximum screen brightness, and the tone mapping parameters provided by Libtonemap, that is, adjusts the brightness of the HDR layer.

Tone mapping is a computer graphics technology that approximates the display of high dynamic range images on limited dynamic range media. The tone mapping algorithm will optimize the HDR display layer according to the color space of the HDR display layer and the color space of the display output, as well as the maximum screen brightness of the display, to ensure that the content of the HDR layer can better adapt to different displays, thereby obtaining better display effects.

After using Libsharder to process the display layers of SDR layers and HDR layers, the processed layers can be sent to the GPU for synthesis. Then, when the layer synthesis results corresponding to the GPU do not need to be processed further, they can be sent to the display screen for display after being transmitted via the HWC; or, the layer synthesis results corresponding to the GPU can also be transmitted to the Frame buffer included in the SF service module for temporary storage. Subsequently, the HWC can call the temporarily stored layer synthesis results from the Frame buffer for further synthesis with other layers, and then send the final synthesis results to the display screen for display.

Optionally, as shown in FIG6 , before tone mapping is performed on the display layer of the HDR layer, AGP services can also read the parameters corresponding to the previous frame when the HWC calls the 3D LUT to generate the display result, and send it to Libsharder through Libtonemap, so that the parameters can be combined and processed when tone mapping is performed on the display layer in the next frame.

It should be understood that by adjusting the parameters corresponding to the display interface generated by HWC calling 3D LUT during tone mapping, the adjusted display layer can be kept consistent with the GPU and DPU processing effects, avoiding display effect jumps.

It should be understood that when adjusting the display layer that is an SDR layer or an HDR layer in nature, in combination with the brightness control parameters, the maximum screen brightness and the parameters provided by the 3D LUT, the brightness control parameters can also be dynamically changed based on the specific display content of the display layer to adapt to the display requirements of the display content.

After adjustment, the brightness of the HDR layer is greater than the brightness of the SDR layer. In this way, when it is displayed subsequently, it can not only meet the display effect of the HDR layer, improve the presentation of details and the contrast between light and dark, but also meet the display requirements of the SDR layer without being too abrupt, thus avoiding visual discomfort and fatigue for users when watching.

S15. The SF service module sends the adjusted display layer to the display screen for display.

Optionally, the SF service module may send the adjusted display layer to the GPU for synthesis and then transmit it to the display screen for display, or send the adjusted display layer to the GPU and HWC for synthesis and then transmit it to the display screen for display.

It should be understood that the adjusted display layer can be called the second interface when displayed on the display screen.

In the brightness control method provided in the embodiment of the present application, by identifying that the first displayed interface includes an HDR layer, the brightness module is triggered to select a target strategy and transmit the target strategy to the SF service module; then, the SF service module can calculate the dimming rate corresponding to the SDR layer based on the target strategy (brightness control parameters corresponding to the SDR layer), and then adjust the display layer of the SDR layer in combination with the dimming rate; the SF service module can also adjust the display layer of the SDR layer in combination with the target strategy (brightness control parameters corresponding to the HDR layer), the preset screen brightness or the maximum screen brightness obtained, the parameters sent by Libtonemap, etc. After adjustment, the brightness of the display layer of the HDR layer is greater than the brightness of the display layer of the SDR layer.

In this way, after the adjusted HDR layer and SDR layer are synthesized and displayed on the display screen, the brightness of the HDR layer can be improved to ensure the display effect of the HDR layer, and the brightness of the HDR layer can be made higher than that of the SDR layer, so that the display of the SDR layer will not be too abrupt and affect the user's visual experience when watching.

On the basis of the above, since electronic devices may involve various display scenarios such as screenshots, split screens, rotation, screen recording, and screen projection during the display process, in order to improve the display effect in each scenario, this application provides corresponding brightness control methods for each scenario, so as to adaptively adjust and optimize the display effect corresponding to each scene, solve some problems such as flickering, and uneven effects after pictures and videos are projected or sent to the opposite device, and ensure that the display in each scenario can meet the visual requirements.

The following describes in detail the rotating scene and the brightness control method in the rotating scene in conjunction with FIG. 7 to FIG. 11 .

Exemplarily, FIG7 shows a schematic diagram of a rotation scene. Taking the electronic device as a mobile phone as an example, when the mobile phone receives a trigger operation from a user for an icon indicating a video in a gallery APP, an interface as shown in (a) of FIG7 may be displayed, which may be a non-full-screen video playback interface 201. The interface may include a status bar, a playback interface corresponding to an HDR video, a volume adjustment control, a rotation icon 202, etc., and the rotation icon 202 is used to indicate that the playback interface is rotated. In response to a user's click operation on the rotation icon 202, the non-full-screen playback interface 201 can be rotated 90 degrees, and the display method after rotation can be shown in (b) of FIG7.

Exemplarily, FIG8 shows a schematic diagram of a related interface. As shown in (a) of FIG8 , a control 203 for controlling screen rotation may be included in the interface 203 of the drop-down menu of the electronic device; subsequently, in response to a user's click operation on the control 203, the control 203 may be in a selected state, and accordingly, the function of screen rotation is turned on. As shown in (b) of FIG8 , the setting interface 205 of the electronic device may also include an icon for controlling screen rotation and a corresponding switch control 206. For example, on the setting interface 205, when the user clicks on the switch control 206, the switch control 206 may be in an on state, and accordingly, the function of screen rotation is turned on. Subsequently, when the user rotates the mobile phone, the interface displayed on the electronic device may switch from portrait mode to landscape mode, or may switch from landscape mode to portrait mode.

In combination with the rotation scenes shown in FIG. 7 and FIG. 8 and the related display interface for turning on the screen rotation function, FIG. 9 shows a brightness control method in a rotation scene. As shown in FIG. 9 , the method 20 is applied to an electronic device including a display screen. The method 20 may include the following S21 to S28, and steps S21 to S28 are described in detail below.

S21. The display screen displays the third interface.

The third interface is used to indicate a video playback interface or a picture display interface, and the video playback interface or the picture display interface can be displayed in a non-full screen or in a full screen. The picture can be an HDR picture or an SDR picture, and the video can be an HDR video or an SDR video.

The third interface may be the same as the first interface or may be different. If the third interface is the same as the first interface, reference may be made to the description of the first interface shown in (b) of FIG1 , which will not be repeated here.

In addition, the third interface may be a single-task interface or a multi-task interface. The single-task interface may be an interface for displaying pictures or videos, such as a non-full-screen video playback interface as shown in (b) of FIG1. The multi-task interface may be an interface in which multiple layers are arranged in a superimposed or partitioned manner. For example, after the screen is split, the upper half of the screen displays an HDR video, and the lower half displays a chat interface in the WeChat APP.

S22: Receive a second operation from the user.

Exemplarily, the second operation can be a pressing operation on a physical button, a clicking operation on a virtual button, a double-click operation with a knuckle on the display screen, a sliding operation, a dragging operation on an icon displayed on the display screen, a rotation operation on the physical space of the display screen, or a voice operation, etc. The embodiments of the present application do not impose any limitations on this.

It should be noted that the second operation here can be called a screenshot operation, and the screenshot operation can include operations for triggering screenshot, triggering rotation, and triggering split screen.

For example, the screenshot operation may be a click operation on a rotation icon displayed on the display screen, such as a click operation on the rotation icon 201 as shown in (a) of FIG7 . Alternatively, the screenshot operation may be a rotation operation on the display screen, rotating the display screen from a horizontal screen posture by 90 degrees in physical space to a vertical screen posture, or rotating from a vertical screen posture by 90 degrees to a horizontal screen posture.

S23. In response to the second operation, the target application is configured with a first identifier for indicating different scene types; the scene types at least include screenshot, rotation, and split screen.

Here, the target application is used to indicate the application program corresponding to the third interface.

Figure 10 shows a schematic diagram of a data structure. As shown in Figure 10, the target application can modify the data structure included in the electronic device, and add an int type flag in CaptrueArgs to mark different scene types, wherein the flag value can default to 0 to indicate that the scene type defaults to the screenshot scene.

Exemplarily, if the WMS does not start the rotation animation module, there is no need to modify the flag value, that is, the flag value continues to be configured as 0 to indicate that the scene type is a screenshot scene.

In response to the second operation, if WMS starts the rotation animation module, the target application can continue to determine whether the screen is rotated. If so, it can determine that the scene type is a rotation scene and configure the flag value corresponding to mScreenShotFlag to 1 to indicate that the current scene is a rotation scene.

If there is no rotation, it can be determined that the scene type is a split-screen scene, and the flag value corresponding to mScreenShotFlag is set to 2 to indicate that the current scene is a split-screen scene.

It should be understood that the size of the flag value corresponding to mScreenShotFlag and the corresponding schematic relationship with the scene type can be set and adjusted as needed, and the embodiments of the present application do not impose any limitations on this.

It should be understood that the present application is based on configuring different first identifiers for different scene types, so that the subsequent SF service module can identify the scene type based on the first identifier and can perform different brightness controls for different scenes.

S24. The WMS identifies the scene type by identifying the first identifier.

The WMS can obtain the flag value corresponding to mScreenShotFlag in CaptrueArgs, and then identify the type of scene triggered by the user based on the size of the value (ie, the first identifier).

For example, when WMS obtains that the flag value corresponding to mScreenShotFlag in CaptrueArgs is 1, it can be determined that the current scene type is a rotating scene.

The WMS sends a first identifier for indicating a rotating scene to the SF service module.

For example, the WMS may use the first interface in the native path of the Android system to send the first identifier to the SF service module; for example, the first interface may indicate the setDisplayBrightness interface.

S25. In response to the second operation, the SF service module creates a virtual screen, and the interface on the virtual screen is the fourth interface.

It should be understood that the virtual screen is a simulated screen, such as a display instance (display), and the interface on the virtual screen is not displayed to the user. The display screen described previously in this application is a physical screen, and the virtual screen corresponds to the display screen, that is, the size and resolution of the virtual screen and the display screen are the same. In the embodiment of the present application, the fourth interface corresponds to the third interface, the only difference is that the third interface is displayed to the user, and the fourth interface is not displayed to the user. Apart from this, the fourth interface and the third interface are the same.

It should be understood that the interface on the virtual screen created by the SF service module is the same as the interface on the display screen, including multiple SDR layers and/or HDR layers. For example, when the third interface displayed on the display screen is formed by superimposing 3 SDR layers and 1 HDR layer, the interface on the virtual screen created by the SF service module should also be formed by superimposing the same 3 SDR layers and 1 HDR layer. Here, in order not to affect the normal display of the display screen, the SF service module can be used to create a virtual screen, and the data on the virtual screen that is not displayed to the user can be used for subsequent processing.

It should also be understood that step S24 and step S25 may be performed in a certain order during the actual processing, and step S24 is usually performed first and step S25 is performed later.

S26. When the identified scene type is a rotating scene, tone mapping is performed on all HDR layers in the fourth interface according to the maximum screen brightness, and darkening is performed on all SDR layers.

Among them, dimming processing refers to determining the dimming rate corresponding to the SDR layer, and then processing the SDR layer according to the determined dimming rate.

Optionally, a GPU is used to synthesize all processed HDR layers and darkened SDR layers to obtain a first screenshot; then, the synthesized first screenshot is transferred to a cache and returned to SF.

For example, the Android system has a built-in screenshot command screencap. In response to the trigger operation, the application can send the screenshot command to the SF service module. After executing the command, the SF service module can process the HDR layer included in the fourth interface displayed on the virtual screen in combination with the maximum screen brightness, and perform dimming and other processing on the SDR layer, that is, darken the SDR layer; in this way, the processed HDR layer and the darkened SDR layer can be synthesized into a picture through GPU synthesis and saved in the Frame Buffer. The synthesized picture can be called the first screenshot, and the property corresponding to the first screenshot is the SDR layer.

Optionally, the method provided in the present application may further include the following step S28.

S28. The SF service module obtains the maximum screen brightness.

Exemplarily, the bottom layer will configure the screen parameters according to the screen capabilities, and will additionally add the configuration item of the maximum screen brightness. Then, through the path shown in FIG11 , it can be read when SF is started, and the maximum screen brightness can be saved in sOEMMaxLumiance. For example, if the maximum screen brightness supported by the screen is 1000nit, 1000nit can be saved in sOEMMaxLumiance for subsequent call when needed.

For example, as shown in FIG11 , SF can obtain the maximum screen brightness from HWC, which is configured by the driver; send the maximum screen brightness to mDisplayColorProfile; and then, obtain the maximum screen brightness through getDesiredMaxLuminance of mDisplayColorProfile.

When the identified scene type is a split-screen scene, the maximum screen brightness can be determined by obtaining the value of sOEMMaxLumiance. Then, the HDR layer is tone mapped in combination with the maximum screen brightness, so that the brightness of the corresponding area of the processed HDR layer is higher than that of the corresponding area of the SDR layer, thereby effectively bringing into play the capabilities of the screen.

S27. Obtain a first screenshot whose layer property is an SDR layer based on the processed HDR layer and the processed SDR layer, and do not perform darkening processing on the first screenshot.

It should be understood that in the native rotation scene control method of the Android system, only tone mapping is performed on the HDR layer without dimming, while the SDR layer is dimmed. Furthermore, since the first screenshot obtained is an ordinary SDR layer, it will be dimmed as a whole once. In this way, it is equivalent to dimming the brightness of the area corresponding to the SDR layer in the screenshot twice, and dimming the area corresponding to the HDR layer in the screenshot once. This will cause visual flickering to the human eye during rotation, which does not meet viewing requirements.

Therefore, in response to the above situation, in the brightness control method provided in the embodiment of the present application, for the rotating scene, in order to retain the original interface effect, the HDR layer is only subjected to tone mapping processing without dimming, and the SDR layer is subjected to dimming processing. Then, for the first screenshot obtained as an SDR layer, the overall dimming will no longer be performed. This is equivalent to retaining the effect of the brightness layer control and avoiding the phenomenon of rotating flashing black in the related art.

Optionally, when determining that the screen direction is rotated, the WMS may also send a second identifier for indicating the screen direction rotation to the SF service module. Then, the SF service module determines that the first screenshot is for a rotated scene based on the second identifier, and therefore does not darken the first screenshot.

For example, the WMS may send the second flag for indicating screen direction rotation to the SF service module through the newly added setExtensionLayerFlag interface. In other words, the SF service module may identify the second flag through the newly added setExtensionLayerFlag interface.

FIG12 shows a software system structure diagram provided by the present application with the setExtensionLayerFlag interface added. Based on this, after responding to the screenshot operation, the SF service module identifies the second identifier through the setExtensionLayerFlag interface.

It should be understood that the SF service module can determine that the current scene type is a rotation scene by identifying the second identifier, and thus based on the second identifier, the first screenshot whose layer property is an SDR layer stored in rotationLayer may not be darkened.

In the brightness control method provided in the above-mentioned embodiment of the present application, the target application first identifies the scene type and configures different first identifiers for different scene types, so that the WMS can determine the scene type according to the first identifier, and then the SF service module can select different brightness controls for different scene types.

For scenes with rotating scene types, in order to avoid the process of obtaining screenshots interfering with the normal display process on the display screen, in this application, in response to the screenshot operation, a virtual screen can be created separately, and the fourth interface displayed on the virtual screen is the same as the third interface displayed on the display screen. In this way, this application can process the fourth interface on the virtual screen to obtain a screenshot that meets the display effect. For example, the HDR layer included in the fourth interface on the virtual screen can be tone mapped based on the maximum screen brightness, and the SDR layer can be darkened. After the processed HDR layer and the SDR layer are synthesized, a first screenshot with an SDR layer property is obtained. Then, the first screenshot is no longer darkened.

It should be understood that in the native brightness control method of the Android system corresponding to the rotating scene, only tone mapping is performed on the HDR layer without dimming, and the SDR layer is dimmed. Furthermore, since the first screenshot obtained is an ordinary SDR layer, it will be dimmed once as a whole. In this way, it is equivalent to dimming the brightness of the area corresponding to the SDR layer in the screenshot twice, and dimming the area corresponding to the HDR layer in the screenshot once. This will cause visual flickering to the human eye during rotation, which does not meet the viewing requirements. However, this application no longer dims the first screenshot, which is equivalent to retaining the effect of the previous layer-level brightness control, avoiding the phenomenon of rotating flashing black in the related art.

The following is a detailed introduction to the screen projection scenario, screen recording scenario, and brightness control method in the screen projection and screen recording scenarios in conjunction with Figures 13 to 16.

Exemplarily, FIG13 shows a schematic diagram of a screen projection scenario. The screen projection scenario generally includes a first electronic device as a transmitting end and a second electronic device as a receiving end. The first electronic device can display its own display content (including pictures, videos, audio and waiting for image data to be projected) on the second electronic device through screen projection technology. For example, when the wireless screen projection technology is Miracast, using the mirror projection, the first electronic device (such as the mobile phone shown in FIG13) can take screenshots and record its own display content, and synchronously send the recorded screenshot data to the second electronic device (such as the TV 200 shown in FIG13), which is played by the second electronic device to complete the screen projection.

Exemplarily, FIG14 shows a schematic diagram of a related interface. As shown in FIG14, taking the first electronic device with a mobile phone as the transmitting end as an example, the interface 203 of the drop-down menu of the first electronic device may include a control 207 for controlling screen projection; subsequently, in response to the user's click operation on the space 207, the control 207 can be placed in a selected state, and accordingly, the screen projection function is turned on. As shown in (b) of FIG14, the setting interface 205 of the first electronic device may also include an icon for controlling screen projection and a corresponding switch control 208. For example, on the setting interface 205, when the user clicks on the switch control 208, the switch control 208 can be placed in an on state, and accordingly, the screen projection function is turned on. Subsequently, when the user selects the second electronic device at the receiving end, the image data to be projected on the first electronic device can be projected to the receiving end for display.

For example, a schematic diagram of a related interface is shown in Figure 15. As shown in Figure 15, the interface 203 of the drop-down menu of the electronic device may include a control 209 for recording the screen; in response to a user's click operation on the control 209, the control 209 may be in a selected state, and accordingly, the function of the screen recording product is turned on, and the content on the interface displayed by the electronic device is recorded.

Combined with the screen projection scenes shown in Figures 13 and 14 and the related display interfaces for turning on the screen projection function, and the related interfaces for turning on the screen recording function shown in Figure 15, Figure 16 shows a brightness control method that can be adapted to screen projection scenes and screen recording scenes.

As shown in FIG. 16 , the method 30 may include the following S31 to S35 , and steps S31 to S35 are described in detail below.

S31. The display screen displays the fifth interface.

S32: Receive a third operation from the user.

The fifth interface may be the same as or different from the third interface; the third operation may be the same as or different from the second operation. When the fifth interface is the same as the third interface, and the third operation is the same as the second operation, the description of S31 and S32 may refer to the description of S21 and S22.

S33. In response to the third operation, the target application is configured with a first identifier for indicating different scene types; the scene types include at least screenshot, rotation, split screen, screen projection, and screen recording.

Here, the target application is used to indicate the application program corresponding to the fifth interface.

As shown in FIG. 10 , the target application may modify the data structure included in the electronic device and add an int type flag in CaptrueArgs to mark different scene types, wherein the flag value may default to 0 to indicate that the scene type defaults to the screenshot scene.

Exemplarily, when the third operation is a screen projection operation, in response to the third operation, the target application may configure the flag value corresponding to mScreenShotFlag to 5 to indicate that the current scene is a screen projection scene.

Exemplarily, when the third operation is a screen recording operation, in response to the third operation, the target application may configure the flag value corresponding to mScreenShotFlag to 6 to indicate that the current scene is a screen recording scene.

S34. The WMS identifies the scene type by identifying the first identifier.

S35. When the identified scene type is screen projection or screen recording, the SF service module turns off the brightness control method and does not process all layers.

Optionally, the SF service module may further include a switch called Output between SF and Libsharder. When the scene type is identified as screen projection or screen recording, the SF service module may control the Output to be closed, thereby achieving the purpose of not performing any layer-level brightness adjustment on the layer.

It should be understood that in the screen projection and screen recording scenarios, since the device as the receiving end does not necessarily support HDR images and HDR videos, in order to avoid the large difference between the data sent and the source data due to the layer-level brightness control, in the brightness control method provided in the embodiment of the present application, when the scene type is identified as a screen projection scene or a screen recording scene in the SF service module, the layer-level brightness control path will be closed, that is, no brightness adjustment will be made to the HDR layer and SDR layer sent for display. In this way, it can be ensured that the source data of the screen projection or screen recording can also be displayed normally on other electronic devices.

The brightness control method, software system, and hardware system provided by the present application are described above in conjunction with Figures 1 to 16. The chip system of the electronic device to which the present application is applicable will be described below in conjunction with Figure 17. It should be understood that the chip system in the embodiment of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

Fig. 17 shows a schematic diagram of the structure of an electronic device provided by the present application. The dotted lines in Fig. 17 indicate that the unit or the module is optional, and the electronic device 400 can be used to implement the brightness control method described in the above method embodiment.

The electronic device 400 includes one or more processors 401, which can support the electronic device 400 to implement the method in the method embodiment. The processor 401 can be a general-purpose processor or a special-purpose processor. For example, the processor 401 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

The processor 401 may be used to control the electronic device 400, execute software programs, and process data of the software programs. The electronic device 400 may also include a communication unit 405 to implement input (reception) and output (transmission) of signals.

For example, the electronic device 400 may be a chip, the communication unit 405 may be an input and/or output circuit of the chip, or the communication unit 405 may be a communication interface of the chip, and the chip may be a component of the electronic device or other electronic devices.

For another example, the electronic device 400 may be an electronic device, and the communication unit 405 may be a transceiver of the electronic device, or the communication unit 405 may be a transceiver circuit of the electronic device.

The electronic device 400 may include one or more memories 402 on which a program 404 is stored. The program 404 can be executed by the processor 401 to generate instructions 403, so that the processor 401 executes the brightness control method described in the above method embodiment according to the instructions 403.

Optionally, data may be stored in the memory 402. Optionally, the processor 401 may read data stored in the memory 402. The data may be stored at the same storage address as the program 404, or may be stored at a different storage address than the program 404.

The processor 401 and the memory 402 may be provided separately or integrated together; for example, integrated on a system on chip (SOC) of the electronic device.

Exemplarily, the memory 402 may be used to store the program 404 related to the brightness control method provided in the embodiment of the present application, and the processor 401 may be used to call the program 404 related to the brightness control method stored in the memory 402 during the upgrade to execute the brightness control method in the embodiment of the present application. For example:

The display screen displays a third interface; a second operation is received; in response to the second operation, the WMS determines whether the screen direction of the display screen is rotated; when the screen direction is rotated, the WMS sends a second identifier for indicating the rotation of the screen direction to the SF service module; the SF service module combines the second identifier and does not darken the first screenshot whose layer property is an SDR layer.

The present application also provides a computer program product, which, when executed by the processor 401, implements the brightness control method described in any method embodiment of the present application.

The computer program product may be stored in the memory 402 , for example, a program 404 , which is converted into an executable target file that can be executed by the processor 401 after preprocessing, compiling, assembling, and linking.

The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the brightness control method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

Optionally, the computer-readable storage medium is, for example, a memory 402. The memory 402 may be a volatile memory or a non-volatile memory, or the memory 402 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and/or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship.

In short, the above is only a preferred embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application should be included in the protection scope of this application.

Claims (11)

1. A brightness control method, characterized in that it is applied to an electronic device including a display screen, a WMS and a SF service module, and the method comprises: The display screen displays a third interface; receiving a second operation for the display screen; In response to the second operation, the WMS determines whether the screen direction of the display screen is rotated; When it is determined that the screen direction is rotated, the SF service module does not perform darkening processing on the first screenshot of the SDR layer. 2. The brightness control method according to claim 1, wherein when it is determined that the screen direction is rotated, the SF service module does not perform darkening processing on the first screenshot of the SDR layer, comprising: When it is determined that the screen direction is rotated, the WMS sends a second identifier for indicating the screen direction rotation to the SF service module; After receiving the second identifier, the SF service module does not perform darkening processing on the first screenshot of the SDR layer. 3. The brightness control method according to claim 2, characterized in that the method further comprises: When it is determined that the screen direction is rotated, a first identifier is configured for the target application corresponding to the third interface, where the first identifier is used to indicate a rotation scene; Based on the first identifier, the WMS identifies that the scene type is a rotating scene; The WMS sends the first identifier to the SF service module through a first interface. 4. The brightness control method according to claim 3, characterized in that after receiving the second identifier, before the SF service module does not perform darkening processing on the first screenshot of the SDR layer, the method further comprises: In response to the second operation, a virtual screen is created, on which a fourth interface is displayed, the fourth interface being the same as the third interface; After receiving the first identifier, the SF service module performs tone mapping processing on the HDR layer included in the fourth interface and performs darkening processing on the included SDR layer according to the maximum screen brightness of the display screen; Based on the processed HDR layer and the processed SDR layer, the first screenshot is obtained. 5. The brightness control method according to claim 3 or 4, characterized in that the electronic device further comprises a rotation animation module, wherein in response to the second operation, the WMS determines whether the screen direction of the display screen is rotated, comprising: In response to the second operation, the target application determines whether to start the rotation animation module; If so, the WMS determines whether the screen direction of the display screen is rotated. 6. The brightness control method according to any one of claims 3 to 5, characterized in that the WMS sends a second identifier for indicating screen direction rotation to the SF service module, comprising: The WMS sends the second identifier to the SF service module through a second interface, where the second interface is different from the first interface; The first interface is the setDisplayBrightness interface of the WMS; the second interface is the setExtensionLayerFlag interface of the WMS. 7. The brightness control method according to any one of claims 3 to 6, wherein the electronic device further comprises a brightness module, and the method further comprises: In response to a first operation on a first application, the display screen displays a first interface, the first application includes the target application; when the first application is the target application, the first interface includes the third interface; When the first application recognizes that the first interface includes an HDR layer, sending a first instruction to the brightness module; After receiving the first instruction, the brightness module determines a target strategy and transmits the target strategy to the SF service module, where the target strategy includes brightness control parameters corresponding to the HDR layer and the SDR layer respectively; The SF service module adjusts the brightness of the display layer based on the target strategy; Sending the display layer after brightness adjustment to the display screen for display; Among them, the display layer includes an HDR layer and/or an SDR layer; when the display layer includes the HDR layer and the SDR layer, in the display layer after brightness adjustment, the brightness of the display layer that is the HDR layer is greater than the brightness of the display layer that is the SDR layer. 8. The brightness control method according to claim 7, wherein the electronic device further comprises a hardware synthesizer HWC, and the method further comprises: The SF service module obtains the maximum screen brightness of the display screen through the HWC; The SF service module adjusts the brightness of the display layer based on the target strategy, including: The SF service module performs tone mapping processing on the display layer which is the HDR layer based on the target strategy and the maximum screen brightness. 9. An electronic device, comprising a processor and a memory; The memory is used to store a computer program executable on the processor; The processor is used to execute the brightness control method according to any one of claims 1 to 8. 10. A chip system, comprising: a processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the brightness control method according to any one of claims 1 to 8. 11. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the brightness control method according to any one of claims 1 to 8.