WO2020034077A1

WO2020034077A1 - Display device and display data generating method

Info

Publication number: WO2020034077A1
Application number: PCT/CN2018/100329
Authority: WO
Inventors: Tsuruma TAKEYUKI; Takatori Kenichi; Akira Sakaigawa; Yasunori Kijima
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2018-08-14
Filing date: 2018-08-14
Publication date: 2020-02-20
Also published as: CN112514075B; CN112514075A

Abstract

The present disclosure provides a display device in which the region of location of electromagnetic-wave detection elements and the region of location of a display unit do not restrict each other, and a display data generating method for the display device. The display device includes: a display unit including a first area having a first pattern of pixels, and a second area having a second pattern of pixels having a pixel density lower than a pixel density of the first pattern of pixels; and a detection element disposed in the second area on a rear surface side of the display unit to detect an electromagnetic wave transmitting through the second area.

Description

DISPLAY DEVICE AND DISPLAY DATA GENERATING METHOD

Technical Field

The present disclosure relates to a display device and a display data generating method, and more particularly to a display device with an element, which detects an electromagnetic wave like light, such as a camera imaging device or an optical sensor, and a display data generating method.

Background Art

Portable terminals such as a smartphone are known as this type of display device. A portable terminal includes a display unit via which electromagnetic-wave detection elements make inputting of information by light besides display of information to a user and inputting of information by a user.

Summary

The present disclosure provides a display device in which the region of location of electromagnetic-wave detection elements and the region of location of a display unit do not restrict each other, and a display data generating method for the display device.

In a first aspect, there is provided a display device comprising:

a display unit including a first area with a first pattern of pixels, and a second area with a second pattern of pixels, the second pattern of pixels having a pixel density lower than a pixel density of the first pattern of pixels; and

a detection element disposed in the second area on a rear surface side of the display unit and configured to detect an electromagnetic wave transmitting through the second area.

According to a first possible implementation of the first aspect, the second area has a quarter of a pixel density of the first area.

In the first aspect or the first possible implementation of the first aspect, according to a second possible implementation of the first aspect, one pixel in the second area includes three subpixels respectively corresponding to three colors.

In the second possible implementation of the first aspect, according to a third possible implementation of the first aspect, one pixel area in the first area includes four pixels, and one pixel area in the second area is determined based on the four pixels.

In the third possible implementation of the first aspect, according to a fourth possible implementation of the first aspect, given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of the four pixels are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G'and B'of the three subpixels are determined based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α where α denotes an adjustment coefficient.

In the first aspect or the first to fourth possible implementations of the first aspect, according to a fifth possible implementation of the first aspect, each of the subpixels included in the first area and the second area is a rectangle or a circle.

In the first aspect or the first to fifth possible implementations of the first aspect, according to a sixth possible implementation of the first aspect, each of the subpixels included in one pixel area in the second area is arranged at an apex of a regular rectangle.

In the sixth possible implementation of the first aspect, according to a seventh possible implementation of the first aspect, a center of the regular rectangle coincides with a center of the one pixel area.

In the first aspect or the first to seventh possible implementations of the first aspect, according to an eighth possible implementation of the first aspect, emission areas of the first area and the second area are defined based on a life time and brightness of the display unit.

In the first aspect or the first to eighth possible implementations of the first aspect, according to a ninth possible implementation of the first aspect, the display unit further includes a third area between the first area and the second area, the third area with a third pattern of pixels having a pixel density lower than the pixel density of the first pattern of pixels and higher than the pixel density of the second pattern of pixels.

In the ninth possible implementation of the first aspect, according to a tenth possible implementation of the first aspect, the pixel density of the third area is half of the pixel density of the first area.

In the ninth or tenth possible implementation of the first aspect, according to an eleventh possible implementation of the first aspect, one pixel area in the first area includes four pixels, and one pixel area in the third area is determined based on the four pixels.

In the eleventh possible implementation of the first aspect, according to a twelfth possible implementation of the first aspect, given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of the four pixels are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G1', G2'and B'of four subpixels included in the one pixel area in the third area are determined based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.

In the first aspect or the first to twelfth possible implementations of the first aspect, according to a thirteenth possible implementation of the first aspect, a display mode for displaying an image in the second area includes a first mode and a second mode, the second mode allowing the image to be displayed in the second area at a brightness lower than a brightness in the first mode.

In the thirteenth possible implementation of the first aspect, according to a fourteenth possible implementation of the first aspect, an image of an icon is displayed in the second area.

In the first aspect or the first to fourteenth possible implementations of the first aspect, according to a fifteenth possible implementation of the first aspect, the display device is an organic EL (Electro Luminescence) display or a micro LED (Light Emitting Diode) display.

In the first aspect or the first to fifteenth possible implementations of the first aspect, according to a sixteenth possible implementation of the first aspect, the second area has an aperture at a position corresponding to at least one of the detection elements.

According to a second aspect, there is provided a display device comprising:

a display unit including a first area with pixels at a high density and a second area with pixels at a low density, the second area allowing therethrough of an electromagnetic wave signal input to an electromagnetic-wave detection element to be transmitted;

a first conversion unit configured to convert an input signal into a first signal to be given to the first area; and

a second conversion unit configured to convert the first signal into a second signal to be given to the second area, values of individual colors included in the second signal being set based on values of individual colors included in the first signal.

According to a first possible implementation of the second aspect, given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel area in the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G'and B'of three subpixels included in one pixel area in the second area are determined based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient.

In the second aspect or the first possible implementation of the second aspect, according to a second possible implementation of the second aspect, the display unit further includes a third area between the first area and the second area, the third area having a pixel density lower than a pixel density of the first area and higher than a pixel density of the second area.

In the second possible implementation of the second aspect, according to a third possible implementation of the second aspect, one pixel area of the third area is determined based on the four pixels.

In the third possible implementation of the second aspect, according to a fourth possible implementation of the second aspect, values R', G1', G2'and B'of four subpixels included in the one pixel area in the third area are determined based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.

According to a third aspect, there is provided a display-data generating method for a display device comprising a display unit including a first area with a first pattern of pixels, and a second area with a second pattern of pixels, the second pattern of pixels having a pixel density lower than a pixel density of the first pattern of pixels, and a detection element disposed in the second area on a rear surface side of the display unit and configured to detect an electromagnetic wave transmitting through the second area, the method comprising:

generating a signal of the first pattern of pixels for display in the first area, based on a plurality of input signals; and

generating a signal of the second pattern of pixels for display in the second area, based on the generated signal of the first pattern of pixels.

According to a first possible implementation of the third aspect, given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel of the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and the generating a signal of the second pattern of pixels determines values R', G'and B'of three subpixels included in one pixel area in the second area based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient.

In the third aspect or the first possible implementation of the third aspect, according to a second possible implementation of the third aspect, a display mode for displaying an image in the second area includes a first mode and a second mode, the second mode allows the image to be displayed at a brightness lower than a brightness in the first mode.

In the third aspect or the first or second possible implementation of the third aspect, according to a third possible implementation of the third aspect, the method further comprises:

generating a signal of a third pattern of pixels for display in a third area disposed between the first area and the second area of the display unit based on the signal of the first pattern of pixels, the third pattern of pixels having a pixel density lower than the pixel density of the first pattern of pixels and higher than the pixel density of the second pattern of pixels.

In the third possible implementation of the third aspect, according to a fourth possible implementation of the third aspect, given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel of the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and the generating a signal of the third pattern of pixels determines values R', G'1, G'2 and B'of four subpixels included in one pixel area in the third area based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.

According to a fourth aspect, there is provided a computer program for causing a computer to execute the method according to any one of the third aspect and the first to fourth possible implementations of the third aspect.

According to a fifth aspect, there is provided a computer readable storage medium storing a computer program for causing a computer to execute the method according to any one of the third aspect and the first to fourth possible implementations of the third aspect.

According to the foregoing configurations, a pixel in a low-resolution area includes subpixels similar to those of an input signal, and has a resolution lower than the resolution of the input signal. Therefore, the display area of the display unit can be extended to above the detection elements.

Brief Description of Drawings

For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings:

[Fig. 1A] Fig. 1A is a front view of a part of a smartphone which is a display device according to an embodiment; [Fig. 1B] Fig. 1B is a cross-sectional view of a low-resolution area of the smartphone shown in Fig. 1A;

[Fig. 1C] Fig. 1C is a cross-sectional view of a high-resolution area of the smartphone shown in Fig. 1A;

[Fig. 2] Fig. 2 is a diagram showing the functional configuration of a display device according to an embodiment; [Fig. 3A] Fig. 3A is a diagram showing the structure of pixels of a display unit according to an embodiment;

[Fig. 3B] Fig. 3B is a diagram showing the structure of pixels of a display unit according to an embodiment;

[Fig. 4A] Fig. 4A is a diagram showing the arrangement of subpixels included in a pixel in the low-resolution area;

[Fig. 4B] Fig. 4B is a diagram showing the arrangement of subpixels included in a pixel in the low-resolution area;

[Fig. 5A] Fig. 5A is a flowchart showing a process of converting an input signal into an output signal;

[Fig. 5B] Fig. 5B is a diagram for describing the process of converting an input signal into an output signal;

[Fig. 6] Fig. 6 is a diagram showing a display unit of a conventional smartphone;

[Fig. 7] Fig. 7 is a front view of a part of a smartphone which is a display device according to an embodiment;

[Fig. 8] Fig. 8 is a diagram showing the functional configuration of a display device according to an embodiment;

[Fig. 9A] Fig. 9A is a diagram showing the structure of pixels of a display unit according to an embodiment;

[Fig. 9B] Fig. 9B is a diagram showing the structure of pixels of a display unit according to an embodiment;

[Fig. 10A] Fig. 10A is a flowchart showing a process of converting an input signal into an output signal;

[Fig. 10B] Fig. 10B is a diagram for describing the process of converting an input signal into an output signal;

[Fig. 11] Fig. 11 is a diagram for describing advantages of an embodiment;

[Fig. 12] Fig. 12 is a diagram showing examples of display images in high-resolution area, a low-resolution area, and a comparative example;

[Fig. 13] Fig. 13 is a diagram showing examples of a display image in a low-resolution area; and

[Fig. 14] Fig. 14 is a front view of a part of a smartphone which is a display device according to an embodiment.

Description of Embodiments

The following will describe, in detail, the embodiments of the present disclosure referring to the accompanying drawings.

(First Embodiment)

Fig. 1A partially shows a front view of a smartphone which is a display device according to a first embodiment. The smartphone 1 includes a display unit 100 constituted by an organic EL (Electro Luminescence) display having organic light emitting diodes (hereinafter, referred to as OLEDs) as emission elements. In this embodiment, the display has an array of red (R) , green (G) and blue (B) display elements. The display unit 100 includes a low-resolution area 101 and a high-resolution area 102. In this embodiment, as will be described later, the low-resolution area 101 and the high-resolution area 102 serve as a single continual display area. For example, there are two modes: a mode in which the low-resolution area 101 and the high-resolution area 102 together display a single image, and a mode in which the low-resolution area 101 and the high-resolution area 102 respectively display different images. In the latter mode, the low-resolution area 101 displays an image of, for example, a highly-visible icon, and the high-resolution area 102 displays a high-definition image.

It should be noted that the application of the present disclosure is not limited to a display unit having OLEDs as display elements. For example, it would be apparent from the following description that the present disclosure is also applicable to a display unit that uses micro LEDs (Light Emitting Diode) or liquid crystal elements for the display elements.

Fig. 1B is a cross-sectional view of the low-resolution area 101. The low-resolution area 101 is configured in such a way that a display drive layer 105 for causing emission of display elements made of OLEDs to display an image, and a deflecting plate 104 for controlling light transmitting through the display unit 100 are laminated on a glass substrate 107.

In the low-resolution area 101, a detection layer 108 is provided under the glass substrate 107. Referring to Fig. 1A, the detection layer 108 includes an infrared sensor 108A behind an icon for vibrating phone. The infrared sensor 108A is an example of an electromagnetic-wave detection element. The electromagnetic-wave detection element may include an element such as an infrared camera, an optical sensor, an infrared sensor, an infrared projector for face recognition and an infrared communication interface. Any electromagnetic-wave detection element is configured to detect an electromagnetic wave radiated from or reflected at a detection target positioned on an upper side in Fig. 1B. That is, the glass substrate 107, the display drive layer 105 and the deflecting plate 104 are configured to allow transmission of an electromagnetic wave input to an electromagnetic-wave detection element. OLEDs, a molybdenum layer, an electrode and the like (portions surrounded by broken lines in Fig. 1B) in the display drive layer 105 have a property to shield the electromagnetic wave as will be described below. In this embodiment of the present disclosure, as will be described later, the density of the OLEDs (emission elements) in the low-resolution area 101 is set lower to reduce the amount of light to be shielded. Accordingly, it is possible to provide such a display device that even if the electromagnetic-wave detection elements are disposed below the display drive layer 105 (on the rear surface side as opposed to the front surface side which is a display surface of the display unit of the smartphone) , the performances of the individual electromagnetic-wave detection elements can be demonstrated and the region of location of electromagnetic-wave detection elements and the region of location of the display unit do not restrict each other.

Referring to Fig. 1B again, the display drive layer 105 is configured in such a way as to have, on a first SiO ₂ layer 1001, a lamination of a second SiO ₂ layer 1002, a first intermediate dielectric layer 1003, a second intermediate dielectric layer 1007, a planarizing film 1010 made of polyimide, a first mixed film 1013 made of SiN _x/SiO _x, a sealing thin film 1019 for inhibiting injection of water and oxygen, and a second mixed film 1020 made of SiN _x/SiO _x.

The display drive layer 105 includes inside a thin film transistor (TFT) 1011, a capacitive element 1006 including

molybdenum layers

1023 and 1024, and an organic light emitting diode (OLED) 1022, a combination of which forms a circuit for emission from a single-color emission element.

The TFT 1011 includes a source electrode/

drain electrode

1008, 1009 connected to a low-temperature polysilicon layer 1004 inside the second SiO ₂ layer 1002, and a gate electrode 1005 provided in the first intermediate dielectric layer 1003.

The OLED 1022 includes an emission organic layer 1017 between

pixel isolation layers

1014 and 1015 provided inside the first mixed film 1013. The emission organic layer 1017 is connected to the TFT 1011 via a first electrode 1012 made of ITO/Ag/ITO. A second electrode 1018 made of MgAg is provided on the emission organic layer 1017. A single emission organic layer 1017 constitutes a single display element, and the size of a subpixel is defined by the size of the emission organic layer 1017. In this embodiment, a single display element is formed of, but not limited to, any one of R, G and B subpixels.

The capacitive element 1006 is electrically connected to the TFT 1011. The TFT 1011 serves as a switch for applying a voltage to the liquid crystal of each subpixel, and when pixel data in a subpixel is held in the capacitive element 1006, a current flows through the first electrode 1012 by means of the TFT 1011 according to the voltage of the pixel data, causing emission from the OLED 1022.

In the above configuration, the

molybdenum layers

1023 and 1024, the gate electrode 1005, and the source electrode/

drain electrode

1008, 1009 have a light shielding property. Other areas are formed of materials which pass an electromagnetic wave like light therethrough.

Fig. 1C is a cross-sectional view of the high-resolution area 102. The high-resolution area 102, like the low-resolution area 101, has a display drive layer 106 and the deflecting plate 104 provided on the glass substrate 107. Further, in the high-resolution area 102, unlike in the low-resolution area 101, a touch sensor 103 for detecting an instruction to the smartphone 1 given by a human finger or the like is laminated on the deflecting plate 104. The display drive layer 106 also includes another set of an OLED 1022, a TFT 1011 and a capacitive element 1006.

Light shielding areas (portions surrounded by broken lines in Fig. 1C) similar to those of the low-resolution area 101 are present in the high-resolution area 102.

Next, referring to Fig. 2, the functional configuration of a display device according to this embodiment will be described. A display device 200 includes a signal processing unit 202, a display control unit 204, and the display unit 100. The signal processing unit 202 processes an input signal input from a control device to generate an output signal. The input signal has a combination of R, G and B values each ranging, for example, between 0 and 255. The signal processing unit 202 converts this input signal into an output signal (display data) to be reproduced on the display unit 100, and sends the output signal to the display control unit 204. The input signal is a combination of the R, G and B values corresponding to pixels constituting an image to be displayed, and the output signal is a combination of the values of subpixels corresponding to each display element for each of the low-resolution area 101 and the high-resolution area 102 as will be described below.

The signal processing unit 202 includes an SPR-Aunit 214 that performs first subpixel rendering (SPR) on an input value carried by the input signal to generate a signal of a first pattern of pixels to be displayed in the high-resolution area 102, and an SPR-B unit 216 that further performs second subpixel rendering on the signal of the first pattern of pixels to generate a signal of a second pattern of pixels to be displayed in the low-resolution area 101.

The display control unit 204 controls emission of display elements corresponding to the subpixels of the display unit 100 based on the pixel-by-pixel output signals from the signal processing unit 202. The display control unit 204 performs control to display an image in the high-resolution area 102 based on the signal of the first pattern of pixels, and displays an image in the low-resolution area 101 based on the signal of the second pattern of pixels.

In an example, the

units

202 and 204 may be implemented via processing unit (s) . The processing unit (s) may include application-specific integrated circuit (ASIC) logic, graphics processor (s) , general purpose processor (s) , or the like. In an example, the

units

202 and 204 may be implemented via hardware, imaging dedicated hardware, or the like.

Next, referring to Figs. 3A and 3B, the structure of pixels of display data to be displayed on the display unit according to this embodiment will be described. Fig. 3A shows an example of a pattern of pixels when subpixels corresponding to display elements are rectangles, and Fig. 3B shows an example of a pattern of pixels when subpixels corresponding to display elements are circles. Those diagrams differ from each other merely in the shapes and sizes of subpixels, and are identical in what is shown, so that the following description will be given with reference only to Fig. 3A.

In Fig. 3A, a pixel area 301 indicates a pixel area comprised of a single pixel, and a pixel area 302 indicates a pixel area comprised of four pixels. In other words, in this embodiment, for a pixel which is one unit of display data, the low-resolution area 101 has a quarter of the resolution of the high-resolution area 102. That is, one pixel of display data in the low-resolution area 101 corresponds to four pixels of display data in the high-resolution area 102.

The pixel area (pixel) 301 of the low-resolution area 101 is expressed by display data of an R subpixel, a G subpixel and a B subpixel. Hereinafter, the subpixels that cause emission from R, G and B display elements are respectively referred to as R subpixel, G subpixel and B subpixel. Each of four

pixels

302A, 302B, 302C and 302D is expressed by display data of two subpixels. Specifically, the pixel 302A includes a G subpixel 3021 and an R subpixel 3022, the pixel 302B includes a G subpixel 3023 and a B subpixel 3024, the pixel 302C includes a G subpixel 3025 and a B subpixel 3026, and the pixel 302D includes a G subpixel 3027 and an R subpixel 3028.

Because the pixel area (emission area) is designed in consideration of the trade-off between the life time and brightness of the display unit, the R subpixel, the G subpixel and the B subpixel differ from one another in size. Since a blue emission element generally has faster deterioration and shorter life time, the device is designed in such a way as to elongate the life time of the emission element by increasing the area of the emission area of the blue emission element to reduce the current density.

Further, the subpixels of individual colors included in the pixels 302A to 302D have larger emission areas than the subpixels of the corresponding colors of the pixel area 301.

With a single pixel area 301 in comparison with a single pixel area 302 of the same area as the pixel area 301, the quantity (density) of subpixels, that is, display elements included in the pixel area 301 in the low-resolution area 101 is smaller than that of the pixel area 302 in the high-resolution area 102. Accordingly, at the time a detection element located on the rear surface side of the display unit detects a target located on the front surface side of the display unit, the ratio of the light (electromagnetic wave) which is shielded by the display element can be reduced, thus achieving the transmittance of the light needed for detection.

Next, referring to Figs. 4A and 4B, the arrangement of subpixels included in a single pixel in the low-resolution area will be described. Fig. 4A shows the pixel area 301 shown in Fig. 3A. A rectangular R subpixel 3011, a rectangular G subpixel 3012, and a rectangular B subpixel 3013 are arranged in such a way that the center of the rectangular pixel indicated by a mark "×" in the left-hand figure substantially coincides with the center of a regular triangle formed by connecting the centers of the subpixels and indicated by a mark "×" in the right-hand figure. The distance between the individual subpixels may be set less than the distance from each subpixel to a nearest one of the subpixels included in an adjoining pixel. Fig. 4B shows the area shown in Fig. 3B. A circular R subpixel, a circular G subpixel, and a circular B subpixel are arranged in such a way that the center of the rectangular pixel indicated by a mark "×" in the left-hand figure substantially coincides with the center of a regular triangle formed by connecting the centers of the subpixels and indicated by a mark "×" in the right-hand figure.

Arranging subpixels symmetrically from the center of a pixel in the above manner enables display control in such a way that even if the orientation of a cellular phone, for example, is changed, the quality of an image does not vary.

Next, referring to Figs. 5A and 5B, the process of converting an input signal into an output signal as mentioned above with reference to Fig. 2 will be described.

Fig. 5A is a flowchart showing the procedures of the process of converting an input signal into an output signal. In S502, a plurality of input signals are converted to generate signals of individual pixels in the high-resolution area. Specifically, subpixel rendering (SPR) is performed on an input signal to generate a signal comprised of two subpixels per pixel (signal of a first pattern of pixels) (SPR-A) . Anti-aliasing of an OLED serving as a display element can be taken into consideration by the generation of a pixel signal comprised of two subpixels in this way.

Next, in S504, the generated signal of each pattern of pixels in the high-resolution area is converted to generate a signal of a pattern of pixels in the low-resolution area (signal of a second pattern of pixels) . Specifically, subpixel rendering (SPR) is performed to generate a signal comprised of three subpixels per pixel (SPR-B) .

In the embodiment shown in Fig. 5B, there are four input signals (Input) for emission of four emission elements of pixels. The input signals have R, G and B values as display data, and are expressed by R1G1B1, R2G2B2, R3G3B3, and R4G4B4, respectively, in the inputting order.

The following will describe in detail the SPR-Awhich is the first conversion process and the SPR-B which is the second conversion process.

SPR-A

First, single subpixels B1, R2, R3, B4, ... are respectively eliminated from pixels R1G1B1, R2G2B2, R3G3B3, R4G4B4, ... within the input signals, thereby generating pairs of two subpixels G1R1, G2B2, G3B3, G4R4, ....

Next, the values of the subpixels in each subpixel pair are obtained. This process can be carried out by, for example, an SPR filter (3×3 filter) . In the case of the 3×3 filter, the value of each subpixel is determined using data around 3×3 display data with the target subpixel being centered.

The reason why the G subpixel is not eliminated is that the G subpixel strongly affects the brightness. As a result, the ratio of the R, G and B values (magnitude of the current) becomes 1: 2: 1.

SPR-B

Next, a signal of a pattern of pixels which is generated for the high-resolution area is converted to generate a signal of a pattern of pixels which is generated for the low-resolution area. This process is carried out by generating the signal of the pattern of pixels in such a way that a single R subpixel, a single G subpixel, and a single B subpixel are included within a single pixel. The high-resolution area includes four sets of pixels in a single pixel area, whereas the low-resolution area includes one set of pixels in a single pixel area, so that the resolution of the low-resolution area is quarter of the resolution of the high-resolution area.

Specifically, the values R', G'and B'of the three colors of subpixels (magnitude of the current) in one pixel area in the low-resolution area are determined using the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient. In the above formulas, the value of G'is set to an average of the values of G1 to G4, the value of R'is set to a half of an average of the values of R1 and R4, and the value of B'is set equal to the value of R', so that the ratio of the values of R, G and B is 1: 2: 1. As a result, the ratio of the values of R', G'and B'becomes equal to the ratio for the high-resolution area.

Further, as for the input signal corresponding to one pixel in the high-resolution area, a single R subpixel, a single G subpixel, and a single B subpixel are included within a single pixel in the low-resolution area.

According to the above-described embodiment, electromagnetic-wave detection elements for performing detection via the low-resolution area are provided on the rear surface side of the low-resolution area, thus making it possible to widen the display area of the display unit. For example, the conventional smartphone has a notch 601 provided in the display unit, and the electromagnetic-wave detection elements are implemented in the notch portion as shown in Fig. 6. According to this embodiment, implementing sensors directly under the display can achieve a full-view display.

Moreover, a signal of a pattern of pixels in the low-resolution area can be generated for each pixel area without complicated calculation using information on pixels across a boundary of different resolution areas.

Although the display device is configured as a smartphone according to this embodiment, the display device may be configured as an information processing device, such as a cellular phone or a tablet terminal, in some embodiments. (Second Embodiment)

Fig. 7 is partially shows a front view of a smartphone which is a display device according to a second embodiment. The display unit 100 of the smartphone 1 includes a compensation area 110 between the low-resolution area 101 and the high-resolution area 102.

Next, the functional configuration of a display device 200 according to this embodiment will be described with reference to Fig. 8. Since those components in the display device 200 whose reference numerals are similar to those given in Fig. 2 have similar functional structures, their descriptions will be omitted. The signal processing unit 202 includes, in addition to the SPR-Aunit 214 and the SPR-B unit 216, an SPR-C unit 802 for further performing third subpixel rendering on the signal of the first pattern of pixels to generate a signal of a third pattern of pixels to be displayed in the compensation area 110.

Next, referring to Figs. 9A and 9B, the structure of pixels of display data shown on the display unit according to this embodiment will be described. Fig. 9A shows an example of a pattern of pixels when subpixels corresponding to the display elements are rectangles, and Fig. 9B shows an example of a pattern of pixels when subpixels corresponding to the display elements are circles. Those diagrams differ from each other merely in the shapes and sizes of subpixels, and are identical in what is shown, so that the following description will be given with reference only to Fig. 9A.

Fig. 9A shows an example of the structure of a pixel including rectangular subpixels. The low-resolution area 101, the compensation area 110 and the high-resolution area 102 are continual, and have different patterns of pixels. In the compensation area 110, a pixel area 901 indicates a pixel area comprised of two pixels. In this embodiment, for a pixel which is one unit of display data, the compensation area 110 has a half of the resolution of the high-resolution area 102. That is, two pixels of display data in the compensation area 110 correspond to four pixels of display data in the high-resolution area 102.

In the compensation area 110, the pixel area 901 is expressed by display data including a single R subpixel, two G subpixels, and a single B subpixel. Two

pixels

901A and 901B in the pixel area 901 in the compensation area 110 are each expressed by display data of two subpixels. Specifically, the pixel 901A includes a G subpixel 9011 and a B subpixel 9012, and the pixel 901B includes a G subpixel 9013 and an R subpixel 9014. In each pixel, the size of the subpixel area varies depending on the life time of each emission element. Since a blue emission element generally has faster deterioration and shorter life time, the device is designed in such a way as to elongate the life time of the emission element by increasing the area of the emission area of the blue emission element to reduce the current density.

In addition, the brightness of subpixels of the individual colors included in the pixel area 901 are adjusted gradually toward the low-resolution area from the high-resolution area. It should be noted however that as the brightness increases, the current density increases, which would shorten the life time of the emission element. Accordingly, the life times of the individual emission elements can be set to about the same life time by tuning up the current densities for the individual colors by adjusting the areas of the emission elements. In other words, the pixels are varied in such a way as to have larger emission areas toward pixels adjacent to the high-resolution area 102 from pixels adjacent to the low-resolution area 101 according to the locations of the pixels.

Next, referring to Figs. 10A and 10B, the process to be performed by the display device shown in Fig. 8 to convert an input signal to an output signal for displaying an image in the compensation area.

Fig. 10A is a flowchart showing the procedures of the process of converting an input signal into an output signal.

First, display data R1G1B1, R2G2B2, R3G3B3, R4G4B4, ... are input in the order of pixels indicated by the numerals "1" , "2" , "3" , "4" and so forth. The foregoing SPR-Aand SPR-B are performed on those input signals (S502, S504) . In this embodiment, subpixel rendering SPR-C is performed based on the data generated through the SPR-Ain parallel with the SPR-B. That is, in S1006, a signal of a third pattern of pixels to be displayed in the compensation area, which has a pixel density lower than the pixel density of the first pattern of pixels and higher than the pixel density of the second pattern of pixels, is generated.

The following will describe the details of the SPR-C which is the third conversion process with reference to Fig. 10B.

SPR-C

Next, a signal of a pattern of pixels for the compensation area is generated based on the signal of the pattern of pixels generated for the high-resolution area, that is, four pixels G1R1, G2B2, G3B3, and G4R4. This process is carried out by generating the signal of the pattern of pixels in such a way that a single R subpixel, two G subpixels, and a single B subpixel are included within a single pixel area. The high-resolution area includes four sets of pixels in a single pixel area, whereas the compensation area includes two sets of pixels in a single pixel area, so that the resolution of the compensation area is half of the resolution of the high-resolution area.

Given that the values of the R subpixel, the two G subpixels and the B subpixel in the compensation area are expressed by R', G'1 and G'2, and B', those values are determined using the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient. As a result, the quantity of pixels included in a single pixel area becomes half of the quantity of pixels in the high-resolution area. Further, the composition ratio of R, G and B becomes 1: 2: 1 as for the high-resolution area.

Through the above-described processing, an image can be changed seamlessly between the low-resolution area and the high-resolution area. In addition, the signal of the pattern of pixels in the compensation area can be generated pixel by pixel without considering adjoining pixels.

Figs. 11 and 12 show the advantages of this embodiment. The example shown in Fig. 11 shows the result of comparison of displayed images when circular subpixels are used. The row indicated by "PATTERN" shows patterns of pixels in the high-resolution area (Hi-Reso) , the low-resolution area (Low-Reso1) and a comparative example (Low-Reso2) in order from the left. "QUALITY" shows the results of actually observing images the low-resolution area, "○" indicating a good quality while "x" indicates a poor quality. Fig. 12 shows displayed images in (1) the high-resolution area, (2) the low-resolution area and (3) the comparative example of the low-resolution area in order from above.

The comparative example includes two subpixels in a single pixel. The pattern of pixels in the comparative example corresponds to an example where eight subpixels G1R1, G2B2, G3B3, G4B4 included in a single pixel in the high-resolution area are simply reduced to a quarter. In this case, there are two types of subpixel pairs, GB and GR, included in a single pixel.

In the case of (3) the comparative example of the low-resolution area, when a pattern of pixels in the low-resolution area is generated from four RGB input signals, a single R subpixel or a single B subpixel is present in two pixels. As a result, the R subpixels and B subpixels are reduced in quantity to 1/8 of those subpixels in the original signal, so that the resolution becomes considerably low. On the other hand, each of R, G and B subpixels are included in in a single pixel in the low-resolution area according to this embodiment. When a single pixel in the low-resolution area is generated from four RGB input signals, the quantities of R subpixels and B subpixels become a quarter of the quantities of R subpixels and B subpixels in the original data. According to the generation of the compensation area according to this embodiment, therefore, the reduction in quantity of R subpixels and B subpixels is suppressed as compared with the comparative example, from which the quality is considered to have been improved.

(Third Embodiment)

In other embodiments, the brightness at the time of displaying an image in the low-resolution area may be controlled. Perception of the contrast changes according to the ambient environment (Bartleson Breneman effect) . In this embodiment, the mode for controlling the brightness of the low-resolution area can be selected using the difference in vision. A boost mode and a normal mode may be provided as this mode.

In boost mode, the brightness of the low-resolution area may be set to 100%of the brightness of the high-resolution area. In normal mode, the brightness of the low-resolution area may be set to 60 to 70%of the brightness of the high-resolution area.

The current that is applied in boost mode should be greater than that in normal mode, and the value of the current may be set according to the sizes of the individual pixels.

In consideration of the Bartleson Breneman effect, it can be said that even if the brightness is reduced by 20 to 40%under the of high-contrast condition, the reduction may not be perceptive to humans. When the background image is dark and a target image is bright, therefore, the brightness can be reduced.

Fig. 13 shows a display example of images in the low-resolution area according to this embodiment, in which example the upper image has a brightness of 100%and the lower image has a brightness of 60%. As shown in the diagram, it is apparent that images of icons displayed in the black background has a good visibility even when the brightness is reduced.

Therefore, the use of the low-resolution area for displaying the image of an icon makes a change in displayed image less recognizable while reducing the consumption power in normal mode.

(Another Embodiment)

Fig. 14 shows the external appearance of a smartphone which is a display device according to another embodiment. As shown in Fig. 14, a low-resolution area of a display unit may be configured not to cover some sensors. The display unit according to this embodiment has an aperture 108C provided at a position corresponding to a lens of a camera 108B. In this way, the area to be covered with the low-resolution area may be adaptively designed according to the sensitivities of detection elements and requirements.

In an implementation process, each step of the foregoing method may be completed by using an integrated logic circuit of hardware in the processor of a display device or an instruction in a form of a computer program. The steps of the method disclosed with reference to the embodiments of the present invention may be directly performed by a hardware processor, or may be performed by using a combination of hardware in the processor and a program module. The program module may be located in a mature computer-readable storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory. The processor reads information in the memory and completes the steps in the foregoing method in combination with the hardware of the processor.

The foregoing descriptions are merely specific implementations, but are not intended to limit the protection scope of the present invention. Any variation or replacement readily figured out by a person skilled in the art within the disclosed technical scope shall fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

A display device comprising:

a display unit including a first area with a first pattern of pixels, and a second area with a second pattern of pixels, the second pattern of pixels having a pixel density lower than a pixel density of the first pattern of pixels; and

a detection element disposed in the second area on a rear surface side of the display unit and configured to detect an electromagnetic wave transmitting through the second area.
The display device according to claim 1, wherein the second area has a quarter of a pixel density of the first area.
The display device according to claim 1 or 2, wherein one pixel in the second area includes three subpixels respectively corresponding to three colors.
The display device according to claim 3, wherein one pixel area in the first area includes four pixels, and one pixel area in the second area is determined based on the four pixels.
The display device according to claim 4, wherein given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of the four pixels are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G' and B' of the three subpixels are determined based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient.
The display device according to any one of claims 1 to 5, wherein each of the subpixels included in the first area and the second area is a rectangle or a circle.
The display device according to any one of claims 1 to 6, wherein each of the subpixels included in one pixel area in the second area is arranged at an apex of a regular rectangle.
The display device according to claim 7, wherein a center of the regular rectangle coincides with a center of the one pixel area.
The display device according to any one of claims 1 to 8, wherein emission areas of the first area and the second area are defined based on a life time and brightness of the display unit.
The display device according to any one of claims 1 to 9, wherein the display unit further includes a third area between the first area and the second area, the third area having a third pattern of pixels with a pixel density lower than the pixel density of the first pattern of pixels and higher than the pixel density of the second pattern of pixels.
The display device according to claim 10, wherein the pixel density of the third area is half of the pixel density of the first area.
The display device according to claim 10 or 11, wherein one pixel area in the first area includes four pixels, and one pixel area in the third area is determined based on the four pixels.
The display device according to claim 12, wherein given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of the four pixels are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G1', G2' and B' of four subpixels included in the one pixel area in the third area are determined based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.
The display device according to any one of claims 1 to 13, wherein a display mode for displaying an image in the second area includes a first mode and a second mode, the second mode allowing the image to be displayed in the second area at a brightness lower than a brightness in the first mode.
The display device according to claims 14, wherein an image of an icon is displayed in the second area.
The display device according to any one of claims 1 to 15, wherein the display device is an organic EL (Electro Luminescence) display or a micro LED (Light Emitting Diode) display.
The display device according to any one of claims 1 to 16, wherein the second area has an aperture at a position corresponding to at least one of the detection elements.
A display device comprising:

a display unit including a first area with pixels at a high density and a second area with pixels at a low density, the second area allowing an electromagnetic wave signal input to an electromagnetic-wave detection element to be transmitted;

a first conversion unit configured to convert an input signal into a first signal to be given to the first area; and

a second conversion unit configured to convert the first signal into a second signal to be given to the second area, values of individual colors included in the second signal being set based on values of individual colors included in the first signal.
The display device according to claim 18, wherein given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel area in the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and values R', G' and B' of three subpixels included in one pixel area in the second area are determined based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient.
The display device according to claim 19, wherein the display unit further includes a third area between the first area and the second area, the third area having a pixel density lower than a pixel density of the first area and higher than a pixel density of the second area.
The display device according to claim 20, wherein one pixel area of the third area is determined based on the four pixels.
The display device according to claim 21, wherein values R', G1', G2' and B' of four subpixels included in the one pixel area in the third area are determined based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.
A display-data generating method for a display device comprising a display unit including a first area with a first pattern of pixels, and a second area with a second pattern of pixels, the second pattern of pixels having a pixel density lower than a pixel density of the first pattern of pixels, and a detection element disposed in the second area on a rear surface side of the display unit and configured to detect an electromagnetic wave transmitting through the second area, the method comprising:

generating a signal of the first pattern of pixels for display in the first area, based on a plurality of input signals; and

generating a signal of the second pattern of pixels for display in the second area, based on the generated signal of the first pattern of pixels.
The method according to claim 23, wherein given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel of the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and the generating a signal of the second pattern of pixels determines values R', G' and B' of three subpixels included in one pixel area in the second area based on the following formulas:

R'= { (R1+R4) /2/2} ×α

G'= { (G1+G2+G3+G4) /4} ×α

B'= { (B2+B3) /2/2} ×α

where α denotes an adjustment coefficient.
The method according to claim 23 or 24, wherein a display mode for displaying an image in the second area includes a first mode and a second mode, the second mode allowing the image to be displayed at a brightness lower than a brightness in the first mode.
The method according to any one of claims 23 to 25, further comprising:

generating a signal of a third pattern of pixels for display in a third area disposed between the first area and the second area of the display unit based on the signal of the first pattern of pixels, the third pattern of pixels having a pixel density lower than the pixel density of the first pattern of pixels and higher than the pixel density of the second pattern of pixels.
The method according to claim 26, wherein given that four input signals are R1G1B1, R2G2B2, R3G3B3 and R4G4B4 expressed by three colors of R, G and B, values of four pixels included in one pixel of the first area are expressed by G1R1, G2B2, G3B3 and G4R4, and the generating a signal of the third pattern of pixels determines values R', G'1, G'2 and B' of four subpixels included in one pixel area in the third area based on the following formulas:

R'= (R1+R4) /2×β

G1'= {G1/2+ (G2+G3) /4} ×β

G2'= {G4/2+ (G2+G3) /4} ×β

B'= (B2+B3) /2×β

where β denotes an adjustment coefficient.
A computer program for causing a computer to execute the method according to any one of claims 23 to 27.
A computer readable storage medium storing a computer program for causing a computer to execute the method according to any one of claims 23 to 27.