KR102694995B1 - Electronic device having display and sensor and method for operating thereof - Google Patents

Abstract

The present invention relates to an electronic device including a display and a sensor and a method of operating the same.
The invention relates to an electronic device, comprising: a housing; a display disposed on one surface of the housing; a sensor mounted underneath the display inside the housing and detecting light received through the display; and a processor electrically connected to the display and the sensor, wherein the processor may be set to determine illuminance outside the housing by using sensing data acquired by the sensor while the display displays an image and related information between the display and the sensor.

Description

{ELECTRONIC DEVICE HAVING DISPLAY AND SENSOR AND METHOD FOR OPERATING THEREOF}

The present invention relates to an electronic device including a display and a sensor and a method of operating the same.

Electronic devices can perform a variety of functions in a complex manner, and portable terminals such as smartphones, for example, are evolving to provide users with greater convenience while implementing advanced performance.

As the functions of recent portable terminals have become more diverse, various sensors can be equipped in portable terminals to implement various functions. For example, the brightness of the surroundings can be measured through a light sensor equipped on the front of the portable terminal, and the brightness of the display can be adjusted using the measured value to improve the user's visibility and promote efficient power usage of the portable terminal.

Typically, the above light sensor is installed in a manner that it is built into the electronic device, and may be mounted at a predetermined location on the front panel, for example, around the display, so as to be able to receive ambient light.

However, the light sensor may be mounted around the periphery of the display, for example, in a non-screen area around the screen area where the screen is displayed, which may cause restrictions in the design of the electronic device, and may require consideration of the location of the light sensor when designing components mounted on the electronic device. For example, since the light sensor is generally mounted adjacent to a speaker section, a front camera, a display, etc., the mounting location with respect to these must be considered, and interference with other parts must be taken into consideration, which may cause design restrictions.

As a solution to this problem, mounting the light sensor, which was previously mounted on the outer perimeter of the screen area of the display, on the screen area of the display is being considered. However, in this case, it may be difficult for the light sensor to accurately measure the surrounding brightness due to the influence of the display's light emission.

Meanwhile, the display may experience deterioration of R/G/B sub-pixels after the expected lifespan of the product, and a method needs to be sought to compensate for the deterioration of the display's luminance characteristics caused by such deterioration.

Various embodiments of the present invention can provide an electronic device including a display and a sensor, and an operating method thereof, which enables an illuminance sensor, which was conventionally mounted on the outer periphery of the screen area of a display, to be mounted on the screen area of the display, and minimizes the influence of light emission of the display by utilizing related information between the display and the illuminance sensor, thereby enabling accurate illuminance values to be measured.

In addition, various embodiments of the present invention can provide an electronic device including a display and a sensor, and an operating method thereof, which can enable an illuminance sensor, which was conventionally mounted on the outer periphery of the screen area of a display, to be mounted on the screen area of the display, and can detect and compensate for deterioration of the display by using sensing data acquired by the illuminance sensor when an image is displayed on the display.

According to various embodiments of the present invention, an electronic device comprises: a housing; a display disposed on one surface of the housing; a sensor mounted underneath the display inside the housing and detecting light received through the display; and a processor electrically connected to the display and the sensor, wherein the processor may be set to determine illuminance outside the housing by using sensing data acquired by the sensor while the display displays an image and related information between the display and the sensor.

According to various embodiments of the present invention, an electronic device comprises: a housing; a display disposed on one surface of the housing; a sensor mounted underneath the display inside the housing and detecting light received through the display; and a processor electrically connected to the display and the sensor, wherein the processor may be configured to adjust properties of the display by using first sensing data acquired by the sensor when a first image is displayed on the display and second sensing data acquired by the sensor when a second image is displayed on the display.

According to various embodiments of the present invention, a method of operating an electronic device including a display and a sensor may include: an operation of acquiring sensing data by a sensor that detects light received through the display while the display displays an image; and an operation of determining external illuminance using the acquired sensing data and related information between the display and the sensor.

According to various embodiments of the present invention, a method for operating an electronic device including a display and a sensor may include: an operation of obtaining first sensing data by a sensor that detects light received through the display when a first image is displayed on the display; an operation of obtaining second sensing data by the sensor when a second image is displayed on the display; and an operation of adjusting a property of the display using the first sensing data and the second sensing data.

According to various embodiments of the present invention, a non-transitory computer-readable recording medium having recorded thereon a program to be executed on a computer may include executable instructions that, when executed by a processor, cause the processor to perform an operation of: obtaining sensing data by a sensor that detects light received through a display while the display displays an image; and determining external illuminance by using the obtained sensing data and related information between the display and the sensor.

According to various embodiments of the present invention, a non-transitory computer-readable recording medium having recorded thereon a program to be executed on a computer may include executable instructions to perform the following operations: when a first image is displayed on a display, obtaining first sensing data by a sensor that detects light received through the display; when a second image is displayed on the display, obtaining second sensing data by the sensor; and adjusting properties of the display using the first sensing data and the second sensing data.

According to various embodiments of the present invention, by providing an electronic device including a display and a sensor and an operating method thereof, an illuminance sensor can be mounted on a screen area of a display, thereby expanding the design freedom of mounting the illuminance sensor and securing the freedom of design. In addition, by utilizing related information between the display and the illuminance sensor, the influence of the display's light emission can be minimized, thereby measuring an accurate illuminance value. In addition, when an image is displayed on the display, deterioration of the display can be detected and compensated for by utilizing sensing data acquired by the illuminance sensor.

FIG. 1 is a block diagram illustrating an electronic device within a network environment in various embodiments of the present invention.
FIG. 2 is a block diagram of an electronic device according to various embodiments of the present invention.
FIG. 3 is a block diagram of a program module according to various embodiments of the present invention.
FIG. 4a is a drawing showing the front side of an electronic device according to various embodiments of the present invention.
FIG. 4b is a drawing showing the rear side of an electronic device according to various embodiments of the present invention.
FIG. 4c is a drawing showing a stacking state of an electronic device according to various embodiments of the present invention.
FIG. 5 is a drawing for explaining the operation of a light sensor mounted under a display in an electronic device according to various embodiments of the present invention.
FIG. 6 is a block diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.
FIG. 7 is a flowchart illustrating a procedure for determining external illuminance in an electronic device according to various embodiments of the present invention.
FIG. 8 is a flowchart illustrating a procedure for obtaining related information between a display and a sensor in an electronic device according to various embodiments of the present invention.
FIG. 9 is a diagram for explaining a method of obtaining related information between a display and a sensor in an electronic device according to various embodiments of the present invention.
FIG. 10 is a flowchart illustrating a procedure for determining external illuminance in an electronic device according to various embodiments of the present invention.
FIG. 11 is a flowchart illustrating a procedure for adjusting properties of a display in an electronic device according to various embodiments of the present invention.
FIG. 12 is a flowchart illustrating a procedure for obtaining a difference value for an image displayed on a display in an electronic device according to various embodiments of the present invention.
FIG. 13 is a diagram for explaining a method for obtaining a difference value for an image displayed on a display in an electronic device according to various embodiments of the present invention.
FIG. 14 is a flowchart illustrating a procedure for adjusting properties of a display in an electronic device according to various embodiments of the present invention.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It should be understood that the embodiments and the terms used herein are not intended to limit the technology described in this document to a specific embodiment, but rather to include various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A and/or B" may include all possible combinations of the items listed together. Expressions such as "first," "second," "first," or "second," may modify the corresponding components, regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a certain (e.g., a first) component is "(functionally or communicatively) connected" or "connected" to another (e.g., a second) component, said certain component may be directly connected to said other component, or may be connected through another component (e.g., a third component).

In this document, "configured to" can be used interchangeably with, for example, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to," either hardware-wise or software-wise. In some contexts, the phrase "a device configured to" can mean that the device is "capable of" doing something together with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" can mean a dedicated processor (e.g., an embedded processor) for performing the operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in a memory device.

An electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a medical device, a camera, or a wearable device. A wearable device may include at least one of an accessory (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), an integrated piece of fabric or clothing (e.g., an electronic garment), a body-attached (e.g., a skin pad or tattoo), or an implantable circuit. In some embodiments, the electronic device may include at least one of a television, a digital video disk (DVD) player, an audio system, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync ^™ , AppleTV ^™ , or Google TV ^™ ), a game console (e.g., Xbox ^™ , PlayStation ^™ ), an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include at least one of various medical devices (e.g., various portable medical measuring devices (such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), a camera, or an ultrasound machine), a navigation device, a global navigation satellite system (GNSS), an event data recorder (EDR), a flight data recorder (FDR), an automobile infotainment device, electronic equipment for ships (e.g., a navigation device for ships, a gyrocompass, etc.), avionics, a security device, a head unit for cars, an industrial or household robot, a drone, an ATM of a financial institution, a point of sales (POS) of a store, or an Internet of Things device (e.g., a light bulb, various sensors, a sprinkler device, a fire alarm, a thermostat, a streetlight, a toaster, exercise equipment, a hot water tank, a heater, a boiler, etc.). According to some embodiments, the electronic device may include at least one of a piece of furniture, a building/structure, or a part of a vehicle, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (e.g., a water, electricity, gas, or radio wave measuring device, etc.). In various embodiments, the electronic device may be flexible, or may be a combination of two or more of the various devices described above. The electronic device according to the embodiments of the present document is not limited to the devices described above. In this document, the term user may refer to a person using the electronic device or a device (e.g., an artificial intelligence electronic device) using the electronic device.

Referring to FIG. 1, an electronic device (101) within a network environment (100) according to various embodiments is described. The electronic device (101) may include a bus (110), a processor (120), a memory (130), an input/output interface (150), a display (160), and a communication interface (170). In some embodiments, the electronic device (101) may omit at least one of the components or additionally include other components. The bus (110) may include a circuit that connects the components (110-170) to each other and transmits communication (e.g., control messages or data) between the components. The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may execute, for example, calculations or data processing related to control and/or communication of at least one other component of the electronic device (101).

The memory (130) may include volatile and/or nonvolatile memory. The memory (130) may store, for example, instructions or data related to at least one other component of the electronic device (101). According to one embodiment, the memory (130) may store software and/or programs (140). The programs (140) may include, for example, a kernel (141), middleware (143), an application programming interface (API) (145), and/or an application program (or “application”) (147). At least a portion of the kernel (141), the middleware (143), or the API (145) may be referred to as an operating system. The kernel (141) may control or manage system resources (e.g., a bus (110), a processor (120), or a memory (130)) used to execute operations or functions implemented in other programs (e.g., a middleware (143), an API (145), or an application program (147)). In addition, the kernel (141) may provide an interface that allows the middleware (143), the API (145), or the application program (147) to control or manage system resources by accessing individual components of the electronic device (101).

The middleware (143) may, for example, act as an intermediary so that the API (145) or the application program (147) can communicate with the kernel (141) to exchange data. In addition, the middleware (143) may process one or more work requests received from the application program (147) according to priorities. For example, the middleware (143) may give priority to at least one of the application programs (147) to use the system resources (e.g., the bus (110), the processor (120), the memory (130), etc.) of the electronic device (101) and process the one or more work requests. The API (145) is an interface for the application (147) to control functions provided by the kernel (141) or the middleware (143), and may include at least one interface or function (e.g., a command) for, for example, file control, window control, image processing, or character control. The input/output interface (150) may, for example, transmit commands or data input from a user or another external device to other component(s) of the electronic device (101), or output commands or data received from other component(s) of the electronic device (101) to the user or another external device.

The display (160) may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. The display (160) may, for example, display various contents (e.g., text, images, videos, icons, and/or symbols) to a user. The display (160) may include a touch screen and may receive, for example, touch, gesture, proximity, or hovering inputs using an electronic pen or a part of the user's body. The communication interface (170) may, for example, establish communication between the electronic device (101) and an external device (e.g., the first external electronic device (102), the second external electronic device (104), or the server (106)). For example, the communication interface (170) may be connected to a network (162) via wireless or wired communication to communicate with an external device (e.g., a second external electronic device (104) or a server (106)).

The wireless communication may include, for example, cellular communication using at least one of LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). In one embodiment, the wireless communication may include, for example, at least one of WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication (NFC), Magnetic Secure Transmission, radio frequency (RF), or body area network (BAN). In one embodiment, the wireless communication may include GNSS. The GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard232 (RS-232), a power line communication, or a plain old telephone service (POTS). The network (162) may include at least one of a telecommunications network, for example, a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices (102, 104) may be the same or a different type of device as the electronic device (101). According to various embodiments, all or part of the operations executed in the electronic device (101) may be executed in another one or more electronic devices (e.g., the electronic device (102, 104), or the server (106). According to one embodiment, when the electronic device (101) is to perform a certain function or service automatically or upon request, the electronic device (101) may request at least some functions related thereto from another device (e.g., the electronic device (102, 104), or the server (106)) instead of executing the function or service by itself or in addition. The other electronic device (e.g., the electronic device (102, 104), or the server (106)) may execute the requested function or additional function and transmit the result to the electronic device (101). The electronic device (101) may process the received result as is or additionally to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

FIG. 2 is a block diagram of an electronic device (201) according to various embodiments. The electronic device (201) may include, for example, all or part of the electronic device (101) illustrated in FIG. 1. The electronic device (201) may include one or more processors (e.g., AP) (210), a communication module (220), a subscriber identification module (224), a memory (230), a sensor module (240), an input device (250), a display (260), an interface (270), an audio module (280), a camera module (291), a power management module (295), a battery (296), an indicator (297), and a motor (298). The processor (210) may control a plurality of hardware or software components connected to the processor (210) by, for example, driving an operating system or an application program, and may perform various data processing and calculations. The processor (210) may be implemented as, for example, a SoC (system on chip). According to one embodiment, the processor (210) may further include a GPU (graphic processing unit) and/or an image signal processor. The processor (210) may also include at least some of the components illustrated in FIG. 2 (e.g., a cellular module (221)). The processor (210) may load commands or data received from at least one of the other components (e.g., a non-volatile memory) into the volatile memory and process them, and store the resulting data in the non-volatile memory.

The communication module (220) may have the same or similar configuration as the communication interface (170). The communication module (220) may include, for example, a cellular module (221), a WiFi module (223), a Bluetooth module (225), a GNSS module (227), an NFC module (228), and an RF module (229). The cellular module (221) may provide, for example, a voice call, a video call, a text service, or an Internet service through a communication network. According to one embodiment, the cellular module (221) may perform differentiation and authentication of the electronic device (201) within a communication network by using a subscriber identification module (e.g., a SIM card) (224). According to one embodiment, the cellular module (221) may perform at least some of the functions that the processor (210) may provide. According to one embodiment, the cellular module (221) may include a communication processor (CP). In some embodiments, at least some (e.g., two or more) of the cellular module (221), the WiFi module (223), the Bluetooth module (225), the GNSS module (227), or the NFC module (228) may be included in a single integrated chip (IC) or IC package. The RF module (229) may, for example, transmit and receive communication signals (e.g., RF signals). The RF module (229) may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. In other embodiments, at least one of the cellular module (221), the WiFi module (223), the Bluetooth module (225), the GNSS module (227), or the NFC module (228) may transmit and receive RF signals via a separate RF module. The subscriber identification module (224) may include, for example, a card or embedded SIM including a subscriber identification module and may include unique identification information (e.g., an integrated circuit card identifier (ICCID)) or subscriber information (e.g., an international mobile subscriber identity (IMSI)).

The memory (230) (e.g., memory (130)) may include, for example, built-in memory (232) or external memory (234). The built-in memory (232) may include, for example, at least one of a volatile memory (e.g., DRAM, SRAM, or SDRAM), a nonvolatile memory (e.g., one time programmable ROM (OTPROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, flash memory, a hard drive, or a solid state drive (SSD). The external memory (234) may include a flash drive, for example, a compact flash (CF), a secure digital (SD), a Micro-SD, a Mini-SD, an extreme digital (xD), a multi-media card (MMC), or a memory stick. The external memory (234) may be functionally or physically connected to the electronic device (201) via various interfaces.

The sensor module (240) may, for example, measure a physical quantity or detect an operating state of an electronic device (201), and convert the measured or detected information into an electrical signal. The sensor module (240) may include, for example, at least one of a gesture sensor (240A), a gyro sensor (240B), a pressure sensor (240C), a magnetic sensor (240D), an acceleration sensor (240E), a grip sensor (240F), a proximity sensor (240G), a color sensor (240H) (e.g., an RGB (red, green, blue) sensor), a biometric sensor (240I), a temperature/humidity sensor (240J), an illuminance sensor (240K), or a UV (ultra violet) sensor (240M). Additionally or alternatively, the sensor module (240) may include, for example, an e-nose sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module (240) may further include a control circuit for controlling at least one or more sensors included therein. In some embodiments, the electronic device (201) further includes a processor configured to control the sensor module (240), either as part of or separately from the processor (210), such that the sensor module (240) can be controlled while the processor (210) is in a sleep state.

The input device (250) may include, for example, a touch panel (252), a (digital) pen sensor (254), a key (256), or an ultrasonic input device (258). The touch panel (252) may use, for example, at least one of a capacitive, a pressure-sensitive, an infrared, or an ultrasonic method. In addition, the touch panel (252) may further include a control circuit. The touch panel (252) may further include a tactile layer to provide a tactile response to the user. The (digital) pen sensor (254) may be, for example, a part of the touch panel or may include a separate recognition sheet. The key (256) may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device (258) may detect ultrasonic waves generated from an input tool through a microphone (e.g., a microphone (288)) and confirm data corresponding to the detected ultrasonic waves.

The display (260) (e.g., the display (160)) may include a panel (262), a holographic device (264), a projector (266), and/or a control circuit for controlling them. The panel (262) may be implemented to be, for example, flexible, transparent, or wearable. The panel (262) may be composed of a touch panel (252) and one or more modules. According to one embodiment, the panel (262) may include a pressure sensor (or force sensor) capable of measuring the intensity of pressure for a user's touch. The pressure sensor may be implemented as an integral part of the touch panel (252), or may be implemented as one or more sensors separate from the touch panel (252). The holographic device (264) may display a three-dimensional image in the air by utilizing interference of light. The projector (266) may display an image by projecting light onto a screen. The screen may be located, for example, inside or outside the electronic device (201). The interface (270) may include, for example, HDMI (272), USB (274), an optical interface (276), or D-sub (D-subminiature) (278). The interface (270) may be included, for example, in the communication interface (170) illustrated in FIG. 1. Additionally or alternatively, the interface (270) may include, for example, a mobile high-definition link (MHL) interface, an SD card/multi-media card (MMC) interface, or an infrared data association (IrDA) standard interface.

The audio module (280) can convert, for example, sound and electrical signals bidirectionally. At least some components of the audio module (280) can be included, for example, in the input/output interface (145) illustrated in FIG. 1. The audio module (280) can process sound information input or output through, for example, a speaker (282), a receiver (284), an earphone (286), or a microphone (288). The camera module (291) is, for example, a device capable of capturing still images and moving images, and according to one embodiment, can include one or more image sensors (e.g., a front sensor or a rear sensor), a lens, an image signal processor (ISP), or a flash (e.g., an LED or a xenon lamp). The power management module (295) can manage, for example, power of the electronic device (201). According to one embodiment, the power management module (295) can include a power management integrated circuit (PMIC), a charging IC, or a battery or fuel gauge. The PMIC may have a wired and/or wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include additional circuits for wireless charging, for example, a coil loop, a resonant circuit, or a rectifier. The battery gauge may measure, for example, the remaining capacity of the battery (296), voltage, current, or temperature during charging. The battery (296) may include, for example, a rechargeable battery and/or a solar cell.

The indicator (297) can indicate a specific state of the electronic device (201) or a part thereof (e.g., the processor (210)), for example, a booting state, a message state, or a charging state. The motor (298) can convert an electrical signal into a mechanical vibration and generate a vibration, a haptic effect, or the like. The electronic device (201) can include a mobile TV support device (e.g., a GPU) capable of processing media data according to a standard such as DMB (digital multimedia broadcasting), DVB (digital video broadcasting), or mediaFlo ^TM . Each of the components described in this document can be composed of one or more components, and the names of the corresponding components can vary depending on the type of the electronic device. In various embodiments, the electronic device (e.g., the electronic device (201)) may omit some components, include additional components, or combine some of the components to form a single entity, but may perform the functions of the corresponding components before combination in the same manner.

FIG. 3 is a block diagram of a program module according to various embodiments. According to one embodiment, a program module (310) (e.g., a program (140)) may include an operating system that controls resources related to an electronic device (e.g., an electronic device (101)) and/or various applications (e.g., application programs (147)) that run on the operating system. The operating system may include, for example, Android ^TM , iOS ^TM , Windows ^TM , Symbian ^TM , Tizen ^TM , or Bada ^TM . Referring to FIG. 3, the program module (310) may include a kernel (320) (e.g., kernel (141)), middleware (330) (e.g., middleware (143)), (API (360) (e.g., API (145)), and/or application (370) (e.g., application program (147)). At least a portion of the program module (310) may be preloaded on the electronic device or may be downloaded from an external electronic device (e.g., electronic device (102, 104), server (106), etc.).

The kernel (320) may include, for example, a system resource manager (321) and/or a device driver (323). The system resource manager (321) may perform control, allocation, or recovery of system resources. According to one embodiment, the system resource manager (321) may include a process management unit, a memory management unit, or a file system management unit. The device driver (323) may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an IPC (inter-process communication) driver. The middleware (330) may provide, for example, functions commonly required by the application (370) or provide various functions to the application (370) through the API (360) so that the application (370) may use limited system resources within the electronic device. According to one embodiment, the middleware (330) may include at least one of a runtime library (335), an application manager (341), a window manager (342), a multimedia manager (343), a resource manager (344), a power manager (345), a database manager (346), a package manager (347), a connectivity manager (348), a notification manager (349), a location manager (350), a graphics manager (351), or a security manager (352).

The runtime library (335) may include, for example, library modules used by a compiler to add new functions through a programming language while the application (370) is running. The runtime library (335) may perform input/output management, memory management, or arithmetic function processing. The application manager (341) may manage, for example, the life cycle of the application (370). The window manager (342) may manage GUI resources used on the screen. The multimedia manager (343) may identify a format required for playing media files and perform encoding or decoding of the media files using a codec suitable for the format. The resource manager (344) may manage the source code of the application (370) or the space of the memory. The power manager (345) may manage, for example, the capacity or power of a battery and provide power information required for the operation of the electronic device. According to one embodiment, the power manager (345) may be linked with a basic input/output system (BIOS). A database manager (346) can, for example, create, search, or modify a database to be used by an application (370). A package manager (347) can manage the installation or update of an application distributed in the form of a package file.

The connectivity manager (348) may, for example, manage wireless connections. The notification manager (349) may, for example, provide events such as arrival messages, appointments, and proximity notifications to the user. The location manager (350) may, for example, manage location information of the electronic device. The graphics manager (351) may, for example, manage graphic effects to be provided to the user or a user interface related thereto. The security manager (352) may, for example, provide system security or user authentication. According to one embodiment, the middleware (330) may include a telephony manager for managing voice or video call functions of the electronic device or a middleware module that may form a combination of the functions of the aforementioned components. According to one embodiment, the middleware (330) may provide modules specialized for each type of operating system. The middleware (330) may dynamically delete some of the existing components or add new components. API (360) is, for example, a set of API programming functions, and may be provided in different configurations depending on the operating system. For example, in the case of Android or iOS, one set of APIs may be provided for each platform, and in the case of Tizen, two or more sets of APIs may be provided for each platform.

The application (370) may include, for example, a home (371), a dialer (372), an SMS/MMS (373), an IM (instant message) (374), a browser (375), a camera (376), an alarm (377), a contact (378), a voice dial (379), an email (380), a calendar (381), a media player (382), an album (383), a watch (384), a healthcare (e.g., measuring the amount of exercise or blood sugar, etc.), or an environmental information (e.g., barometric pressure, humidity, or temperature information) providing application. According to one embodiment, the application (370) may include an information exchange application that can support information exchange between the electronic device and an external electronic device. The information exchange application may include, for example, a notification relay application for transmitting specific information to an external electronic device, or a device management application for managing the external electronic device. For example, a notification delivery application may deliver notification information generated from another application of the electronic device to an external electronic device, or receive notification information from an external electronic device and provide it to a user. A device management application may, for example, install, delete, or update a function of an external electronic device that communicates with the electronic device (e.g., turning on/off the external electronic device itself (or some components) or adjusting the brightness (or resolution) of the display), or an application operating on the external electronic device. According to one embodiment, the application (370) may include an application specified according to the property of the external electronic device (e.g., a health management application of a mobile medical device). According to one embodiment, the application (370) may include an application received from an external electronic device. At least a part of the program module (310) may be implemented (e.g., executed) by software, firmware, hardware (e.g., processor (210)), or a combination of at least two or more of these, and may include modules, programs, routines, instruction sets, or processes for performing one or more functions.

The term "module" as used in this document includes a unit composed of hardware, software or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "module" may be an integrally composed component or a minimum unit or a part thereof that performs one or more functions. A "module" may be implemented mechanically or electronically, and may include, for example, an ASIC (application-specific integrated circuit) chip, FPGAs (field-programmable gate arrays), or a programmable logic device, known or to be developed in the future, that performs certain operations. At least a portion of a device (e.g., modules or functions thereof) or a method (e.g., operations) according to various embodiments may be implemented as an instruction stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the instruction is executed by a processor (e.g., processor (120)), the processor may perform a function corresponding to the instruction. The computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (e.g., a magnetic tape), an optical recording medium (e.g., a CD-ROM, a DVD, a magneto-optical medium (e.g., a floptical disk), a built-in memory, etc. The instructions may include codes generated by a compiler or codes executable by an interpreter. A module or program module according to various embodiments may include at least one or more of the above-described components, some of them may be omitted, or other components may be further included. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some of the operations may be executed in a different order, omitted, or other operations may be added.

FIG. 4A is a drawing showing the front side of an electronic device according to various embodiments of the present invention, FIG. 4B is a drawing showing the back side of an electronic device according to various embodiments of the present invention, and FIG. 4C is a drawing showing a stacked state of an electronic device according to various embodiments of the present invention. The electronic device (401) may be a smartphone or a wearable device. With reference to FIGS. 4A to 4C, components of an electronic device such as a smartphone will be described.

According to various embodiments of the present invention, the electronic device (401) may include a housing (490), a display (460), a structure including a through opening (hereinafter referred to as a 'lower panel (460b)'), and a light sensor area (440).

According to various embodiments of the present invention, the housing (490) may include a front surface facing in a first direction, a rear surface facing in a second direction opposite to the first direction, and a side surface surrounding at least a portion of the space between the front surface and the rear surface.

According to various embodiments of the present invention, a display (460) may be arranged at the center of the front surface of the housing (490). The display (460) may be formed to be large enough to occupy most of the front surface of the electronic device (401). According to various embodiments, the display (460) may be a touch screen. Additionally, the display (460) may be an AMOLED (Active Matrix Organic Light Emitting Diode) display.

According to various embodiments of the present invention, the display (460) is a device disposed within the housing (490), and may include a first surface including a display screen exposed through the front surface of the housing (490), and a second surface facing in the opposite direction of the first surface. In addition, the front surface of the housing (490) including the display (460) may be divided into a screen area (AA) on which the screen of the display (460) is displayed, and a non-screen area surrounding the screen area (AA). For example, the screen area (AA) is an area where a screen is displayed by a display panel (466) described below, and input and output can be implemented by a touch panel (463), and the non-screen area is an area where signal lines or terminals provided on a touch panel (463) or a display panel (466) located at the periphery of the screen area (AA) are disposed, and are areas shielded by various colors or the housing (490) to limit exposure of the signal lines or terminals to the outside.

According to various embodiments of the present invention, the light sensor area (440) may be positioned at a location of the display (460), for example, in the screen area (AA) of the display (460), but may be positioned toward an opening formed in a lower panel positioned below the display (460). The light sensor area (440) may be described later.

A home button (451), a menu button (452), and a back button (453) may be formed at the bottom of the display (460). The home button (451) may display a main home screen on the display (460). For example, when the home button (451) is touched while the display (460) displays a home screen other than the main home screen (any home screen) or a menu screen, the main home screen may be displayed on the display (460). In addition, the home button (451) may be used to display recently used applications on the display (460) or to display a task manager. The menu button (452) provides a connection menu that may be used on the display (460). The connection menu may include a widget addition menu, a background change menu, a search menu, an edit menu, a settings menu, etc. The back button (453) can display the screen that was running immediately before the currently running screen, or terminate the most recently used application.

A first camera (421) and a proximity sensor (431) may be placed on the upper front area of the electronic device (401). A second camera (422), a flash, and a speaker may be placed on the rear of the electronic device (401).

According to various embodiments of the present invention, the electronic device (401) may include at least one sensor (e.g., a first camera (421), a second camera (422), a proximity sensor (431), etc.) forming at least a portion of an external surface of the electronic device in a housing (490). Additionally, the sensor (e.g., a light sensor region (440)) may be located between the display (460a) and the housing (490).

A connector (458) may be formed on the lower side of the electronic device (401), into which an electronic pen (459) may be inserted. A plurality of electrodes may be formed on the connector (458), and may be connected to an external device via a wire. An earphone connection jack (432) may be formed on the upper side of the electronic device (401). An earphone may be inserted into the earphone connection jack (432).

According to various embodiments of the present invention, the electronic device (401) may be implemented in various forms, such as a wrap-around form, a full front display form (e.g., a form in which the front is composed of a display and there is no or a minimized bezel), a transparent device, etc., and various embodiments of the present invention are not limited to a specific form of the electronic device (401).

In addition, according to various embodiments of the present invention, when the electronic device (401) is a transparent display or a full front display, the display color can be changed or adjusted according to the color information of the floor surface when the electronic device (401) is placed on the floor.

Referring to FIG. 4c, the display (460a) may have a structure including a surface that is disposed between the second surface of the display (460a) and the rear surface of the housing (490) and contacts a significant portion of the second surface of the display (460). As described below, when viewed from above the front surface of the housing (490), a through opening (hereinafter referred to as a 'first opening' or 'first and second openings') may be included so that a light sensor region (440) can be disposed in the screen area of the display (460a).

According to various embodiments of the present invention, a display (460a) may be provided to display a screen and implement an input. A display (460a) according to one embodiment of the present invention may include a window panel (461a), an adhesive layer such as an optically transparent mounting film (462a, 464, hereinafter referred to as an 'optically transparent mounting film'), a first conductive pattern (hereinafter referred to as a 'touch panel (463)') capable of implementing an input depending on proximity or contact, an OLED layer including a polarizing panel (465) and a display panel (466), first and second polymer layers (467, 468), etc.

According to various embodiments of the present invention, the window panel (461a) may be installed at the frontmost part of the display (460a) and may be made of a material such as transparent glass to protect the display (460a).

According to various embodiments of the present invention, the touch panel (463) may be disposed between the OLED layer described below and the first surface of the display (460a). The touch panel (463) is a component configured to implement an input according to contact or proximity of the screen area (AA). According to various embodiments of the present invention, the touch panel (463) may be implemented in various ways, such as a capacitive method, an electromagnetic resonance method, a resistive film method, an infrared method, or an ultrasonic method, and may also be implemented in a combination of these methods. The touch panel (463) may be disposed on the upper surface of the display (460a) described above, specifically, between the window panel (461a) and the display panel (466).

According to various embodiments of the present invention, an optically clear adhesive (OCA) for mounting may be provided to couple between a window panel (461a) and a touch panel (463) and between a touch panel (463) and a polarizing panel (465). In addition, the optically clear adhesive (OCA) for mounting may be provided to couple the touch panel (463) and the polarizing panel (465) as well as transmit electrical signals between the polarizing panel (465), a second polymer layer (467) such as polyimide, and the touch panel (463).

According to various embodiments of the present invention, the first polymer layer (467) may form a second surface of the display (460a). The first polymer layer (468) may be provided on the lower surface of the second polymer layer (467), and when the second polymer layer (467) is formed thinly, a transparent support polymer panel (PET (polyethylene terephthalate)) may be provided to support and reinforce the second polymer layer (467).

According to various embodiments of the present invention, the second polymer layer (467) may be formed of a polyimide film, and as a component for supplying power to the display panel (466), may be electrically coupled to a flexible printed circuit board (hereinafter, referred to as a first circuit board (447)) to be described later and connected to a main circuit board mounted inside the housing (490).

The second polymer layer (467) may further include a portion (162b) extending from the portion disposed between the first polymer layer (468) and the first surface of the display (460a) and bending in the second direction.

Part (443) according to one embodiment of the present invention can be subjected to COF (chip on film) bending and PI (polyimide) bending depending on the structure in which it is joined and connected, as described later.

The OLED layers (465, 466) may be in contact with the second polymer layer (467) and may be disposed between the second polymer layer (467) and the first surface of the display (460a). The OLED layers (465, 466) may include a polarizing panel (465) and a display panel (466). The polarizing panel (465) may be laminated on the lower surface of an optically transparent mounting film (464) and may be provided to increase the picture quality of a screen of the display panel (466) described below and to improve outdoor reliability.

According to various embodiments of the present invention, the layer for displaying the screen has been described as an AMOLED layer, but is not limited thereto. For example, various modifications or changes may be possible, such as an LCD layer or an LED layer.

As described above, the display (460a) of the present invention may be laminated in the order of a window panel (461a), an optically transparent mounting film (462a), a touch panel (463), an optically transparent mounting film (464), an OLED layer (465, 466), a second polymer layer (467), and a first polymer layer (468) between the first and second surfaces of the display.

Additionally, a second conductive pattern separate from the touch panel may be included between the first polymer layer (468) and the second polymer layer (467).

According to various embodiments of the present invention, a lower panel (460b) may be provided to be laminated on a lower surface of a display (460a) to support the display (460a) and prevent internal modules, etc. from being viewed through the display device. A first opening (449c) may be provided in the lower panel (460b) so that an illuminance sensor area (440), which will be described later, may be installed at a predetermined position in the screen area (AA) of the display (460a).

The lower panel (460b) may be a structure laminated to the lower surface of the second surface of the display (460a) and may include various panels. The lower panel (460b) may include a flexible layer (461b, 462b) and a short-range communication module, etc.

The lower panel (460b) may have a first adhesive layer formed so as to come into contact with the second surface of the display (460a).

A flexible layer (461b, 462b) according to one embodiment of the present invention can be in contact with the first adhesive layer and can be disposed between the first adhesive layer and the second surface of the housing (490).

According to various embodiments of the present invention, the flexible layer (461b, 462b) may include an EMBO panel (461b) and a sponge panel (462b).

The emboss panel (461b) may be made of an opaque material and may be dark in color, used to prevent the internal structure from being visible through the display (460a). It may also serve a supporting role together with the first polymer layer (468) mounted to support or reinforce the second polymer layer (467).

According to various embodiments of the present invention, a sponge panel (462b) may be provided to alleviate and/or eliminate a pressing phenomenon or a lifting phenomenon caused by foreign substances that may occur during bonding during the assembly process of the display (460a).

According to various embodiments of the present invention, a first circuit board (447) may be arranged on a bottom surface of a lower panel (460b). The first circuit board (447) may include a portion arranged between the flexible layers (461b, 462b) and the second surface of the housing (490). The first circuit board may be connected to a main circuit board at a portion including another portion extending from the portion and bent toward the second direction.

The above-described display (460a) or lower panel (460b) must be electrically connected to the first circuit board (447). The display device (460a), for example, the first polymer layer (468) and the first circuit board (447) may be electrically connected through COF bending (chip on film bending). Alternatively, the display (460a), for example, the first polymer layer (468) and the first circuit board (447) may be electrically connected through PI bending (polyImide bending).

According to various embodiments of the present invention, a first opening (449c) may be formed in the lower panel (460b) so that a light sensor area (440) can be provided at a predetermined position corresponding to the screen area (AA).

According to various embodiments of the present invention, the lower panel (460b) may be implemented with an opaque material so as to limit visibility of internal components of the display (460a), unlike the upper panel (460a). Accordingly, a light sensor (441) may be arranged in the lower panel (460b), and the first opening (449c) may be provided so that external ambient light may enter the light sensor (441). Specifically, the display (460a) according to various embodiments of the present invention is formed with a transparent material so as to display a screen generated from the display panel (466), and thus external ambient light may enter the upper panel (460a) and the second polymer layer (467). However, in order to prevent the internal structure from being recognized, a first opening (449c) is formed in the lower panel (460b) so that the illuminance sensor (441) described below can be mounted corresponding to the screen area (AA) and external ambient light can reach the illuminance sensor (441).

According to various embodiments of the present invention, the light sensor area (440), as mentioned above, can be placed at a predetermined position in the screen area (AA) of the display (460a), and for this purpose, can be mounted at the positions of the first and second openings (449c, 448) formed on the screen area (AA) of the display (460a).

The light sensor area (440) according to various embodiments of the present invention may include a second circuit board (442) and a light sensor (441).

The second circuit board (442) may be laminated on the lower surface of the first circuit board (447) and may be provided to cover the second opening (448) corresponding to the position where the second opening (448) is formed. The second circuit board (442) may be provided to be electrically connected to the first circuit board (447) so as to transmit the signal of the light sensor (441) to the first circuit board (447). An illuminance sensor (441) and a connecting member (449a) and a connecting terminal (449b) described below may be provided on one surface of the second circuit board (442).

The light sensor (441) is electrically mounted on the upper portion of the second circuit board (442), and when the second circuit board (442) is laminated to fit into the second opening (448) of the first circuit board (447), the light sensor (441) can be provided to be seated on the inner side of the first and second openings. The light sensor (441) can detect external ambient light that enters from the outside of the display (460a) through the first and second openings.

A bonding member (449a) may be provided at a predetermined position around the light sensor (441) so that the second circuit board (442) can be adhered to the first circuit board (447). The bonding member (449a) may be provided as a conductive double-sided tape, or alternatively, may be provided as a non-conductive double-sided tape.

The connection terminal (440b) is provided at a position different from the position of the coupling member (449a) around the light sensor (441), and is provided at a position corresponding to the terminal portion of the first circuit board (447), so that when the second circuit board (447) is coupled to the first circuit board (447), it can be provided so as to be electrically connected to the terminal portion of the first circuit board (447). As described above, the terminal portion and the connection terminal (449b) can be coupled by a heat-melting method, etc.

According to various embodiments of the present invention, the light sensor region (440) is provided to be mounted on a first circuit board (447) in which a second opening (448) is formed so as to be connected to the first opening (449c) while the light sensor (441) is mounted on the second circuit board (447). Accordingly, the first circuit board (447) can have additional rigidity increased by joining with the second circuit board (442), and when the light sensor region (440) is broken or damaged, the light sensor region (440) can be easily replaced by separating only the second circuit board (442) from the bottom surface of the first circuit board (447), thereby realizing ease of aftercare.

In the electronic device (401) having the structure as described above, the light sensor region (440) may be placed at a predetermined position of the screen area (AA) of the display (460a), for example, at a central portion of the screen area (AA) (see FIG. 4a). The light sensor region (440) according to various embodiments of the present invention is described as being mounted to be positioned at the central portion of the screen area (AA), but is not limited thereto, and may be mounted at any position of the screen area (AA).

External ambient light can pass through the transparent window panel (461a), the touch panel (463), and the display panel (466) to enter the first and second openings, and can enter the light sensor (441) mounted on the inside of the first and second openings. The light sensor (441) mounted on the screen area (AA) of the display (460a) can detect the entered ambient light, and the brightness of the display (460a) can be adjusted according to the external brightness based on the detected value.

FIG. 5 is a drawing for explaining the operation of a light sensor mounted under a display in an electronic device according to various embodiments of the present invention.

Referring to FIG. 5, according to various embodiments of the present invention, a light sensor may be mounted below a screen area of a display to detect external ambient light passing through the display. The external ambient light may be filtered through a light sensor filter of the light sensor. In addition, a photodiode of the light sensor may convert filtering data into an electrical signal. An analog level electrical signal converted by the light sensor may be changed into a digital level electrical signal through a signal conversion module (ADC Converter).

According to various embodiments of the present invention, interference may occur due to light emitted from a display in obtaining full external ambient light entering a light sensor, and a technique for minimizing the influence of light emission from the display is required.

According to various embodiments of the present invention, the actual external illuminance can be detected through a digital level value (digital level data) of ambient light acquired through a light sensor, and the brightness of a display can be automatically adjusted according to the detected value.

According to various embodiments of the present invention, a light sensor is positioned on a lower panel of a back surface of a display so as to be arranged in a screen area of the display. In this case, an error may occur due to the light emission of the display in a low-light environment. Accordingly, if both transmission and reception are possible in a DDI (display driver IC) and the pixel information of the DDI can be interpreted, the error data occurring when the display is illuminated can be subtracted from the information of the ambient light acquired from the light sensor through the information provided by the DDI (e.g., a brightness value, etc.) to determine the external illuminance and transmit this to the display. In other words, it may be possible to correct the actual brightness of the display by reflecting the light generated from the display of the electronic device.

FIG. 6 is a block diagram illustrating the configuration of an electronic device according to various embodiments of the present invention.

Referring to FIG. 6, an electronic device (601) according to various embodiments of the present invention may include, for example, all or part of the electronic device (101) illustrated in FIG. 1 and the electronic device (201) illustrated in FIG. 2, and may include a processor (610), a display (660), a light sensor (641), a sensor module (642), and a display driver (665).

According to various embodiments of the present invention, the display (660) may include all or part of the display (160) illustrated in FIG. 1 and the display (260) illustrated in FIG. 2. The display (660) may include a display screen exposed through the front of the electronic device (601). The display (660) may display an image according to a brightness setting having at least one brightness value. The brightness setting may be classified according to grades preset to have different brightness values. For example, the brightness setting may be classified into five grades, and an image may be displayed on the display with a different brightness value according to each grade. According to various embodiments, the display (660) may include an AMOLED (Active Matrix Organic Light Emitting Diode) display.

According to various embodiments of the present invention, the light sensor (641) is mounted below the display (660) and can detect light received through the display (660). The light sensor (641) can be placed at a predetermined position in the screen area of the display (660), for example, in the center of the screen area. The light sensor (641) is, for example, a sensor capable of sensing a value related to brightness, and is not limited to a sensor having a specific name, and any type of sensor capable of determining illuminance related to brightness by sensing can be included in the light sensor according to an embodiment of the present invention.

According to various embodiments of the present invention, the sensor module (642) may perform the role of an auxiliary processor capable of processing sensing data acquired by the light sensor (641). The sensor module (642) may control the light sensor (641b) to acquire sensing data and transmit the acquired sensing data to the processor (610b).

According to various embodiments of the present invention, the display driver (665) may perform a role of controlling R/G/B pixels included in the display (660) according to the control of the processor (610). The display driver (665) may include a DDI (display driver IC). The display driver (665) may adjust properties of the display (660). The properties of the display (660) may include a luminance value or a gamma value.

According to various embodiments of the present invention, the processor (610) may include all or part of the processor (120) illustrated in FIG. 1 and the processor (210) illustrated in FIG. 2. The processor (610) may process at least some of the information obtained from other components of the electronic device (601) (e.g., the display (660), the light sensor (641), the display driver (665), etc.) and provide the same to the user in various ways. The processor (610) may control all components of the electronic device (601).

According to various embodiments, the processor (610) may determine the illuminance outside the electronic device (601) by using sensing data acquired by the illuminance sensor (641) while the display (660) displays an image and related information between the display (660) and the illuminance sensor (641).

According to various embodiments, the processor (610) can control the display (660) according to a brightness setting having at least one luminance value.

According to various embodiments, the processor (610) may use the first sensing data acquired by the light sensor (641) and the brightness value of the first brightness setting when the first image is displayed on the display (660) set to the first brightness setting to acquire sensing data related to the display (660) set to at least one brightness setting having a different brightness value from the first brightness setting. The processor (610) may acquire related information including at least the sensing data related to the display (660) according to each brightness setting. The processor (610) may control the display (660) so that the first brightness setting is set, and may display the first image on the display (660) set to the first brightness setting. The processor (610) may acquire sensing data by the light sensor (641) while the display (660) displays the first image at the first brightness setting. The processor (610) can obtain sensing data related to the display (660) set to a brightness setting having a different brightness value from the first brightness setting by using the acquired sensing data and the brightness value according to the first brightness setting of the display (660). The processor (610) can calculate a ratio of the acquired sensing data and the brightness value of the first brightness setting, and automatically calculate sensing data under conditions of different brightness settings by applying the brightness value of each brightness setting to the calculated ratio. The first brightness setting can be set to a grade having the maximum brightness value among grades of a plurality of predetermined brightness settings. In addition, the first image can be an image in which a ratio representing the brightness of the image has the maximum value.

According to various embodiments, the processor (610) may compile sensing data related to the display according to each brightness setting to construct a look-up table. The constructed look-up table may be used to remove a brightness factor due to the display from the illuminance value of the external ambient light when measuring the illuminance value due to the external ambient light using the illuminance sensor.

According to various embodiments, the processor (610) can correct the sensing data values of the look-up table according to a ratio representing the brightness of an image displayed on the display (660), and determine the external illuminance using the corrected sensing data.

According to various embodiments of the present invention, the processor (610) may adjust the display properties of the display (660) by using the first sensing data acquired by the light sensor (641) when a first image is displayed on the display (660) and the second sensing data acquired by the light sensor (641) when a second image is displayed on the display (660). The first image and the second image may be images from which different sensing data may be acquired when displayed on the display (660) under the same conditions. In addition, the first image and the second image may be images from which ratios representing the brightness of the images may have different values. For example, the first image may be an image having an image brightness ratio of 100%, which is the maximum value, and the second image may be an image having an image brightness ratio of 0%, which is the minimum value.

According to various embodiments, the processor (610) obtains a difference value between the first sensing data acquired by the light sensor (641) when a first image is displayed on the display (660) and the second sensing data acquired by the light sensor (641) when a second image is displayed on the display (660), and if a difference occurs when comparing the difference value acquired previously under the same conditions, the processor (610) can adjust the properties of the display (660). The processor (610) can adjust the properties of the display (660) so that the acquired difference value corresponds to the previously acquired difference value. The processor (610) can control the display driver (665) to adjust a luminance value or a gamma value of the display (660) in order to adjust the properties of the display (660).

FIG. 7 is a flowchart illustrating a procedure for determining external illuminance in an electronic device according to various embodiments of the present invention.

Referring to FIG. 7, in operation 710, an electronic device (e.g., a processor (610)) may display an image by, for example, a display (e.g., a display (660)) provided on the front of the electronic device.

In operation 720, the electronic device can obtain sensing data on light received through the display by using a sensor (e.g., a light sensor (641)) mounted underneath the display inside the electronic device and detecting light received through the display.

In operation 730, the electronic device can determine the illuminance outside the electronic device by using the acquired sensing data and the related information (e.g., a look-up table) between the display and the sensor that has been previously stored.

FIG. 8 is a flowchart illustrating a procedure for obtaining related information between a display and a sensor in an electronic device according to various embodiments of the present invention, and FIG. 9 is a diagram for explaining a method for obtaining related information between a display and a sensor in an electronic device according to various embodiments of the present invention.

Referring to FIG. 8, in operation 810, an electronic device (e.g., processor (610)) may display a first image on a display (e.g., display (660)) set to a first brightness setting. The brightness setting of the display may be divided into preset grades to have different brightness values. For example, the brightness setting may be divided into 5 grades, and an image may be displayed on the display with different brightness values according to each grade. Additionally, the first image may be an image in which a ratio representing the brightness of the image has a maximum value.

In operation 820, the electronic device may obtain first sensing data regarding light received through the display using a sensor (e.g., a light sensor (641)).

In operation 830, the electronic device can check the brightness value according to the first brightness setting.

In operation 840, the electronic device can obtain sensing data related to other brightness settings using the first sensing data and the luminance value of the first brightness setting.

In operation 850, the electronic device can obtain related information (e.g., a look-up table) including the acquired sensing data.

Referring to FIG. 9, in the method of obtaining related information between the display and the sensor in the operation 850, first, a first brightness setting of the display can be selected. The first brightness setting can be a level having the highest luminance value among each brightness setting. Then, a first image to be displayed on the display can be selected. The first image can be an image having a maximum brightness ratio of the image. For example, the first image can be an image of a Full White Pattern in which R/G/B sub-pixels are (255, 255, 255). Alternatively, the first image can be an image of a pattern in which one pixel among the R/G/B sub-pixels is selected (e.g., Full RED Pattern (255, 0, 0), Full Green Pattern (0, 255, 0), Full Blue Pattern (0, 0, 255), etc.). Through this, it is possible to obtain illuminance sensing data for each R/G/B sub-pixel, so that data for correction of each R/G/B sub-pixel can be obtained.

When the first brightness setting and the first image are selected, the display can be set to the first brightness setting, and the first image can be displayed on the display set to the first brightness setting. While the first image is displayed on the display set to the first brightness setting, sensing data can be acquired using a light sensor. For example, using the formula shown in Fig. 9, a ratio of the acquired sensing data and the brightness value of the first brightness setting can be calculated, and the brightness value of each brightness grade can be applied to acquire estimated sensing data when the first image is displayed on the display with the corresponding brightness grade.

By collecting each sensing data according to each brightness setting, a look-up table can be constructed as related information between the display and the sensor.

In addition, by using the condition that the brightness ratio and illuminance value of the image have a linear effect, the sensing data values of the look-up table are corrected according to the brightness ratio of the image displayed on the display, so that the sensing data can be produced even in a situation where an image with a different brightness ratio from the image used to produce the look-up table is displayed on the display.

FIG. 10 is a flowchart illustrating a procedure for determining external illuminance in an electronic device according to various embodiments of the present invention.

Referring to FIG. 10, in operation 1010, an electronic device (e.g., processor (610)) may display an image on a display (e.g., display (661)).

In operation 1020, the electronic device can obtain sensing data on light received through the display using a sensor (e.g., a light sensor (641)).

In operation 1030, the electronic device can check the brightness setting of the display and obtain the luminance value of that brightness setting.

In operation 1040, the electronic device can check the related information between the display and the sensor corresponding to the brightness setting of the display.

In operation 1050, the electronic device can determine the image brightness ratio of the image displayed on the display.

In operation 1060, the electronic device can compensate for the brightness setting of the display by providing relevant information corresponding to the brightness ratio of the image displayed on the display.

In operation 1070, the electronic device can determine the external illuminance using the acquired sensing data and the corrected related information.

FIG. 11 is a flowchart illustrating a procedure for adjusting properties of a display in an electronic device according to various embodiments of the present invention.

Referring to FIG. 11, in operation 1110, an electronic device (e.g., processor (610)) may display a first image on a display (e.g., display (661)).

In operation 1120, the electronic device may obtain first sensing data regarding light received through the display using a sensor (e.g., a light sensor (641)).

In operation 1130, the electronic device can display a second image on the display.

In operation 1140, the electronic device can obtain second sensing data regarding light received through the display using a sensor.

In operation 1150, the electronic device can adjust properties of the display using the first sensing data and the second sensing data. The properties of the display can include a luminance value or a gamma value.

FIG. 12 is a flowchart illustrating a procedure for obtaining a difference value for an image displayed on a display in an electronic device according to various embodiments of the present invention, and FIG. 13 is a drawing for explaining a method for obtaining a difference value for an image displayed on a display in an electronic device according to various embodiments of the present invention.

Referring to FIGS. 12 and 13, in operation 1210, an electronic device (e.g., processor (610)) may display a first image on a display (e.g., display (661)).

In operation 1220, the electronic device may obtain first sensing data for light received through the display by using a sensor (e.g., a light sensor (641)). At this time, the first sensing data may be sensing data obtained by combining external ambient light (L) and light (L1) by a first image of the display. The first image may be an image having a lowest image brightness ratio. For example, the first image may be an image of a Full Black Pattern in which R/G/B sub-pixels are (0, 0, 0).

In operation 1230, the electronic device can display a second image on the display.

In operation 1240, the electronic device can obtain second sensing data for light received through the display using the sensor. At this time, the second sensing data can be sensing data obtained by compositely obtaining external ambient light (L) and light (L2) by the second image of the display. The second image can be an image having a maximum image brightness ratio. For example, the second image can be an image of a Full White Pattern in which R/G/B sub-pixels are (255, 255, 255). Alternatively, the second image can be an image of a pattern in which one pixel among the R/G/B sub-pixels is selected (e.g., Full RED Pattern (255, 0, 0), Full Green Pattern (0, 255, 0), Full Blue Pattern (0, 0, 255), etc.). Through this, it is possible to obtain illuminance sensing data for each R/G/B sub-pixel, thereby obtaining data for correction of each R/G/B sub-pixel.

In operation 1250, the electronic device can obtain a difference value between the first sensing data and the second sensing data. The electronic device can store and manage the obtained difference value.

FIG. 14 is a flowchart illustrating a procedure for adjusting properties of a display in an electronic device according to various embodiments of the present invention.

Referring to FIG. 14, in operation 1410, an electronic device (e.g., processor (410)) may display a first image on a display (e.g., display (661)).

In operation 1420, the electronic device can obtain first sensing data regarding light received through the display on which the first image is displayed.

In operation 1430, the electronic device can display a second image on the display.

In operation 1440, the electronic device can obtain second sensing data regarding light received through the display on which the second image is displayed.

In operation 1450, the electronic device can obtain a difference value between the first sensing data and the second sensing data.

In operation 1460, the electronic device can check the difference value acquired under the same conditions as those for acquiring the first and second sensing data.

In operation 1470, the electronic device can compare the difference value obtained at the present time with the difference value obtained previously to determine whether a difference occurs. If no difference occurs in the difference value, the process can be performed by returning to operation 1410.

If a difference occurs in the above difference value in operation 1470, then in operation 1480, the electronic device can adjust the properties of the display so that the difference value corresponds to the previous difference value.

In operation 1490, the electronic device can return to operation 1410 and perform the process again as long as the display is not turned OFF.

And the embodiments of the present invention disclosed in this specification and drawings are only specific examples presented to easily explain the technical content of the present invention and to help understand the present disclosure, and are not intended to limit the scope of the present invention. Therefore, the scope of the present invention should be interpreted as including all changes or modified forms derived based on the technical idea of the present invention in addition to the embodiments disclosed herein.

601: Electronic Device 610: Processor
641: Light sensor 642: Sensor module
660: Display 665: Display driver

Claims (29)

In electronic devices,
Memory that stores instructions;
housing;
A display arranged to form at least one surface of the housing;
A sensor mounted underneath the display within the housing and configured to detect light received from the display and light received through the display; and
a processor electrically connected to the display and the sensor;
The above instructions, when executed by the processor, cause the electronic device to:
While the above display displays an image, sensing data is acquired from the above sensor,
Identify the first brightness setting value of the above display,
When the sensor provides the sensing data to the processor, the image brightness ratio of the image displayed on the display is identified,
Based on the sensing data and the compensation value, the external illuminance of the housing is identified, and the compensation value is identified based on the first brightness setting value and the image brightness ratio.
An electronic device configured to set the brightness of said display to a second brightness setting value determined at least in part based on illuminance outside said housing.
In the first paragraph,
The above display is an electronic device including an AMOLED (Active Matrix Organic Light Emitting Diode) display.
In the first paragraph,
The above sensor is an electronic device including a light sensor.
In the first paragraph,
The above instructions, when executed by the processor, cause the electronic device to:
An electronic device configured to control said display according to said brightness setting of said display.
In the first paragraph,
An electronic device wherein the memory stores related information between the image brightness ratio for the image displayed on the display, the sensing data, the first brightness setting value, and the illuminance outside the housing.
delete delete delete An electronic device in accordance with claim 5, wherein the image brightness ratio includes at least one of a red pattern, a green pattern, or a blue pattern and a white pattern.
delete delete delete delete delete delete A method of operating an electronic device including a display and a sensor,
An operation of acquiring sensing data from the sensor for detecting light received through the display and light received from the display while the display is displaying an image;
An action for identifying a first brightness setting value of the above display;
An action of identifying an image brightness ratio of the image displayed on the display when the sensor provides the sensing data to the processor of the electronic device;
An operation for identifying the external illuminance of a housing included in the electronic device based on the sensing data and the compensation value; wherein the compensation value is identified based on the first brightness setting value and the image brightness ratio;
An operating method comprising: setting the brightness of the display to a second brightness setting value determined at least in part based on illuminance outside the housing;
delete In Article 16,
An operating method comprising: storing related information between the image brightness ratio for the image displayed on the display, the sensing data, the first brightness setting value, and the illuminance outside the housing in a memory of the electronic device.
delete delete delete In the 18th paragraph, the operating method wherein the image brightness ratio includes at least one of a red pattern, a green pattern, or a blue pattern and a white pattern.
delete delete delete delete delete In a non-transitory computer-readable recording medium having recorded thereon a program to be executed on a computer,
The above program, when executed by a processor included in an electronic device, causes the processor to:
An operation of acquiring sensing data from a sensor included in the electronic device for detecting light received through the display and light received from the display while the display included in the electronic device displays an image;
An action for identifying a first brightness setting value of the above display;
An operation of identifying an image brightness ratio of the image displayed on the display when the sensor provides the sensing data to the processor;
An operation for identifying the external illuminance of a housing included in the electronic device based on the sensing data and the compensation value; wherein the compensation value is identified based on the first brightness setting value and the image brightness ratio;
A computer-readable recording medium comprising executable instructions that cause the computer to perform the action of setting the brightness of the display to a second brightness setting value determined at least in part based on illuminance outside the housing. delete

Images