KR102739527B1 - Electronic apparatus and the method thereof - Google Patents

Abstract

In one embodiment of the present invention, an electronic device may include a display; and a processor configured to divide an input image into a plurality of frame sets having a first number of frames, divide the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames, identify a predefined key frame among frames included in each subframe set for each of the plurality of subframe sets, and identify an area within the image in which a luminance change of the frame set is less than a threshold value based on a luminance change between the frames using the identified key frame.

Description

ELECTRONIC APPARATUS AND THE METHOD THEREOF

The present disclosure relates to an electronic device and a control method thereof for preventing deterioration of an image displayed on a display.

Recently, various types of display devices such as organic light emitting display devices, liquid crystal display devices, and plasma display devices are widely used. As these display devices are operated for a long time, specific pixels may deteriorate due to the so-called burn-in phenomenon, which may deteriorate their performance. For example, digital information display devices used for information transmission in public places, vehicles, etc. continuously output specific images or letters for a long time, so the deterioration of specific pixels may accelerate and afterimages may occur. In particular, most broadcasts display the program name or the name of the broadcaster at the top of the screen, so pixel deterioration may occur in areas that are fixedly displayed within the channel. To solve this problem, pixel shift technology, which displays images by moving them periodically on the display panel, and luminance reduction technology, which reduces the luminance of a specific area to extend its lifespan, are being used. However, in order to use these technologies, it is necessary to accurately identify the area where pixel degradation occurs, and this is directly related to the efficiency of each technology.

The purpose of the present disclosure is to provide an electronic device and a control method thereof that more effectively prevent deterioration of an image displayed on a display.

In one embodiment of the present invention, an electronic device may include a display; and a processor configured to divide an input image into a plurality of frame sets having a first number of frames, divide the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames, identify a predefined key frame among frames included in each subframe set for each of the plurality of subframe sets, and identify an area within the image in which a luminance change of the frame set is less than a threshold value based on a luminance change between the frames using the identified key frame.

The processor can identify a luminance change in the frame set based on a luminance change between the key frame included in the subframe set and the other frame.

The processor can identify luminance changes in the set of frames based on sequential luminance changes between the keyframe and the frame following the keyframe.

The above keyframe may be the first frame of the set of subframes that includes the above keyframe.

The processor can identify luminance changes in the frame set based on luminance changes between the keyframes.

The above processor can identify a luminance change of the frame set based on a calculation between luminance changes in the plurality of subframe sets.

The above frame set is a first frame set, and the processor can identify an area within the image based on a luminance change in a second frame set in a section moved by a predefined number of subframe sets in the section of the first frame set.

The processor can identify a region within the image based on a change in luminance of the first set of frames, and identify a change in the region based on a change in luminance of the second set of frames based on the identified region.

The processor can identify an area within the image based on calculations between luminance changes of the first frame set and the second frame set.

The above subframe set is a first subframe set, the keyframe is a first keyframe, and the processor divides a second number of frames included in the subframe set into a plurality of second subframe sets having a third number of frames, identifies a predefined second keyframe among the frames included in each second subframe set for each of the plurality of second subframe sets, and identifies an area within the image based on a luminance change between frames using the identified second keyframe.

A method for controlling an electronic device according to one embodiment of the present invention may include: a step of dividing an input image into a plurality of frame sets having a first number of frames; a step of dividing the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames; a step of identifying a predefined key frame among frames included in each subframe set for each of the plurality of subframe sets; and a step of identifying an area within the image in which a luminance change of the frame set is less than a threshold value based on a luminance change between the frames using the identified key frame.

The step of identifying an area within the image may include the step of identifying a luminance change in the set of frames based on a luminance change between the key frame included in the set of subframes and the other frames.

The step of identifying a luminance change of the set of frames may include a step of identifying a luminance change of the set of frames based on sequential luminance changes between the key frame and the frame subsequent to the key frame.

The step of identifying an area within the image may include the step of identifying a luminance change in the set of frames based on a luminance change between the key frames.

The step of identifying an area within the image may include the step of identifying a luminance change in the set of frames based on a calculation between luminance changes in the plurality of sets of subframes.

The above frame set is a first frame set, and the step of identifying an area within the image may include a step of identifying an area within the image based on a luminance change in a second frame set in a section moved by a predefined number of subframe sets in a section of the first frame set.

The step of identifying a region within the image may include the step of identifying a region within the image based on a luminance change of the first frame set; and the step of identifying a change in the region based on a luminance change of the second frame set with the identified region as a reference.

The subframe set may include a first subframe set, the keyframe is a first keyframe, a step of dividing a second number of frames included in the subframe set into a plurality of second subframe sets having a third number of frames; a step of identifying a predefined second keyframe among the frames included in each second subframe set of the plurality of second subframe sets; and a step of identifying an area within the image based on a luminance change between frames using the identified second keyframe.

A computer program including a computer-readable code that performs a method for controlling an electronic device is stored in a recording medium, wherein the method for controlling an electronic device comprises: a step of dividing an input image into a plurality of frame sets having a first number of frames; a step of dividing the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames; a step of identifying a predefined key frame among frames included in each subframe set for each of the plurality of subframe sets; and a step of identifying an area within the image in which a luminance change in the frame set is less than a threshold value based on a luminance change between the frames using the identified key frame.

In order to prevent degradation of images displayed on a display, the area where degradation occurs can be precisely identified. In addition, even when noise or area loss due to compression occurs in the image, the area where degradation occurs can be stably identified.

FIG. 1 is a diagram illustrating the configuration of an electronic device according to one embodiment of the present invention.
FIG. 2 is a block diagram illustrating an operation flow diagram of an electronic device according to one embodiment of the present invention.
FIG. 3 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention.
FIG. 4 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention.
FIG. 5 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention.
FIG. 6 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention.
FIG. 7 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numbers or symbols refer to components that perform substantially the same functions, and the size of each component in the drawings may be exaggerated for clarity and convenience of explanation. However, the technical idea of the present invention and its core configuration and operation are not limited to the configuration or operation described in the following embodiments. In describing the present invention, if it is determined that a specific description of a known technology or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In the embodiments of the present invention, terms including ordinal numbers such as first, second, etc. are used only for the purpose of distinguishing one component from another component, and singular expressions include plural expressions unless the context clearly indicates otherwise. In addition, in the embodiments of the present invention, it should be understood that terms such as 'configured', 'include', 'have', etc. do not exclude in advance the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In addition, in the embodiments of the present invention, a 'module' or 'part' performs at least one function or operation, may be implemented as hardware or software, or may be implemented as a combination of hardware and software, and may be implemented as an integrated unit of at least one module. In addition, in the embodiments of the present invention, at least one of a plurality of elements refers not only to all of the plurality of elements, but also to each one excluding the rest of the plurality of elements or all combinations thereof.

FIG. 1 is a diagram illustrating the configuration of an electronic device according to one embodiment of the present invention.

At this time, the electronic device (100) may be implemented as a display device capable of displaying an image. For example, the electronic device (100) may include a TV, a computer, a smart phone, a tablet, a portable media player, a wearable device, a video wall, an electronic picture frame, etc.

As illustrated in FIG. 1, the electronic device (100) may include an interface unit (110). The interface unit (110) may include a wired interface unit (111). The wired interface unit (111) includes a connector or port to which an antenna capable of receiving a broadcast signal according to a broadcasting standard such as terrestrial/satellite broadcasting is connected, or a cable capable of receiving a broadcast signal according to a cable broadcasting standard is connected. As another example, the electronic device (100) may have a built-in antenna capable of receiving a broadcast signal. The wired interface unit (111) may include a connector or port according to a video and/or audio transmission standard, such as an HDMI port, a DisplayPort, a DVI port, Thunderbolt, composite video, component video, super video, SCART, etc. The wired interface unit (111) may include a connector or port according to a universal data transmission standard, such as a USB port. The wired interface unit (111) may include a connector or port, etc. to which an optical cable may be connected according to an optical transmission standard. The wired interface unit (111) may include a connector or port, etc. to which an external microphone or an external audio device equipped with a microphone is connected, and which may receive or input audio signals from the audio device. The wired interface unit (111) may include a connector or port, etc. to which an audio device such as a headset, earphones, or an external speaker is connected, and which may transmit or output audio signals to the audio device. The wired interface unit (111) may include a connector or port according to a network transmission standard such as Ethernet. For example, the wired interface unit (111) may be implemented as a LAN card, etc. that is wired and connected to a router or gateway.

The wired interface unit (111) receives video/audio signals from the external device or transmits video/audio signals to the external device by being wired 1:1 or 1:N (N is a natural number) with an external device such as a set-top box, an optical media player, or an external display device, speaker, server, etc. through the connector or port. The wired interface unit (111) may also include a connector or port that separately transmits video/audio signals.

And, according to the present embodiment, the wired interface unit (111) is built into the electronic device (100), but may also be implemented in the form of a dongle or module and be detachable from the connector of the electronic device (100).

The interface unit (110) may include a wireless interface unit (112). The wireless interface unit (112) may be implemented in various ways corresponding to the implementation form of the electronic device (100). For example, the wireless interface unit (112) may use wireless communication such as RF (radio frequency), Zigbee, Bluetooth, Wi-Fi, UWB (Ultra WideBand), and NFC (Near Field Communication) as a communication method. The wireless interface unit (112) may be implemented as a wireless communication module that performs wireless communication with an AP according to the Wi-Fi method, or a wireless communication module that performs 1:1 direct wireless communication such as Bluetooth. The wireless interface unit (112) may transmit and receive data packets between the server and the server by wirelessly communicating with the server on the network. The wireless interface unit (112) may include an IR transmitter and/or an IR receiver capable of transmitting and/or receiving an IR (Infrared) signal according to an infrared communication standard. The wireless interface unit (112) may receive or input a remote control signal from a remote control or another external device through the IR transmitter and/or the IR receiver, or may transmit or output a remote control signal to another external device. As another example, the electronic device (100) may transmit and receive a remote control signal to and from a remote control or another external device through a wireless interface unit (112) of another type, such as Wi-Fi or Bluetooth.

If the video/audio signal received through the interface unit (110) is a broadcast signal, the electronic device (100) may further include a tuner that tunes the received broadcast signal by channel.

The electronic device (100) may include a display unit (120). The display unit (120) includes a display panel capable of displaying an image on a screen. The display panel is provided with a light-receiving structure such as a liquid crystal display or a self-luminous structure such as an OLED display. The display unit (120) may further include additional components depending on the structure of the display panel. For example, if the display panel is a liquid crystal display, the display unit (120) includes a liquid crystal display panel, a backlight unit that supplies light, and a panel driving substrate that drives liquid crystals of the liquid crystal display panel.

The electronic device (100) may include a user input unit (130). The user input unit (130) includes various types of input interface-related circuits provided to perform user input. The user input unit (130) may have various configurations depending on the type of the electronic device (100), and for example, may include a mechanical or electronic button unit of the electronic device (100), a remote controller separated from the electronic device (100), an input unit in an external device connected to the electronic device (100), a touchpad, a touch screen installed in the display unit (120), etc.

The electronic device (100) may include a storage unit (140). The storage unit (140) stores digitized data. The storage unit (140) includes a non-volatile storage capable of preserving data regardless of whether power is supplied, and a volatile memory into which data to be processed by the processor (170) is loaded and which cannot preserving data if power is not supplied. The storage includes a flash memory, a hard-disc drive (HDD), a solid-state drive (SSD), a Read Only Memory (ROM), etc., and the memory includes a buffer, a RAM (Random Access Memory), etc.

The electronic device (100) may include a microphone (150). The microphone (150) collects sounds from the external environment, including the user's voice. The microphone (150) transmits signals of the collected sounds to the processor (170). The electronic device (100) may be equipped with a microphone (150) that collects the user's voice, or may receive a voice signal from an external device, such as a remote controller or a smartphone, having a microphone through an interface unit (110). A remote controller application may be installed in the external device to control the electronic device (100) or perform functions such as voice recognition. In the case of an external device on which such an application is installed, the user's voice may be received, and the external device may transmit and receive data and control the electronic device (100) using Wi-Fi/BT or infrared, etc., and thus a plurality of interface units (110) that may implement the above communication method may exist in the electronic device (100).

The electronic device (100) may include a speaker (160). The speaker (160) outputs audio data processed by the processor (170) as sound. The speaker (160) includes a unit speaker provided to correspond to audio data of a certain audio channel, and may include a plurality of unit speakers respectively to correspond to audio data of a plurality of audio channels. As another embodiment, the speaker (160) may be provided separately from the electronic device (100), and in this case, the electronic device (100) may transmit audio data to the speaker (160) through the interface unit (110).

The electronic device (100) may include a processor (170). The processor (170) includes one or more hardware processors implemented as a CPU, chipset, buffer, circuit, etc. mounted on a printed circuit board, and may be implemented as a SOC (system on chip) depending on the design method. When the electronic device (100) is implemented as a display device, the processor (170) includes modules corresponding to various processes such as a demultiplexer, a decoder, a scaler, an audio DSP (Digital Signal Processor), an amplifier, etc. Here, some or all of these modules may be implemented as an SOC. For example, modules related to image processing such as a demultiplexer, a decoder, and a scaler may be implemented as an image processing SOC, and the audio DSP may be implemented as a chipset separate from the SOC.

When the processor (170) obtains a voice signal for a user's voice by a microphone (150), etc., the processor (170) can convert the voice signal into voice data. At this time, the voice data may be text data obtained through a STT (Speech-to-Text) processing process that converts the voice signal into text data. The processor (170) identifies a command indicated by the voice data and performs an operation according to the identified command. The voice data processing process and the command identification and execution process may both be executed in the electronic device (100). However, in this case, since the system load and required storage capacity required for the electronic device (100) become relatively large, at least a part of the process may be performed by at least one server that is communicatively connected to the electronic device (100) via a network.

The processor (170) according to the present invention can call at least one command among the commands of software stored in a storage medium that can be read by a machine (machine), such as an electronic device (100), and execute it. This enables the machine, such as the electronic device (100), to operate to perform at least one function according to the at least one command called. The one or more commands may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, ‘non-transitory’ only means that the storage medium is a tangible device and does not include a signal (e.g., electromagnetic wave), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

Meanwhile, the processor (170) divides an input image into a plurality of frame sets having a first number of frames, divides the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames, identifies a predefined keyframe among the frames included in each subframe set for each of the plurality of subframe sets, and identifies an area within the image in which a luminance change of the frame set is less than a threshold based on a luminance change between frames using the identified keyframe. At least a portion of the data analysis, processing, and result information generation may be performed using at least one of a machine learning, a neural network, or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

For example, the processor (170) can perform the functions of the learning unit and the recognition unit together. The learning unit can perform the function of generating a learned neural network, and the recognition unit can perform the function of recognizing (or inferring, predicting, estimating, or judging) data using the learned neural network. The learning unit can generate or update a neural network. The learning unit can obtain learning data to generate a neural network. For example, the learning unit can obtain learning data from the storage unit (140) or the outside. The learning data can be data used for learning a neural network, and data on which the above-described operation is performed can be used as learning data to train a neural network.

Before training a neural network using training data, the learning unit may perform preprocessing on the acquired training data, or select data to be used for training from among a plurality of training data. For example, the learning unit may process the training data into a preset format, filter it, or add/remove noise to process it into a form of data suitable for training. The learning unit may generate a neural network set to perform the above-described operation using the preprocessed training data.

The learned neural network may be composed of multiple neural networks (or layers). Nodes of the multiple neural networks have weights, and the multiple neural networks may be connected to each other so that the output value of one neural network is used as the input value of another neural network. Examples of the neural network may include models such as a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), and deep Q-networks.

Meanwhile, the recognition unit may obtain target data to perform the above operation. The target data may be obtained from the storage unit (140) or from the outside. The target data may be data to be recognized by the neural network. The recognition unit may perform a preprocessing operation on the obtained target data before applying the target data to the trained neural network, or select data to be used for recognition from among a plurality of target data. For example, the recognition unit may process the target data into a preset format, filter it, or add/remove noise to process it into a form of data suitable for recognition. The recognition unit may obtain an output value output from the neural network by applying the preprocessed target data to the neural network. The recognition unit may obtain a probability value or a reliability value together with the output value.

For example, the control method of the electronic device (100) according to the present invention may be provided as included in a computer program product. The computer program product may include instructions of software executed by the processor (170) described above. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a CD-ROM), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play StoreTM) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or a relay server.

FIG. 2 is a block diagram illustrating an operation flow diagram of an electronic device according to one embodiment of the present invention.

The processor (170) can divide the input image into a plurality of frame sets each having a first number of frames (S210).

An image to be displayed on a display may be composed of multiple frames, and each frame may be composed of multiple pixels. If the electronic device (100) is a TV and the image displayed on the TV is, for example, a broadcast program, the program name or the name of the broadcaster may be displayed at the top of the display. Since the name of the broadcast program or the name of the broadcaster (hereinafter referred to as a logo) is continuously displayed in a fixed area while the broadcast continues, the pixels existing in the fixed area may deteriorate in function, such as degraded, compared to an area where the luminance of the pixels continuously changes. Alternatively, if the electronic device is a smartphone, the logo of the telecommunications company, the signal strength indicator, the remaining battery level, etc. are always displayed in the same position at the top of the smartphone. For this reason, when playing content that uses the entire screen, such as a video, the top part of the smartphone may appear to be stained due to degraded pixels. The following is a solution to solve this problem.

Accordingly, the processor (170) can determine a frame set that groups a plurality of frames existing in a section where pixels are expected to be fixed to a luminance value within a certain range among a plurality of frames constituting an image, that is, a section where a logo is expected to be displayed in the image. According to one embodiment of the present invention, the processor (170) can divide an input image into a plurality of frame sets having a first number of frames. Since an area corresponding to a deterioration target such as a logo (hereinafter referred to as a logo area) is not fixed on the screen and deterioration prevention is continuously required, the processor (170) can identify the logo area based on a plurality of frame sets. The longer the frame set, the more accurately the logo area can be selected by considering frame luminance changes in many areas within the frame. However, since the length of the frame set and the time for selecting the logo area are in a trade-off relationship, the two can be compared to determine an area of an appropriate length.

The processor (170) can divide the first number of frames included in the frame set into a plurality of subframe sets having a second number of frames (S220). The subframe set is a measure of how often the luminance change of the image will be updated, and is obtained by dividing the plurality of frames included in the frame set into the second number of frames. However, the first number and the second number, which are the numbers of frames of the frame set and the subframe set, need not be limited to a certain number.

The processor (170) can identify a predefined key frame among the frames included in each subframe set for each of the plurality of subframe sets (S230). The key frame is used as a criterion for finding a logo area in an image, and the key frame can be identified for each subframe set. The key frame can contribute to more accurately detecting the logo area. This will be described in detail later.

The processor (170) can identify an area within an image where the luminance change of a set of frames is less than a threshold value based on the luminance change between frames using the identified key frames (S240).

The processor (170) can accumulate luminance changes between each subframe set in the image on a pixel basis using keyframes, and use this to calculate luminance changes in the frame set. At this time, since the largest value among each RGB has the greatest influence on pixel deterioration, the largest value of the R, G, and B channels can be selected and used as luminance, but is not limited thereto. If the calculated luminance change value of the frame set does not exceed a threshold, the processor (170) can determine that there is no movement in the image, and identify an area where the luminance change is lower than the threshold as a logo area.

According to one embodiment of the present invention, unnecessary luminance fluctuations can be reduced by using key frames, thereby enabling more accurate identification of an area in which a logo is fixedly displayed in an image.

FIG. 3 is a diagram illustrating frames processed by an electronic device according to an embodiment of the present invention. The present invention proposes an algorithm for selecting a logo area by configuring a plurality of frames in a time pyramid shape, as illustrated in FIG. 3. According to FIG. 3, there is a frame set (310) composed of a first number of frames, and the processor (170) can divide the frame set (310) into a plurality of subframe sets (320), such as subframe set 1, subframe set 2, ..., subframe set N, which are composed of a second number of frames. Each subframe set (320) is composed of frames including a keyframe (330).

In the present invention, the processor (170) configures a frame set (310) in the form of a time pyramid, and can identify a key frame (330) for each subframe set (320). The reason for using the key frame (330) in identifying the logo area is significant as follows. When compression of an image occurs due to a broadcast environment, etc., a fluctuation may occur in the luminance of the logo area within the image. If the luminance change between the previous frame and the current frame is simply calculated due to this fluctuation, the luminance change that changes slightly is accumulated despite being a logo area, so that it cannot be detected as a logo area. When a key frame (330) is placed, the processor (170) calculates and accumulates the luminance change between the key frame (330) and the next frame within each subframe set (320). By using the key frame (330) to set a standard for luminance change, the luminance difference due to the subtle luminance change in the logo area can be reduced, contributing to more accurate detection of the logo area. For example, a key frame (330) may be the first frame of a set of subframes (320) that includes the key frame (330), but is not limited thereto.

In addition, in order to select a stable logo area when the logo is constant in the video, the processor (170) sets a frame set (310) in the form of a time pyramid and then uses this to stably guarantee the logo area while excluding a background other than the logo where brightness changes occur.

Therefore, according to one embodiment of the present invention, even when noise or loss of the logo area due to compression occurs in the logo area, the logo area can be stably selected.

FIG. 4 is a diagram illustrating frames processed by an electronic device according to an embodiment of the present invention. For example, if an image has 30 fps, that is, 30 images (frames) per second, and only the luminance change between two adjacent frames is considered, it is difficult to find a pixel corresponding to a logo area because most of them have similar luminance values except when the scene is switched. Therefore, since an algorithm must be designed by accumulating luminance changes of multiple frames, this diagram examines in detail a method of calculating luminance changes using key frames.

FIG. 4 illustrates a frame set (310) in the form of a time pyramid as described above and a plurality of subframe sets (320) divided therein. The subframe set (320) is again composed of frames, and among them, there is a keyframe (330) which is the very first frame. The processor (170) can identify a luminance change of the frame set (310) based on a luminance change between a keyframe (330) included in the subframe set (320) and other frames. For example, the processor (170) can calculate a luminance change between keyframe 1 (330) included in subframe set 1 (320) and other frames (410, 420, 430, ...), and calculate this for each subframe set (320) to calculate a luminance change of the entire frame set (310).

However, as shown in this drawing, keyframe 1 (330) does not need to calculate the luminance change between all frames included in subframe set 1 (320), but can calculate it only for some frames, and there is no limitation on the method of selecting those frames.

According to another embodiment of the present invention, the processor (170) can identify luminance changes in a set of frames based on luminance changes between a key frame and frames following the key frame. In this case, the key frame and the frames following the key frame do not need to belong to the same subframe set (320).

In the present invention, rather than utilizing the luminance change between two consecutive frames, a frame in the shape of a time pyramid is configured and a stable logo area is ensured by utilizing a key frame.

FIG. 5 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention. This diagram relates to another method of calculating a luminance change using a key frame. In this embodiment, as described above, the frame set (310) is configured in the form of a time pyramid.

The processor (170) can identify the luminance change of the frame set (310) based on the luminance change between key frames (330, 520, ...). If only the luminance change between consecutive frames is used to select the logo area, if the bright background continues in the image, the luminance change of the bright background part in the frame set is constant, so a problem may occur in recognizing it as the logo part. In addition, when the logo is constant in the image, it is important to select a stable logo area, and the logo area shows a large difference depending on the luminance change value of the frame set. Therefore, a key frame can be selected for each subframe set divided into a certain number of frames in the frame set, and the luminance change between the selected key frames can be calculated on a pixel basis. At this time, the number of frames divided into the subframe set may be related to the frequency of occurrence of errors such as luminance change due to image compression, etc. For example, if an error occurs once every certain number of frames in the image, the number of frames included in the subframe set can be appropriately divided so that the key frame, which is the standard for calculating the luminance change, does not correspond to this error cycle. However, since errors may not occur at regular intervals and may be difficult to predict, at least some of the data analysis, processing, and resulting information generation to identify them may be performed using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm.

According to one embodiment of the present invention, a logo area can be accurately and stably identified even when a change in brightness is below a threshold value despite it not being a logo area, when there is an error in the image, etc.

FIG. 6 is a diagram illustrating frames processed by an electronic device according to an embodiment of the present invention. In this diagram, a method is proposed to select a logo area using one frame set and then update a logo area that changes according to subsequent frames. The processor (170) can select a logo area using the first frame set (310). In addition, the processor (170) can identify an area in an image based on a luminance change of a second frame set (610) in a section moved by a predefined number of subframe sets from the section of the first frame set (310). For example, the processor (170) can recalculate the total luminance change for finding a logo area by subtracting the luminance change from the first subframe set of the first frame set (310) for each new subframe set displayed after the first frame set (310) and adding the luminance difference of the new subframe set. Referring to FIG. 6, the first frame set (310) moves by two subframe sets to form the second frame set (610). Accordingly, the processor (170) can obtain the luminance change of the second frame set (610) by subtracting the luminance changes of the subframe set 1 and the subframe set 2 from the calculated luminance value of the first frame set (310) and adding the luminance changes of the two newly added subframe sets. Thereafter, the processor (170) can update the area where the recalculated luminance change does not exceed the threshold value as a new logo area. This enables detection of the logo area when there is a change in the logo area within the image.

The processor (170) can identify an area within an image based on the luminance change of the first frame set (310), and identify a change in the area based on the luminance change of the second frame set (610) based on the identified area rather than the entire area. Since the change in the logo area is identified based on the area identified through the first frame set (310), the speed of processing can be increased and the efficiency can be improved.

And, the processor (170) can identify an area within the image based on calculations between luminance changes of the first frame set (310) and the second frame set (610). After identifying an area within the image through the first frame set (310), the processor (170) can identify an area within the image by comprehensively considering luminance changes of the first frame set (310) and the second sub-frame set (610) shifted by a predefined number of sub-frame sets. Through this, the identified area can be distinguished more precisely.

According to one embodiment of the present invention, after identifying a logo area, it is possible to continuously reflect changes in the logo area, such as movement of the logo area over time, so that deterioration of pixels according to the logo area can be continuously prevented, and processing speed and efficiency can be increased.

FIG. 7 is a diagram illustrating a frame processed by an electronic device according to one embodiment of the present invention. In this diagram, a set of upper subframes is divided into a plurality of lower subframe sets to calculate a change in brightness.

As illustrated in FIG. 7, the processor (170) may divide a second number of frames included in one first sub-frame set (710) among a plurality of first sub-frame sets into a plurality of second sub-frame sets having a third number of frames. In addition, the processor (170) may identify a predefined second key frame (730) among the frames included in each second sub-frame set (720) among the plurality of second sub-frame sets (720), and may identify an area within the image based on a luminance change between frames using the identified second key frame (730). For example, the processor (170) may calculate a luminance change of the second sub-frame set 1 through an operation between the frames included in the second sub-frame set 1 based on the second key frame 1, and this operation may be performed for each second sub-frame set.

In the second sub-frame set (720) set in FIG. 7, if the luminance change does not exceed the threshold value through the luminance change using the second key frame (730), the logo area can be identified through this, and the logo area can be selected by using the luminance difference of a certain second sub-frame set (720) rather than the entire frame set (310). Accordingly, a shorter frame can be considered rather than using the luminance change of the entire frame set (310), and thereafter, the processor (170) can filter out the part where the luminance changes, such as the background part, through the operation of each second sub-frame set (720), so that only the desired logo area can be selected. However, as in the previous embodiment, the second number and the third number, which are the numbers of frames of the first sub-frame set and the second sub-frame set, need not be limited to a certain number.

According to one embodiment of the present invention, by configuring a frame in the form of a time pyramid, it is more effective in removing noise than when using only a set of subframes at one stage, and has a robust property in detecting a logo area.

Additionally, the logo and logo area mentioned throughout this specification are only one example of where pixel degradation may occur, and the present invention is not limited thereto, and may be applied to all embodiments that cause pixel performance degradation by continuously having a luminance change below a threshold value.

100: Electronic devices
110: Interface section
120: Display section
130: User Input
140: Storage
150: Microphone
160: Speaker
170: Processor

Claims (20)

In electronic devices,
Display and;
Divide the input video into multiple frame sets each having a first number of frames,
Dividing a first number of frames included in the above frame set into a plurality of subframe sets having a second number of frames,
Identifying a predefined key frame among the frames included in each subframe set for each of the above multiple subframe sets,
Based on the luminance change between the frames using the identified key frames, an area within the image where the luminance change of the frame set is less than a threshold value is identified,
An electronic device comprising a processor that moves the identified area at predetermined time intervals. In the first paragraph,
The above processor,
An electronic device that identifies a luminance change in the frame set based on a luminance change between the key frame included in the subframe set and the other frames. In the second paragraph,
The above processor,
An electronic device that identifies luminance changes in a set of frames based on sequential luminance changes between said keyframe and said frame subsequent to said keyframe. In the first paragraph,
The above keyframe is an electronic device that is the first frame of the set of subframes that includes the above keyframe. In the first paragraph,
The above processor,
An electronic device that identifies luminance changes in a set of frames based on luminance changes between said key frames. delete In the first paragraph,
The above frame set is the first frame set,
The above processor,
An electronic device that identifies an area within the image based on a change in luminance of a second frame set in an area shifted by a predefined number of subframe sets from an area of the first frame set. In Article 7,
The above processor,
Identifying an area within the image based on the luminance change of the first frame set,
An electronic device that identifies a change in the area based on a change in luminance of the second frame set relative to the identified area. delete In the first paragraph,
The above subframe set is the first subframe set, the above keyframe is the first keyframe,
The above processor,
Dividing the second number of frames included in the above subframe set into a plurality of second subframe sets having a third number of frames,
Identifying a predefined second key frame among the frames included in each second subframe set for each of the above plurality of second subframe sets,
An electronic device that identifies an area within an image based on luminance changes between frames using the identified second key frame. In a method for controlling an electronic device,
A step of dividing an input image into a plurality of frame sets each having a first number of frames;
A step of dividing a first number of frames included in the above frame set into a plurality of subframe sets having a second number of frames;
A step of identifying a predefined keyframe among the frames included in each subframe set for each of the plurality of subframe sets;
A step of identifying an area within the image where the luminance change of the frame set is less than a threshold value based on the luminance change between the frames using the identified key frames; and
A method for controlling an electronic device, comprising the step of moving the identified area at a predetermined time cycle. delete delete delete delete delete delete delete delete A computer program including a code that performs a method of controlling an electronic device as a computer-readable code is stored on a recording medium, wherein the method of controlling the electronic device is:
A step of dividing an input image into a plurality of frame sets each having a first number of frames;
A step of dividing a first number of frames included in the above frame set into a plurality of subframe sets having a second number of frames;
A step of identifying a predefined keyframe among the frames included in each subframe set for each of the plurality of subframe sets;
A step of identifying an area within the image where the luminance change of the frame set is less than a threshold value based on the luminance change between the frames using the identified key frames; and
A computer-readable recording medium having recorded thereon a computer-readable program, characterized by including a step of moving the identified area at a predetermined time cycle.

Images