WO2026023977A1 - Display device and display method using same - Google Patents

Abstract

A display device according to embodiments of the present disclosure may comprise: a display; a memory storing a program including at least one instruction; and at least one processor which is connected to the memory and executes the at least one instruction of the program stored in the memory. The at least one processor may apply a virtual aim point to an input image, set a region of interest including the virtual aim point, adjust the size or effect of a first image corresponding to the region of interest, generate a second image by combining the input image and the first image, and display the second image through the display.

Description

Display device and display method using the same

Various embodiments of the present disclosure relate to a display device, and more particularly, to a display device for setting a virtual aim point and a display method using the same.

With the advancement of digital technology, various types of electronic devices, such as smart TVs, smartphones, tablet PCs, electronic notebooks, personal digital assistants (PDAs), and wearable devices, are being used. In particular, these various types of electronic devices can be implemented as display devices that output images to a display panel based on input data.

Display technology is rapidly advancing, focusing on high resolution, high image quality, and ultra-fast processing speeds. With the commercialization of 4K and 8K resolution displays, display devices are capable of delivering sharper and more vivid images. For example, display devices can deliver high resolution and/or high quality images even in real-time video content such as first-person shooter (FPS) games.

Meanwhile, in video content like FPS games, accurately aiming and shooting at targets is crucial. Accordingly, a virtual aim point can be provided to facilitate easy targeting of in-video targets in such games. Users can use the virtual aim point to improve aiming accuracy and become more immersed in the gameplay.

Various embodiments of the present disclosure can provide a display device and a display method using the same that increase immersion in image content by applying a virtual target point within an input image and adjusting the size or effect of an area of interest including the virtual target point.

A display device according to embodiments of the present disclosure may include a display, a memory storing a program including at least one command, and at least one processor connected to the memory and executing at least one command of the program stored in the memory. The processor may apply a virtual aim point to an input image, set a region of interest including the virtual aim point, adjust a size or effect of a first image corresponding to the region of interest, synthesize the input image and the first image to generate a second image, and display the second image through the display.

In one embodiment, the at least one processor may cause at least one of: zooming in, zooming out, sharpening, blurring, contrast adjustment, color adjustment, or applying a distortion filter to the first image.

In one embodiment, the region of interest has a shape centered around the virtual aiming point and may include part or all of the target.

In one embodiment, the at least one processor can change the position of the virtual aiming point based on a change in the position of the target within the input image, and change the region of interest in response to the change in the position of the virtual aiming point.

In one embodiment, the at least one processor may identify at least one object of the input image using a deep learning model, determine a motion of the at least one object using a motion tracking algorithm, obtain a background characteristic including at least one of brightness, color, or contrast of the input image, and apply the virtual aiming point based on at least one of the position of the at least one object, the motion of the at least one object, or the background characteristic.

In one embodiment, the at least one processor can determine a target based on the importance of at least one object included in the input image, and determine a location of the virtual aiming point within the input image based on coordinates of the target.

In one embodiment, the at least one processor can detect at least one of a movement direction or a movement speed of the input image, and change the size or the effect of the first image based on at least one of the movement direction or the movement speed.

In one embodiment, the at least one processor can change the size of the first image based on the movement speed of the input image as it moves forward or backward.

In one embodiment, the at least one processor can change the effect of the first image based on the movement speed of the input image based on whether the input image moves in a horizontal or vertical direction.

In one embodiment, the at least one processor can display the first image in a non-display area.

In one embodiment, the at least one processor can display the first image in a picture-in-picture (pip) manner.

In one embodiment, the at least one processor can detect an embedded virtual aim point included in the input image, and based on the detection of the embedded virtual aim point, disable application of the virtual aim point to the input image.

In one embodiment, the at least one processor can set the region of interest including the built-in virtual aiming point and adjust the size or effect of a third image corresponding to the region of interest.

In one embodiment, the at least one processor receives a control signal from a control device, determines whether the control signal is a first control signal unrelated to adjustment of the size or the effect of the first image corresponding to the region of interest, or a second control signal that adjusts the size or the effect of the first image, and transmits the first control signal to a server based on the control signal being the first control signal, and adjusts the size or the effect of the first image based on the control signal being the second control signal.

A display method according to embodiments of the present disclosure may include an operation of applying a virtual aim point to an input image, an operation of setting a region of interest including the virtual aim point, an operation of adjusting a size or effect of a first image corresponding to the region of interest, an operation of generating a second image by synthesizing the input image and the first image, and an operation of displaying the second image through a display.

In one embodiment, the operation of applying the virtual aiming point may include an operation of identifying at least one object of the input image using a deep learning model, an operation of determining a motion of the at least one object using a motion tracking algorithm, an operation of obtaining a background characteristic including at least one of brightness, color, and contrast of the input image, and an operation of applying the virtual aiming point based on at least one of the position of the at least one object, the motion of the at least one object, or the background characteristic.

In one embodiment, the operation of applying a virtual aiming point to the input image may include an operation of determining a target based on the importance of at least one object included in the input image, and an operation of determining a location of the virtual aiming point within the input image based on coordinates of the target.

In one embodiment, the region of interest has a shape centered around the virtual aiming point and may include part or all of the target.

In one embodiment, the act of adjusting the size or effect of the first image may include performing at least one of enlarging, reducing, sharpening, blurring, adjusting contrast, adjusting color, or applying a distortion filter to the first image.

In one embodiment, the display method may further include an operation of detecting an embedded virtual aim point included in the input image, an operation of deactivating application of the virtual aim point to the input image based on the detection of the embedded virtual aim point, an operation of setting the region of interest including the embedded virtual aim point, and an operation of adjusting the size or effect of a third image corresponding to the region of interest.

According to various embodiments of the present disclosure, the display device and display method using the same can apply a virtual aim point to an input image and display an image in which the size or effect of a region of interest centered on the virtual aim point is adjusted. Accordingly, the display device and display method using the same of the present disclosure can enhance the sense of immersion and user satisfaction with video content by increasing the aiming accuracy of game play.

Furthermore, the display device of the present disclosure and the display method using the same detect a virtual aiming point embedded in an input image, thereby preventing the redundant application of the virtual aiming point, and can change the size or effect of the adjusted image based on the movement direction or movement speed of the input image. Therefore, the display device of the present disclosure and the display method using the same can provide a flexible screen configuration suitable for the display environment and provide image content optimized for game play in real time.

The effects that can be obtained from the exemplary embodiments of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure pertain from the following description. In other words, unintended effects resulting from implementing the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

FIG. 1 is a drawing showing a display device according to one embodiment of the present disclosure.

FIG. 2 is a block diagram showing the configuration of a display device according to one embodiment of the present disclosure.

FIG. 3 is a flowchart illustrating the operation of a display device according to one embodiment of the present disclosure.

FIG. 4 is an exemplary diagram showing an input image of a display device according to one embodiment of the present disclosure.

FIG. 5 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure to apply a virtual aiming point to an input image.

FIG. 6 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure analyzing image characteristics of an input image and applying a virtual aiming point based on the analysis.

FIG. 7 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure to set a region of interest.

FIG. 8 is a diagram illustrating a change in an area of interest due to movement of a virtual aiming point according to one embodiment of the present disclosure.

FIGS. 9A to 9C are exemplary diagrams showing an operation of a display device according to one embodiment of the present disclosure to adjust the size or effect of a first image.

FIGS. 10A and 10B are exemplary diagrams showing a second image generated by a display device according to one embodiment of the present disclosure.

FIG. 11 is a flowchart illustrating an operation of a display device according to one embodiment of the present disclosure to change the size or effect of a first image.

FIG. 12 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure to change the size of a first image according to a moving direction and a moving speed.

FIG. 13 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure to change the effect of a first image according to a moving direction and a moving speed.

FIGS. 14A to 14C are diagrams illustrating various ways in which a display device according to one embodiment of the present disclosure displays a first image.

FIG. 15 is a flowchart illustrating the operation of a display device according to one embodiment of the present disclosure when an embedded virtual aiming point exists in an input image.

FIG. 16 is an exemplary diagram showing an input image of a display device according to one embodiment of the present disclosure.

FIG. 17 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure to adjust the size or effect of a third image.

FIG. 18 is a flowchart showing the operation of a display device according to a type of control signal according to one embodiment of the present disclosure.

FIGS. 19a and 19b are diagrams showing operations performed according to the type of control signal of a display device according to one embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions.

Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

Here, the term '~ part' used in this embodiment means software or hardware components such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be on an addressable storage medium or may be configured to reproduce one or more packet processing devices. Therefore, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. In addition, the components and '~parts' may be implemented to play one or more central processing units (CPUs) within the device or secure multimedia card. In addition, in the embodiment, the '~parts' may include one or more packet processing devices.

FIG. 1 is a drawing showing a display device (100) according to one embodiment of the present disclosure.

Referring to FIG. 1, in one embodiment, the display device (100) may display an image based on input data. For example, the display device (100) may be a TV that displays an image, but this is only one example of the display device (100), and the display device (100) may be implemented as an electronic device including at least one display.

For example, the display device (100) may be, but is not limited to, a smartphone, a tablet PC, a PC, a smart TV, a mobile phone, a PDA (personal digital assistant), a laptop, a media player, a micro server (2), a digital broadcasting terminal, a navigation system, a kiosk, a home appliance, and other mobile or non-mobile computing devices. In addition, the display device (100) may be fixed or mobile, and may be a digital broadcasting receiver capable of receiving digital broadcasting. In addition, the display device (100) may be a wearable terminal such as a watch or glasses that can perform various computing functions such as real-time video viewing and communication. The display device (100) may be a terminal in various forms without being limited to the above-described contents.

For example, the display device (100) may be implemented as a flat display device (100), a curved display device (100) having a screen with a curvature, or a flexible display device (100) whose curvature can be adjusted. For example, the output resolution of the display device (100) may include HD (High Definition), Full HD, Ultra HD, or a resolution clearer than Ultra HD.

The display device (100) of the present disclosure can communicate with an external server (2) via a network. The display device (100) can receive a video signal from the server (2). For example, the display device (100) can receive a video signal from the server (2) and display an image based on the video signal. The display device (100) can transmit a control signal to the server (2). For example, the display device (100) can receive a user input and transmit a control signal based on the user input to the server (2).

Here, the network can include both wired and wireless networks. Wired networks include cable networks or telephone networks, while wireless networks can include any network that transmits and receives signals via radio waves. Wired and wireless networks can be interconnected.

For example, the network may include a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an access point (AP), and a short-range wireless network that does not use an access point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth™ (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

The display of the present disclosure can apply a virtual aim point to an input image and display an image with the size or effect of an area of interest centered around the virtual aim point adjusted. Therefore, the display device (100) of the present disclosure can enhance the immersion and user satisfaction with video content by increasing the aiming accuracy of game play.

In addition, the display device (100) of the present disclosure can detect a virtual aiming point embedded in an input image, thereby preventing the redundant application of the virtual aiming point, and can change the size or effect of the adjusted image based on the movement direction or movement speed of the input image. Accordingly, the display device (100) of the present disclosure can provide a flexible screen configuration suitable for the display environment and provide image content optimized for game play in real time.

FIG. 2 is a block diagram showing the configuration of a display device (200) according to one embodiment of the present disclosure.

The display device (200) of FIG. 2 may be the display device (100) of FIG. 1.

According to one example, the memory (220) is a storage medium used by the display device (200) and can store data such as at least one command (221) or setting information corresponding to at least one program. The program may include an operating system (OS) program and various application programs.

In one example, the memory (220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable ROM (EEPROM), a programmable ROM (PROM), a magnetic memory, a magnetic disk, and an optical disk.

According to one example, the video input unit (230) can receive viewing video and viewing video information through a tuner (not shown), an input/output unit (not shown), or a communication unit (250). The video input unit (230) can include at least one of the tuner and the input/output unit. The tuner can select and tune only the frequency of a broadcast channel to be received by the display device (200) among many radio wave components through amplification, mixing, resonance, etc. of a broadcast signal received wirelessly or wiredly. The broadcast signal can include video, audio, and additional data (e.g., EPG (Electronic Program Guide)). The tuner can receive broadcast channels (or viewing video) from various broadcast sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, and Internet broadcasting. The tuner can be implemented as an integrated unit with the display device (200) or as a separate tuner electrically connected to the display device (200). The above input/output unit may include at least one of an HDMI (High Definition Multimedia Interface) input port, a component input jack, a PC input port, and a USB input jack, which can receive viewing images and viewing image information from an external device of the display device (200) under the control of the processor (210). It is obvious to those skilled in the art that the above input/output unit may be added, deleted, and/or changed depending on the performance and structure of the display device (200).

According to one example, the display (240) can perform functions for outputting information in the form of numbers, characters, images, and/or graphics. The display (240) can include at least one hardware module for outputting. The at least one hardware module can include, for example, at least one of a Liquid Crystal Display (LCD), a Light Emitting Diode (LED), a Light Emitting Polymer Display (LPD), an Organic Light Emitting Diode (OLED), an Active Matrix Organic Light Emitting Diode (AMOLED), or a Flexible LED (FLED). The display (240) can display a screen corresponding to data received from the processor (210). The display (240) may be referred to as an 'output unit', a 'display unit', or other terms having an equivalent technical meaning thereto.

According to an example, the communication unit (250) may provide a wired/wireless communication interface that enables communication with an external device. The communication unit (250) may include at least one of a wired Ethernet, a wireless LAN communication unit, and a short-range communication unit. The wireless LAN communication unit may include, for example, Wi-Fi, and may support the wireless LAN standard (IEEE802.11x) of the Institute of Electrical and Electronics Engineers (IEEE). The wireless LAN communication unit may be wirelessly connected to an AP (Access Point) under the control of the processor (210). The short-range communication unit may wirelessly perform short-range communication with an external device under the control of the processor (210). Short-range communication may include Bluetooth, Bluetooth Low Energy, infrared communication (IrDA: Infrared Data Association), UWB (Ultra WideBand), WiFi Direct, and NFC (Near Field Communication). The above external devices may include server devices that provide video services, etc., and mobile terminals (e.g., smartphones, tablets, etc.).

According to one example, the processor (210) may include a processing circuit. The processor (210) may execute at least one instruction (221) stored in the memory (220), thereby executing calculations or data processing related to control and/or communication of at least one other component of the display device (200). The processor (210) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have multiple cores.

In one example, the processor (210) may apply a virtual aim point to an input image. The processor (210) may set a region of interest including the virtual aim point. The processor (210) may adjust the size or effect of a first image corresponding to the region of interest. The processor (210) may generate a second image by synthesizing the input image and the first image. The processor (210) may display the second image through the display.

FIG. 3 is a flowchart showing the operation of a display device (100) according to one embodiment of the present disclosure.

Referring to FIG. 3, the display device (100) can increase immersion in video content by applying a virtual aiming point (VAP) to an input image and adjusting the size or effect of a region of interest (ROI) including the virtual aiming point (VAP). In one example, the display device (100) can apply a virtual aiming point (VAP) to an input image (operation 310), set a region of interest (ROI) including the virtual aiming point (VAP) (operation 320), adjust the size or effect of a first image (Image1) corresponding to the region of interest (ROI) (operation 330), synthesize the input image and the first image (Image1) to generate a second image (Image2) (operation 340), and display the second image (Image2) through the display (operation 350).

In the following example operations, the operations may be performed sequentially, but are not necessarily sequential. For example, the order of the operations may be changed, or at least two operations may be performed in parallel.

As an example, in operation 310, the display device (100) may apply a virtual aiming point (VAP) to the input image. Hereinafter, operation 310 of the display device (100) will be described in detail with reference to FIGS. 4 and 5.

FIG. 4 is an exemplary diagram showing an input image of a display device (100) according to one embodiment of the present disclosure, and FIG. 5 is an exemplary diagram showing an operation of a display device (100) according to one embodiment of the present disclosure applying a virtual aiming point (VAP) to an input image.

Referring to FIG. 4, the input image may include at least one object. For example, the input image may be real-time video content of a First-Person Shooter (FPS) game.

The display device (100) can receive an input image from the server (2) in a real-time streaming manner. For example, the display device (100) can output the input image in a live streaming manner. For example, the display device (100) can output the input image in a video-on-demand (VOD) manner. For example, the display device (100) can output the original image for the input image by decoding the compressed input image using at least one video codec.

Referring to FIG. 5, the display device (100) can determine a target among a plurality of objects and apply a virtual aiming point (VAP) corresponding to the coordinates of the target.

The display device (100) can determine a target based on the importance of at least one object included in the input image. For example, the display device (100) can calculate the importance of multiple objects within the input image. For example, the display device (100) can set priorities for the objects based on the importance of the multiple objects. For example, the display device (100) can determine the object with the highest priority as the target within the game.

The display device (100) can determine the position of the virtual aiming point (VAP) within the input image by calculating the coordinates of the target. For example, the position of the virtual aiming point (VAP) can be determined within a predetermined distance from the coordinates of the target. The display device (100) can display the virtual aiming point (VAP) on the input image in a graphic overlay manner. The display device (100) can adjust at least one of the color, size, or brightness of the virtual aiming point (VAP). For example, the display device (100) can adjust at least one of the color, size, or brightness of the virtual aiming point (VAP) based on the image characteristics of the input image so that a user can easily recognize the virtual aiming point (VAP).

FIG. 6 is an exemplary diagram showing an operation of a display device according to one embodiment of the present disclosure analyzing image characteristics of an input image and applying a virtual aiming point based on the analysis.

In one embodiment, the display device can analyze the characteristics of an input image and apply a virtual aiming point (VAP) based on the characteristics of the input image. For example, as shown in FIG. 6, the display device (100) can analyze the image characteristics of the input image. The display device (100) can extract individual frames from the input image. For example, the display device (100) can analyze the image characteristics of the input image based on the extracted individual frames.

The display device (100) can identify at least one object in the input image using an artificial intelligence model (e.g., a deep learning model). For example, the display device (100) can identify a key object including at least one enemy character, target, or environmental element in the input image using the deep learning model. For example, the display device (100) can use a deep learning model such as a pre-trained CNN (Convolutional Neural Network) or YOLO (You Only Look Once) for object recognition. The display device (100) can accurately determine an area occupied by at least one object by extracting the boundary of the object. For example, at least one of the plurality of objects may be a target that is an aiming target in a game.

The display device (100) can determine the motion of at least one object using a motion tracking algorithm. For example, the display device (100) can determine the direction and speed of movement of the object by tracking the motion of the object in the input image. The display device (100) can recognize the behavior pattern of the object based on the direction and speed of movement of the object. For example, the display device (100) can use a motion tracking algorithm such as Optical Flow or Kalman Filter. The display device (100) can determine the in-game motion of the object by analyzing the behavior pattern of the at least one object.

The display device (100) can acquire background characteristics including at least one of brightness, color, or contrast of the input image. For example, the display device (100) can calculate the average brightness of the input image and analyze the brightness characteristics of each sub-region of the background. For example, the display device (100) can extract the main colors constituting the background and analyze the color characteristics of the background. For example, the display device (100) can analyze the contrast between the background and an object.

In one embodiment, the display device (100) can apply a virtual aiming point based on at least one of the position of at least one object, the motion of the at least one object, or the background characteristics.

The display device may set a virtual aiming point (VAP) based on a preset value for the input image, as illustrated in FIG. 5, or may set a virtual aiming point (VAP) based on analysis of the input image, as illustrated in FIG. 6.

According to an example, in operation 330, the display device (100) can set a region of interest (ROI) including the virtual aiming point (VAP). Hereinafter, operation 330 of the display device (100) will be described in detail with reference to FIGS. 7 and 8.

FIG. 7 is an exemplary diagram showing an operation of a display device (100) according to one embodiment of the present disclosure to set a region of interest (ROI).

Referring to Figure 7, a region of interest (ROI) is an area of high importance within an input image, which may be determined based on a virtual aiming point (VAP). For example, the region of interest (ROI) may have a predetermined shape centered around the virtual aiming point (VAP) and may include part or all of a target.

The display device (100) can set a region of interest (ROI) having a predetermined size and shape. For example, the size and shape of the region of interest (ROI) can be determined based on a user's settings. For example, the size and shape of the region of interest (ROI) can be automatically determined based on the image characteristics of the input image.

The display device (100) can adjust the size of the region of interest (ROI) based on at least one of the size, importance, or location of the target within the input image. For example, the display device (100) can increase the size of the region of interest (ROI) in proportion to the size of the target. For example, the display device (100) can increase the size of the region of interest (ROI) in proportion to the importance of the target.

The display device (100) can adjust the shape of the region of interest (ROI) based on the shape of the target or the user's settings. For example, the display device (100) can set the region of interest (ROI) to a shape similar to the shape of the target. For example, the display device (100) can set the region of interest (ROI) to a circle based on the user's settings.

FIG. 8 is a diagram illustrating a change in a region of interest (ROI) according to movement of a virtual aiming point (VAP) according to one embodiment of the present disclosure.

Referring to FIG. 8, the display device (100) can change the region of interest (ROI) based on a change in the virtual aiming point (VAP). For example, the display device (100) can change the position of the virtual aiming point (VAP) based on a change in the position of the target within the input image. For example, the display device (100) can change the region of interest (ROI) based on (e.g., in response to) a change in the position of the virtual aiming point (VAP).

In one embodiment, when the position of the virtual aiming point (VAP) changes, the display device (100) can change at least one of the position of the region of interest (ROI), the size of the region of interest (ROI), or the shape of the region of interest (ROI). For example, when the position of a target in an input image changes, a first virtual aiming point (VAP1) corresponding to the position of the target before the change may change to a second virtual aiming point (VAP2) corresponding to the position of the target after the change. For example, when the first virtual aiming point (VAP1) changes to the second virtual aiming point (VAP2), the display device (100) can change the position of the region of interest (ROI) from the position of the first region of interest (ROI1) to the position of the second region of interest (ROI2).

For example, in operation 340, the display device (100) can adjust the size or effect of the first image (Image1) corresponding to the region of interest (ROI). Hereinafter, operation 340 of the display device (100) will be described in detail with reference to FIGS. 9A to 9C.

FIGS. 9A to 9C are exemplary diagrams showing an operation of a display device (100) according to one embodiment of the present disclosure to adjust the size or effect of a first image (Image1).

Referring to FIGS. 9A to 9C, the display device (100) can adjust the size or effect of the first image (Image1) corresponding to the region of interest (ROI). For example, the display device (100) can perform at least one of enlargement, reduction, sharpening, blurring, contrast adjustment, color adjustment, or distortion filter application to the first image (Image1).

As shown in FIG. 9A, the display device (100) can perform at least one of zooming in and zooming out on a first image (e.g., Image1a). The display device (100) can zoom in on an image within a region of interest (ROI) set around a virtual aiming point (VAP) to increase visibility of the target. For example, when aiming at an enemy character in an FPS game, the user can easily recognize details of the enemy using the zoomed-in first image (e.g., Image1b). For example, when a user needs to secure a wider field of view in an FPS game, the user can easily find the enemy character using the zoomed-out first image (Image1).

As shown in FIG. 9B, the display device (100) can perform at least one of blurring, contrast adjustment, or color adjustment on the first image (e.g., Image1a). The display device (100) can apply various image filters to more clearly express the image within the region of interest (ROI). For example, the display device (100) can apply a blurring filter to blur the background outside the region of interest (ROI), thereby focusing the user's attention on the virtual aiming point (VAP). For example, the display device (100) can increase the contrast of the first image (e.g., Image1c) within the region of interest (ROI) through contrast adjustment, thereby making the target more visible. For example, the display device (100) can use a color adjustment filter to emphasize or convert a specific color, thereby making the target within the input image clearly distinguishable from the background.

As shown in FIG. 9c, the display device (100) can apply a distortion filter to the first image (e.g., Image1a). For example, the display device (100) can provide a specific visual effect by applying the distortion filter to a region of interest (ROI) around a virtual aiming point (VAP). For example, the display device (100) can apply a fish-eye lens effect to the first image (e.g., Image1a). In the case of the first image (e.g., Image1d) to which the fish-eye lens effect is applied, as the center of the region of interest (ROI) is enlarged and the periphery is distorted, a target corresponding to the center of the region of interest (ROI) can be further emphasized. For example, the display device (100) can apply a perspective distortion filter to the first image (Image1). When the perspective distortion filter is applied to the first image (Image1), the user's aiming convenience can be increased as the visual depth in a specific direction of the region of interest (ROI) is emphasized.

In one example, in operation 350, the display device (100) can generate a second image (Image2) by synthesizing the input image and the first image (Image1). In one example, in operation 360, the display device (100) can display the second image (Image2) through the display. Hereinafter, operations 350 and 360 of the display device (100) will be described in detail with reference to FIGS. 10a and 10b.

FIGS. 10A and 10B are exemplary diagrams showing a second image (Image2) generated by a display device (100) according to one embodiment of the present disclosure.

Referring to FIGS. 10A and 10B , the display device (100) can generate a second image (Image2) by synthesizing an input image and a first image (Image1), and display the second image (Image2). For example, the display device (100) can determine the position and size of the first image (Image1) to be synthesized on the input image. For example, the display device (100) can synthesize the first image (Image1) on a predetermined area of the input image in a graphic overlay manner. For example, the display device (100) can use an alpha blending technique to smoothly synthesize the first image (Image1) and the input image. The display device (100) can generate a second image (Image2) in which the first image (Image1) is synthesized on the input image.

The display device (100) can display a second image (Image2). For example, the display device (100) can adjust at least one of the resolution, brightness, or contrast of the second image (Image2) obtained by synthesizing the first image (Image1) onto the input image. For example, as shown in FIG. 10a, the second image (Image2a) may be an image obtained by synthesizing the first image (Image1) corresponding to a rectangular region of interest (ROI) onto the input image in an enlarged manner using a graphic overlay. For example, as shown in FIG. 10b, the second image (Image2b) may be an image obtained by synthesizing the first image (Image1) corresponding to a circular region of interest (ROI) onto the input image in an enlarged manner using a graphic overlay.

In one embodiment, the display device (100) can apply a virtual aiming point (VAP) to an input image and display an image with the size or effect of a region of interest (ROI) centered around the virtual aiming point (VAP) adjusted. The display device (100) can enhance the immersion and user satisfaction with video content by increasing the aiming accuracy of game play.

In one embodiment, the display device (100) detects a virtual aiming point (VAP) embedded in an input image, thereby preventing the redundant application of the virtual aiming point (VAP) and changing the size or effect of the adjusted image based on the movement direction or movement speed of the input image. The display device (100) can provide a flexible screen configuration suitable for the display environment and provide image content optimized for game play in real time.

FIG. 11 is a flowchart showing an operation of changing the size or effect of a first image (Image1) by a display device (100) according to one embodiment of the present disclosure.

Referring to FIG. 11, the display device (100) can detect at least one of a movement direction and a movement speed of the input image (operation 1110), and change the size or the effect of the first image (Image1) based on at least one of the movement direction and the movement speed (operation 1120).

For example, in operation 1110, the display device (100) can detect at least one of the movement direction and movement speed of the input image. The display device (100) can extract consecutive frames and recognize screen changes for each frame. For example, the display device (100) can identify the location and shape of a key object in each frame.

The display device (100) can determine the direction of movement of an input image by analyzing the direction of movement of an object. For example, the display device (100) can determine the direction of movement of the input image as forward or backward. For example, the display device (100) can determine the direction of movement of the input image as horizontal or vertical.

The display device (100) can calculate the movement distance by calculating the change in the position of an object in consecutive frames. For example, the display device (100) can calculate the movement speed of an input image based on the movement distance per unit time (e.g., number of pixels).

For example, in operation 1120, the display device (100) may change (or adjust) the size or the effect of the first image (Image1) based on at least one of the movement direction and the movement speed. The display device (100) may cause the size or effect of the first image to be changed (or adjusted) based on the movement speed of the input image based on whether the input image moves forward or backward. The display device (100) may cause the size or effect of the first image to be changed (or adjusted) based on the movement speed based on whether the input image moves in a horizontal direction or a vertical direction. Hereinafter, operation 1110 of the display device (100) will be exemplarily described with reference to FIGS. 12 and 13.

FIG. 12 is an exemplary diagram showing an operation of a display device (100) according to one embodiment of the present disclosure to change the size of a first image (Image1) according to a movement direction and movement speed.

Referring to FIG. 12, the display device (100) can change the size of the first image (Image1) based on the movement speed of the input image when the input image moves forward or backward. For example, the movement direction of the input image may be the Z-axis direction (e.g., D1).

For example, when the input image moves forward (e.g., D1), the display device (100) can reduce the size of the first image (Image1) because the distance from the target becomes closer. When the input image moves forward (e.g., D1), the display device (100) can change the enlarged first image (Image1) into the original image. For example, when the input image moves backward, the display device (100) can enlarge the size of the first image (Image1) because the distance from the target becomes farther. For example, the display device (100) can reduce or enlarge the size of the first image (Image1) based on (e.g., proportionally) the movement speed.

FIG. 13 is an exemplary diagram showing an operation of a display device (100) according to one embodiment of the present disclosure to change the effect of a first image (Image1) according to a movement direction and movement speed.

Referring to FIG. 13, the display device (100) can adjust the effect of the first image (e.g., Image1e) when the input image moves in the horizontal or vertical direction. For example, the movement direction of the input image can be the X-axis direction (e.g., D2) or the Y-axis direction.

For example, when an input image moves in a horizontal or vertical direction, the display device (100) can enlarge the size of the first image (e.g., Image1e) because the visibility of the target may decrease as the screen moves. The display device (100) can enlarge the first image (e.g., Image1e) at a set magnification. The display device (100) can adjust the image quality of the first image (e.g., Image1f) so that the enlarged first image (e.g., Image1f) can be displayed clearly. For example, when an input image moves in a horizontal or vertical direction, the display device (100) can adjust (or change) the effect of the first image (e.g., Image1e) to obtain an emphasized first image (e.g., Image1f). For example, when an input image moves in a horizontal direction (e.g., D2), the display device (100) can emphasize the movement direction by adjusting the contrast or color. For example, the display device (100) can enhance the effect of the first image (e.g., Image1f) in proportion to the movement speed. For example, when the input image moves in the vertical direction, the display device (100) can smoothly express the movement by adjusting the blurring or sharpening effect.

FIGS. 14A to 14C are drawings showing various ways in which a display device (100) according to one embodiment of the present disclosure displays a first image (Image1).

Referring to FIGS. 14A to 14C , the display device (100) of the present disclosure can display a first image (Image1) corresponding to a region of interest (ROI) at various locations. For example, the display device (100) can display the first image (Image1) in a non-display area (nda). For example, the display device (100) can display the first image (Image1) in a picture-in-picture (PIP) manner.

For example, as shown in FIG. 14a, the display device (100) may include a non-display area (nda) on the left or right side of the display area, depending on the difference between the supported resolution of the display device (100) and the resolution of the input image. The display device (100) may display the first image (Image1) on the non-display area (nda) on the left or right side where no image content is displayed.

For example, as shown in FIG. 14b, the display device (100) may include a non-display area (nda) at the top or bottom of the display area, depending on the difference between the supported resolution of the display device (100) and the resolution of the input image. The display device (100) may display the first image (Image1) in the non-display area (nda) at the top or bottom where no image content is displayed.

For example, as shown in FIG. 14c, the display device (100) can output a sub-screen displaying a first image (Image1) within a main screen displaying an input image in a pip manner. The display device (100) can change the position and size of the sub-screen displaying the first image (Image1). For example, the display device (100) can change the position or adjust the size of the sub-screen displaying the first image (Image1) based on user settings.

FIG. 15 is a flowchart illustrating the operation of a display device (100) according to one embodiment of the present disclosure when an embedded virtual aiming point (EVAP) exists in an input image.

Referring to FIG. 15, the display device (100) detects an embedded virtual aim point (EVAP) included in the input image (operation 1510), and based on the detection of the embedded virtual aim point (EVAP), disables the redundant application of the virtual aim point (VAP) to the input image (operation 1520), sets the region of interest (ROI) including the embedded virtual aim point (EVAP) (operation 1530), and adjusts the size or effect of a third image (Image3) corresponding to the region of interest (ROI) (operation 1540).

According to an example, in operations 1510 and 1520, the display device (100) may detect an embedded virtual aiming point (EVAP) included in an input image, and based on the detection of the embedded virtual aiming point (EVAP), may disable the redundant application of the virtual aiming point (VAP) to the input image. Hereinafter, operations 1510 and 1520 of the display device (100) will be exemplarily described with reference to FIG. 16.

FIG. 16 is an exemplary diagram showing an input image of a display device (100) according to one embodiment of the present disclosure.

Referring to FIG. 16, the display device (100) can analyze the image characteristics of an input image. The display device (100) can extract individual frames from the input image. For example, the display device (100) can analyze the image characteristics of the input image based on the extracted individual frames. Based on the image characteristics, the display device (100) can identify an embedded virtual aiming point (EVAP) included in the input image.

The display device (100) may disable the redundant application of the virtual aiming point (VAP) to the input image based on the detection of the built-in virtual aiming point (EVAP). For example, the display device (100) may set a flag to prevent the redundant application of the virtual aiming point (VAP) to the input image, thereby preventing the application of an additional virtual aiming point (VAP) to an input image including the built-in virtual aiming point (EVAP).

For example, in operations 1530 and 1540, the display device (100) may set a region of interest (ROI) including an embedded virtual aiming point (EVAP) and adjust the size or effect of a third image (Image3) corresponding to the region of interest (ROI). Hereinafter, operations 1530 and 1540 of the display device (100) will be exemplarily described with reference to FIG. 17.

FIG. 17 is an exemplary diagram showing an operation of a display device (100) according to one embodiment of the present disclosure to adjust the size or effect of a third image (Image3).

Referring to FIG. 17, the display device (100) can set a region of interest (ROI) by utilizing an embedded virtual aiming point (EVAP). For example, the display device (100) can adjust the size of the region of interest (ROI) based on at least one of the size, importance, or location of the target within the input image. For example, the display device (100) can adjust the shape of the region of interest (ROI) based on the shape of the target or a user's settings.

The display device (100) can adjust the size or effect of the third image (Image3) corresponding to the region of interest (ROI). For example, the display device (100) can perform at least one of enlargement, reduction, sharpening, blurring, contrast adjustment, color adjustment, or distortion filter application to the third image (Image3).

The display device (100) can perform at least one of magnification and reduction on the third image (Image3). The display device (100) can magnify an image within a region of interest (ROI) set around a virtual aiming point (VAP), thereby increasing the visibility of the target. For example, when aiming at an enemy character in an FPS game, a user can easily recognize details of the enemy using the magnified third image (Image3).

The display device (100) can perform at least one of blurring, contrast adjustment, or color adjustment on the third image (Image3). The display device (100) can apply various image filters to more clearly express the image within the region of interest (ROI). For example, the display device (100) can apply a blurring filter to blur the background outside the region of interest (ROI), thereby focusing the user's attention on the virtual aiming point (VAP). For example, the display device (100) can increase the contrast of the image within the region of interest (ROI) through contrast adjustment, thereby making the target more visible.

FIG. 18 is a flowchart showing an operation according to a type of control signal of a display device (100) according to one embodiment of the present disclosure, and FIGS. 19a and 19b are drawings showing an operation performed according to a type of control signal of a display device (100) according to one embodiment of the present disclosure.

Referring to FIG. 18, the display device (100) receives a control signal from the control device (3) (operation 1810), determines whether the control signal is a first control signal unrelated to adjusting the size or the effect of the first image (Image1) corresponding to the region of interest (ROI), or a second control signal that adjusts the size or the effect of the first image (Image1) (operation 1820), and if the control signal is the first control signal, transmits the first control signal to the server (2) (operation 1830), and if the control signal is the second control signal, adjusts the size or the effect of the first image (Image1) based on the second control signal (operation 1840).

According to an example, in operation 1810, the display device (100) can receive a control signal from the control device (3). As shown in FIGS. 19A and 19B , the control device (3) can output a user input to the display device (100). The control device (3) can be connected to the display device (100) via a wireless communication network. For example, the control device (3) can be located within a predetermined distance from the display device (100) and communicate with the display device (100) via a BLE (Bluetooth Low Energy) communication network. For example, the control device (3) can generate a user input based on a user action and transmit the user input to the display device (100) via the BLE communication network.

For example, the control device (3) may be, but is not limited to, a remote control (3a), a game controller (3b), a keyboard (3c), a smartphone, a tablet PC, a PC, a smart TV, a mobile phone, a PDA (personal digital assistant), a laptop, a media player, a micro server (2), a GPS (global positioning system) device, an e-book reader, a digital broadcasting terminal, a navigation device, a kiosk, an MP3 player, a digital camera, and other mobile or non-mobile computing devices. In addition, the control device (3) may be a wearable device having a communication function and a data processing function.

According to an example, in operation 1820, the display device (100) can determine whether the control signal received from the control device (3) is a first control signal or a second control signal. For example, the display device (100) can determine whether the control signal is a first control signal that is unrelated to adjusting the size or the effect of the first image (Image1) corresponding to the region of interest (ROI), or a second control signal that adjusts the size or the effect of the first image (Image1). For example, the first control signal may be a control signal that attacks a target in an input image. For example, the second control signal may be a control signal that changes the effect of the first image (Image1) corresponding to the region of interest (ROI).

According to an example, in operation 1830, the display device (100) may transmit the first control signal to the server (2) if the control signal is the first control signal. For example, as shown in FIG. 19A, the display device (100) may identify the first control signal (cont1) and request the server (2) to perform a control operation corresponding to the first control signal (cont1).

For example, in operation 1840, the display device (100) may adjust the size or the effect of the first image (Image1) based on the second control signal, if the control signal is the second control signal. For example, as shown in FIG. 19b, the display device (100) may identify the second control signal (cont2) and perform an image adjustment operation corresponding to the second control signal (cont2). Here, the display device (100) may not transmit the second control signal (cont2) to the server (2).

In one embodiment, a display device (100) and a display method using the same can apply a virtual aiming point (VAP) to an input image and display an image in which the size or effect of a region of interest (ROI) centered on the virtual aiming point (VAP) is adjusted. The display device (100) and a display method using the same can enhance the immersion and user satisfaction with video content by increasing the aiming accuracy of game play.

In one embodiment, the display device (100) and the display method using the same can detect a virtual aiming point (VAP) embedded in an input image, thereby preventing the redundant application of the virtual aiming point (VAP), and can change the size or effect of an image adjusted based on the movement direction or movement speed of the input image. The display device (100) and the display method using the same can provide a flexible screen configuration suitable for the display environment, and can provide image content optimized for game play in real time.

The display devices according to the various embodiments disclosed in this document may take various forms. The display devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. The display devices according to the embodiments of this document are not limited to the aforementioned devices.

The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor of the machine (e.g., an electronic device) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims (15)

In the display device (200), Display (240); A memory (220) storing a program including at least one instruction; and At least one processor (210) connected to the memory and executing at least one instruction of a program stored in the memory, At least one processor, Apply a virtual aim point to the input image, Set an area of interest that includes the above virtual aiming point, Adjust the size or effect of the first image corresponding to the above area of interest, By synthesizing the input image and the first image, a second image is generated, Displaying the second image through the display, Display device. In the first paragraph, At least one processor, Performing at least one of enlarging, reducing, sharpening, blurring, adjusting contrast, adjusting color, or applying a distortion filter to the first image; Display device. In claim 1 or 2, The above areas of interest are: Having a shape centered on the virtual aiming point and including part or all of the target, Display device. In any one of the first to third paragraphs, At least one processor, Change the position of the virtual aiming point based on the change in the position of the target within the input image, In response to a change in the position of the virtual aiming point, the area of interest is changed. Display device. In any one of the first to fourth paragraphs, At least one processor, Identifying at least one object in the input image using a deep learning model, Determining the motion of at least one object using a motion tracking algorithm, Obtaining a background feature including at least one of brightness, color, or contrast of the input image, Applying the virtual aiming point based on at least one of the position of the at least one object, the motion of the at least one object, or the background characteristics; Display device. In any one of the first to fifth paragraphs, At least one processor, Determine a target based on the importance of at least one object included in the input image, Based on the coordinates of the target, determining the position of the virtual aiming point within the input image; Display device. In any one of claims 1 to 6, At least one processor, Detecting at least one of the movement direction or movement speed of the input image, Changing the size or the effect of the first image based on at least one of the movement direction or the movement speed, Display device. In paragraph 7, At least one processor, Based on the movement of the input image forward or backward, changing the size of the first image based on the movement speed. Display device. In paragraph 7 or 8, At least one processor, Based on the movement of the input image in the horizontal or vertical direction, changing the effect of the first image based on the movement speed. Display device. In any one of claims 1 to 9, At least one processor, Displaying the first image in a non-display area, Display device. In any one of claims 1 to 10, At least one processor, Displaying the first image in a pip (picture-in-picture) manner, Display device. In any one of claims 1 to 11, At least one processor, Detecting an embedded virtual aim point included in the above input image, Disabling application of the virtual aiming point to the input image based on the detection of the built-in virtual aiming point; Display device. In Article 12, At least one processor, Setting the area of interest including the built-in virtual aiming point, Adjusting the size or effect of the third image corresponding to the above area of interest, Display device. In any one of claims 1 to 13, At least one processor, Receive a control signal from the control device, Determine whether the control signal is a first control signal that is unrelated to adjusting the size or the effect of the first image corresponding to the region of interest, or a second control signal that adjusts the size or the effect of the first image, Based on the above control signal being the first control signal, transmitting the first control signal to the server, Adjusting the size or the effect of the first image based on the above control signal being the second control signal, Display device. In the display method, An action of applying a virtual aim point to an input image (310); An operation (320) of setting an area of interest including the above virtual aiming point; An action (330) of adjusting the size or effect of the first image corresponding to the above area of interest; An operation (340) of generating a second image by synthesizing the input image and the first image; and An operation (350) of displaying the second image through a display; How to display.