KR102457247B1 - Electronic device for processing image and method for controlling thereof - Google Patents

Abstract

An electronic device according to various embodiments of the present disclosure may include a camera module and an image processing module. The image processing module is configured to obtain a first image and a second image for the scene including the first object and the second object, and use the first image and the second image to obtain a first image corresponding to the first object A first amount of motion and a second amount of motion corresponding to the second object are identified, and the first image region corresponding to the first object and the second object are at least based on the first amount of motion and the second amount of motion. Different effects may be applied to the second image area corresponding to .

Description

ELECTRONIC DEVICE FOR PROCESSING IMAGE AND METHOD FOR CONTROLLING THEREOF

The present invention relates to an electronic device for processing an image and a method for controlling the same.

Recently, an electronic device (eg, a portable electronic device) including a camera module has been widely used. Accordingly, the user can easily take an image using the electronic device at a desired time point. In particular, recent electronic devices may include a camera module on a front part and a rear part, respectively. Accordingly, when the user wants to photograph the exterior, the user may photograph the image using the camera module disposed on the rear portion of the electronic device. In addition, when the user wants to take a picture of himself/herself, the user may take an image by using a camera module disposed on the front side of the electronic device.

As described above, the electronic device may capture an image using the camera module disposed on the front part. However, as the electronic device is miniaturized, the camera module included in the electronic device may also be relatively small. Accordingly, the camera module included in the electronic device has no choice but to perform a relatively limited function. For example, a photographing function that requires driving of a lens, such as an out focus effect, is difficult to operate in a camera module included in a miniaturized electronic device.

According to various embodiments of the present disclosure, an electronic device including the camera module having the above-described limited functions may also perform various functions such as an out-focus function, a segmentation function, and generation of a depth map.

In various embodiments of the present disclosure, an electronic device may include a camera module for photographing a scene; and an image processing module operatively connected to the camera module, wherein the image processing module is configured to: acquire a first image and a second image for the scene including a first object and a second object; identifying a first motion amount corresponding to the first object and a second motion amount corresponding to the second object by using the first image and the second image; and different effects may be applied to the first image area corresponding to the first object and the second image area corresponding to the second object based on at least the first motion amount and the second motion amount. The display may show an image to which the effect has been applied or the memory may store the image to which the effect has been applied.

In various embodiments of the present disclosure, an electronic device may include a camera module for capturing a scene including one object; and an image processing module operatively connected to the camera module, wherein the image processing module is configured to: acquire a plurality of images for the scene; check whether a motion amount corresponding to the object in the plurality of images satisfies a specified value; and a specified effect may be applied to the image area corresponding to the object based at least on the confirmation result.

According to various embodiments of the present disclosure, an electronic device capable of performing various functions, such as an out-focus function, a segmentation function, and a depth map generation, even in a small electronic device that cannot control the depth of a camera, and a method for controlling the same This can be provided.

1A is a block diagram of an electronic device and a network according to various embodiments of the present disclosure;
1B is a block diagram of an electronic device according to various embodiments of the present disclosure;
2 is a block diagram of an electronic device according to various embodiments of the present disclosure;
3 is a block diagram of a program module according to various embodiments of the present disclosure;
4 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
5A to 5E are conceptual diagrams of an electronic device according to various embodiments of the present disclosure.
6 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
7 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.
8 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
9A is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
9B is a conceptual diagram of a tri map according to various embodiments of the present invention.
10 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
11 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
12 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
13 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
14 is an image in which a background is blurred according to various embodiments of the present disclosure.
15 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
16 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
17 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.
18 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
19 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.
20 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. However, it is not intended to limit the technology described in this document to specific embodiments, and it should be understood to include various modifications, equivalents, and/or alternatives of the embodiments of this document. . In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "has," "may have," "includes," or "may include" refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as "A or B," "at least one of A and/and B," or "one or more of A or/and B" may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used herein, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of the rights described in this document, a first component may be referred to as a second component, and similarly, the second component may also be renamed as a first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

As used herein, the expression "configured to (or configured to)" depends on the context, for example, "suitable for," "having the capacity to ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning to the meaning in the context of the related art, and unless explicitly defined in this document, ideal or excessively formal meanings is not interpreted as In some cases, even terms defined in this document cannot be construed to exclude embodiments of the present document.

The electronic device according to various embodiments of the present document may include, for example, a smartphone, a tablet personal computer, a mobile phone, a video phone, and an e-book reader. , desktop personal computer, laptop personal computer, netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, mobile It may include at least one of a medical device, a camera, and a wearable device. According to various embodiments, the wearable device may be an accessory type (eg, a watch, ring, bracelet, anklet, necklace, eyeglass, contact lens, or head-mounted-device (HMD)), a fabric or an integral piece of clothing ( It may include at least one of, for example, electronic clothing), a body attachable type (eg, a skin pad or tattoo), or a bioimplantable type (eg, an implantable circuit).

In some embodiments, the electronic device may be a home appliance. Home appliances are, for example, televisions, digital video disk (DVD) players, audio systems, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air purifiers, set-top boxes, home automation controls. panel (home automation control panel), security control panel (security control panel), TV box (eg Samsung HomeSync ^TM , Apple TV ^TM , or Google TV ^TM ), game console (eg Xbox ^TM , PlayStation ^TM ), electronic dictionary , an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include various medical devices (eg, various portable medical measuring devices (eg, a blood glucose meter, a heart rate monitor, a blood pressure monitor, or a body temperature monitor), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), Computed tomography (CT), imagers, or ultrasound machines, etc.), navigation devices, satellite navigation systems (global navigation satellite system (GNSS)), event data recorder (EDR), flight data recorder (FDR), automotive infotainment ) devices, ship electronic equipment (e.g. ship navigation devices, gyro compasses, etc.), avionics, security devices, vehicle head units, industrial or domestic robots, automatic teller's machines (ATMs) in financial institutions. , point of sales (POS) in stores, or internet of things (e.g. light bulbs, sensors, electricity or gas meters, sprinkler devices, smoke alarms, thermostats, street lights, toasters) , exercise equipment, hot water tank, heater, boiler, etc.) may include at least one.

According to some embodiments, the electronic device is a piece of furniture or part of a building/structure, an electronic board, an electronic signature receiving device, a projector, or various measuring instruments (eg, water, electricity, gas, or a radio wave measuring device). In various embodiments, the electronic device may be a combination of one or more of the various devices described above. The electronic device according to an embodiment may be a flexible electronic device. In addition, the electronic device according to the embodiment of the present document is not limited to the above-described devices, and may include a new electronic device according to technological development.

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. In this document, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

1A , an electronic device 101 in a network environment 100 is described, in various embodiments. The electronic device 101 may include a bus 110 , a processor 120 , a memory 130 , an input/output interface 150 , a display 160 , and a communication module 170 . In some embodiments, the electronic device 101 may omit at least one of the components or may additionally include other components.

The bus 110 may include, for example, a circuit that connects the components 110 - 170 to each other and transmits communication (eg, a control message and/or data) between the components.

The processor 120 may include one or more of a central processing unit (CPU), an application processor (AP), and a communication processor (CP). The processor 120 may, for example, execute an operation or data processing related to control and/or communication of at least one other component of the electronic device 101 .

The memory 130 may include volatile and/or non-volatile memory. The memory 130 may store, for example, commands or data related to at least one other component of the electronic device 101 . According to one embodiment, the memory 130 may store software and/or a program 140 . Program 140 may include, for example, kernel 141 , middleware 143 , application programming interface (API) 145 , and/or application programs (or “applications”) 147 , etc. may include At least a portion of the kernel 141 , the middleware 143 , or the API 145 may be referred to as an operating system (OS).

The kernel 141 is, for example, system resources (eg, middleware 143, API 145, or application program 147) used to execute an operation or function implemented in other programs (eg, middleware 143, API 145, or the application program 147). : The bus 110, the processor 120, the memory 130, etc.) can be controlled or managed. In addition, the kernel 141 may provide an interface capable of controlling or managing system resources by accessing individual components of the electronic device 101 from the middleware 143 , the API 145 , or the application program 147 . can

The middleware 143 may, for example, play an intermediary role so that the API 145 or the application program 147 communicates with the kernel 141 to send and receive data.

Also, the middleware 143 may process one or more work requests received from the application program 147 according to priority. For example, the middleware 143 may use a system resource (eg, the bus 110 , the processor 120 , or the memory 130 ) of the electronic device 101 for at least one of the application programs 147 . You can give priority. For example, the middleware 143 may process the one or more work requests according to the priority given to the at least one, thereby performing scheduling or load balancing for the one or more work requests.

The API 145 is, for example, an interface for the application 147 to control functions provided by the kernel 141 or the middleware 143, for example, file control, window control, image processing, or text. It may include at least one interface or function (eg, a command) for control and the like.

The input/output interface 150 may serve as an interface capable of transmitting, for example, a command or data input from a user or other external device to other component(s) of the electronic device 101 . Also, the input/output interface 150 may output a command or data received from other component(s) of the electronic device 101 to a user or other external device.

Display 160 may be, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, or microelectromechanical systems (MEMS) displays, or electronic paper displays. The display 160 may, for example, display various contents (eg, text, image, video, icon, or symbol) to the user. The display 160 may include a touch screen, and may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body.

The communication module 170, for example, sets up communication between the electronic device 101 and an external device (eg, the first external electronic device 102 , the second external electronic device 104 , or the server 106 ). can For example, the communication module 170 may be connected to the network 162 through wireless communication or wired communication to communicate with an external device (eg, the second external electronic device 104 or the server 106 ).

Wireless communication is, for example, a cellular communication protocol, for example, long-term evolution (LTE), LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal (UMTS). At least one of mobile telecommunications system), Wireless Broadband (WiBro), and Global System for Mobile Communications (GSM) may be used. Also, wireless communication may include, for example, short-range communication 164 . The short-range communication 164 may include, for example, at least one of wireless fidelity (WiFi), Bluetooth, near field communication (NFC), or global navigation satellite system (GNSS). GNSS is, depending on the area or bandwidth used, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter “Beidou”) or Galileo, the European global satellite-based navigation system. It may include at least one. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". Wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), or plain old telephone service (POTS). The network 162 may include at least one of a telecommunications network, for example, a computer network (eg, LAN or WAN), the Internet, or a telephone network.

In various embodiments of the present invention, a camera module (not shown) may capture a scene. The image processing module 180 may input/output data with other hardware through the bus 110 . In various embodiments of the present disclosure, the image processing module 180 obtains a first image and a second image for the scene including a first object and a second object, and generates the first image and the second image. a first motion amount corresponding to the first object and a second motion amount corresponding to the second object using Different effects may be applied to the first image region and the second image region corresponding to the second object.

In various embodiments of the present disclosure, the first amount of motion corresponds to a difference between a position of the first object in the first image and a position of the first object in the second image, wherein the second The amount of motion may correspond to a difference between a position of the second object in the first image and a position of the second object in the second image.

In various embodiments of the present disclosure, the image processing module 180 may select, as the first image and the second image, a pair of images in which the difference in the amount of motion is greater than or equal to a specified value among a plurality of preview images for the scene. have.

In various embodiments of the present disclosure, the image processing module 180 compares the first image and the second image, and selects at least one of the first motion amount or the second motion amount based on the comparison result. can be obtained

In various embodiments of the present disclosure, the image processing module 180 compares the first image and the second image, and determines whether a difference between the first motion amount and the second motion amount satisfies a specified value. and, based on the determination, the first image area and the second image area may be determined.

In various embodiments of the present disclosure, the image processing module 180 may separate the first object from the second object in the first image based on the first motion amount and the second motion amount. have.

In various embodiments of the present disclosure, the first image area may correspond to a first depth for the scene, and the second image area may correspond to a second depth for the scene.

In various embodiments of the present disclosure, the image processing module 180 classifies the first image region based on specified color information, the first motion amount, and a region to be classified around the first image region. determine the difference in the amount of motion of , and if the difference satisfies the specified value, determine the region of the classification target as a part of the corresponding first image region, and if the difference does not satisfy the specified value, the classification target It may be determined that the area of ' does not belong to the first image area.

In various embodiments of the present disclosure, the first object includes a person and the second object includes a background, and the image processing module 180 blurs the background, the person and the An image including the blurred background may be output as an image corresponding to the scene.

In various embodiments of the present disclosure, the first image may be obtained by photographing the electronic device at a first location, and the second image may be obtained by photographing the electronic device at a second location. The first motion amount and the second motion amount may be generated by a position difference between the first position and the second position.

In various embodiments of the present disclosure, the image processing module 180 is configured to set a first depth value to the first object based on the first motion amount and the second motion amount, and to set a second depth value to the second object. 2 You can set the depth value.

In various embodiments of the present disclosure, the image processing module 180 acquires a plurality of images for the scene, checks whether a motion amount corresponding to the object in the plurality of images satisfies a specified value, and A specified effect may be applied to the image region corresponding to the object based at least on the result of the check. The amount of motion may correspond to a difference between a position of the object in a first image among the plurality of images and a position of the object in a second image among the plurality of images.

In various embodiments of the present disclosure, the image processing module 180 may select, as the first image and the second image, a pair of images in which the difference in the amount of motion is greater than or equal to a specified value among a plurality of preview images for the scene. have. The first image area may correspond to a first depth for the scene, and the second image area may correspond to a second depth for the scene.

In various embodiments of the present disclosure, the object includes a person, the image processing module 180 blurs a background excluding the person, and generates an image including the person and the blurred background. An image corresponding to the scene may be output. Each of the plurality of images may be obtained by photographing the electronic device at different locations.

In various embodiments of the present disclosure, the image processing module 180 obtains a first image and a second image, compares the first image and the second image, and each of at least one object of the first image check the amount of motion of , obtain depth information of each of the at least one object based on the amount of motion of each of the at least one object, and based on the obtained depth information, a depth map for the first image can create

Each of the first and second external electronic devices 102 and 104 may be the same as or different from the electronic device 101 . According to one embodiment, server 106 may include a group of one or more servers. According to various embodiments, all or part of the operations executed by the electronic device 101 may be executed by one or a plurality of other electronic devices (eg, the electronic devices 102 and 104 , or the server 106 ). Accordingly, when the electronic device 101 is to perform a function or service automatically or upon request, the electronic device 101 performs at least some functions related thereto instead of or in addition to executing the function or service itself. may request from another device (eg, electronic device 102, 104, or server 106) The other electronic device (eg, electronic device 102, 104, or server 106) may request the requested function or The additional function may be executed and the result may be transmitted to the electronic device 101. The electronic device 101 may provide the requested function or service by processing the received result as it is or additionally. For example, cloud computing, distributed computing, or client-server computing technology may be used.

1B is a block diagram of an electronic device according to various embodiments of the present disclosure; Referring to FIG. 1B , the image processing module 180 may include an image acquisition device 181 , an object analysis module 182 , and a processing module 186 .

The image acquisition device 181 may be an interface device capable of receiving an image input from a camera module, a communication module, or a memory. Alternatively, according to implementation, the image acquisition device 181 may include at least one of a camera module, a communication module, and a memory capable of acquiring and/or storing an image.

The object analysis module 182 may receive an image from the image acquisition device 181 . The face detection module 183 may detect the face of the person from the acquired image. The face detection module 183 may detect a face of a person from an image based on various face recognition algorithms, which will be described later in more detail. The motion detection module 184 may detect whether at least one object in the first image moves or the amount of motion by comparing the first image and the second image. The motion detection module 184 may detect whether there is a motion or an amount of motion according to the comparison of the position of the object in each of the first image and the second image. The depth detection module 185 may detect a depth value of an object in an image. For example, the camera module may output an image including depth information, and the depth detection module 185 may detect a depth value for each object based on the acquired depth information. Alternatively, the depth detection module 185 may detect a depth value for each object by analyzing an image using a pre-stored algorithm.

The processing module 186 may receive an image on which at least one of face detection, motion detection, and depth detection is performed. In various embodiments of the present disclosure, the processing module 186 may include an object separation module 187 and an image effect application module 188 . The object separation module 187 may separate, for example, segment an object from the image. The object separation module 187 according to various embodiments of the present disclosure may perform object separation based on the amount of motion, or may perform object separation using at least one of the amount of motion, color information, and depth value. . The image effect application module 188 may apply an image effect to each separated object. In various embodiments of the present disclosure, the image effect application module 168 may apply a blur effect. The image to which the image effect is applied may be displayed on the display 180 or stored in the memory 130 .

2 is a block diagram of an electronic device 201 according to various embodiments. The electronic device 201 may include, for example, all or a part of the electronic device 101 illustrated in FIG. 1A . The electronic device 201 includes one or more processors (eg, an application processor (AP)) 210 , a communication module 220 , (a subscriber identification module 224 , a memory 230 , a sensor module 240 , an input device ( 250 ), a display 260 , an interface 270 , an audio module 280 , a camera module 291 , a power management module 295 , a battery 296 , an indicator 297 , and a motor 298 . can

The processor 210 may control a plurality of hardware or software components connected to the processor 210 by, for example, driving an operating system or an application program, and may perform various data processing and operations. The processor 210 may be implemented as, for example, a system on chip (SoC). According to an embodiment, the processor 210 may further include a graphic processing unit (GPU) and/or an image signal processor. The processor 210 may include at least some of the components shown in FIG. 2 (eg, the cellular module 221 ). The processor 210 may load and process commands or data received from at least one of other components (eg, non-volatile memory) into a volatile memory, and store various data in the non-volatile memory. have.

The communication module 220 may have the same or similar configuration to the communication interface 170 of FIG. 1A . The communication module 220 is, for example, a cellular module 221 , a WiFi module 223 , a Bluetooth module 225 , a GNSS module 227 (eg, a GPS module, a Glonass module, a Beidou module, or a Galileo module). , may include an NFC module 228 and a radio frequency (RF) module 229 .

The cellular module 221 may provide, for example, a voice call, a video call, a text service, or an Internet service through a communication network. According to an embodiment, the cellular module 221 may use a subscriber identification module (eg, a SIM card) 224 to identify and authenticate the electronic device 201 within a communication network. According to an embodiment, the cellular module 221 may perform at least some of the functions that the processor 210 may provide. According to an embodiment, the cellular module 221 may include a communication processor (CP).

Each of the WiFi module 223 , the Bluetooth module 225 , the GNSS module 227 , or the NFC module 228 may include, for example, a processor for processing data transmitted and received through a corresponding module. According to some embodiments, at least some (eg, two or more) of the cellular module 221 , the WiFi module 223 , the Bluetooth module 225 , the GNSS module 227 , or the NFC module 228 is one integrated chip. (IC) or contained within an IC package.

The RF module 229 may transmit/receive a communication signal (eg, an RF signal), for example. The RF module 229 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 221 , the WiFi module 223 , the Bluetooth module 225 , the GNSS module 227 or the NFC module 228 transmits and receives an RF signal through a separate RF module. can

Subscriber identification module 224 may include, for example, a card including a subscriber identification module and/or an embedded SIM (SIM), and may include unique identification information (eg, integrated circuit card identifier (ICCID)) or Subscriber information (eg, international mobile subscriber identity (IMSI)) may be included.

The memory 230 (eg, the memory 130 ) may include, for example, an internal memory 232 or an external memory 234 . The built-in memory 232 may include, for example, a volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), a non-volatile memory (eg, a non-volatile memory). One time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (such as NAND flash or NOR flash); It may include at least one of a hard drive and a solid state drive (SSD).

The external memory 234 is a flash drive, for example, compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme xD (xD). digital), a multi-media card (MMC), or a memory stick may be further included. The external memory 234 may be functionally and/or physically connected to the electronic device 201 through various interfaces.

The sensor module 240 may, for example, measure a physical quantity or sense an operating state of the electronic device 201 , and convert the measured or sensed information into an electrical signal. The sensor module 240 is, for example, a gesture sensor 240A, a gyro sensor 240B, a barometric pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, a proximity sensor ( 240G), color sensor (240H) (e.g. RGB (red, green, blue) sensor), biometric sensor (240I), temperature/humidity sensor (240J), illuminance sensor (240K), or UV (ultra violet) sensor ) may include at least one of the sensors 240M. Additionally or alternatively, the sensor module 240 may include, for example, an olfactory sensor (E-nose sensor), an electromyography sensor (EMG sensor), an electroencephalogram sensor (EEG sensor), an electrocardiogram sensor (ECG sensor) , an infrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one or more sensors included therein. In some embodiments, the electronic device 201 further comprises a processor configured to control the sensor module 240 , either as part of the processor 210 or separately, while the processor 210 is in a sleep state; The sensor module 240 may be controlled.

The input device 250 may include, for example, a touch panel 252 , a (digital) pen sensor 254 , a key 256 , or an ultrasonic input device ( 258) may be included. The touch panel 252 may use, for example, at least one of a capacitive type, a pressure sensitive type, an infrared type, and an ultrasonic type. Also, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user.

The (digital) pen sensor 254 may be, for example, a part of a touch panel or may include a separate recognition sheet. Key 256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 258 may detect an ultrasonic wave generated by an input tool through a microphone (eg, the microphone 288 ), and check data corresponding to the sensed ultrasonic wave.

The display 260 (eg, the display 160 ) may include a panel 262 , a hologram device 264 , or a projector 266 . The panel 262 may have the same or similar configuration to the display 160 of FIG. 1A . The panel 262 may be implemented to be flexible, transparent, or wearable, for example. The panel 262 may be composed of the touch panel 252 and one module. The hologram device 264 may display a stereoscopic image in the air by using light interference. The projector 266 may display an image by projecting light onto the screen. The screen may be located inside or outside the electronic device 201 , for example. According to one embodiment, the display 260 may further include a control circuit for controlling the panel 262 , the hologram device 264 , or the projector 266 .

The interface 270 is, for example, a high-definition multimedia interface (HDMI) 272 , a universal serial bus (USB) 274 , an optical interface 276 , or a D-subminiature (D-sub). ) (278). The interface 270 may be included, for example, in the communication interface 170 shown in FIG. 1A . Additionally or alternatively, the interface 270 may be, for example, a mobile high-definition link (MHL) interface, a secure digital (SD) card/multi-media card (MMC) interface, or an infrared (IrDA) interface. data association) standard interface.

The audio module 280 may interactively convert a sound and an electrical signal, for example. At least some components of the audio module 280 may be included in, for example, the input/output interface 150 illustrated in FIG. 1A . The audio module 280 may process sound information input or output through, for example, the speaker 282 , the receiver 284 , the earphone 286 , or the microphone 288 .

The camera module 291 is, for example, a device capable of capturing still images and moving images, and according to an embodiment, one or more image sensors (eg, a front sensor or a rear sensor), a lens, and an image signal processor (ISP). , or a flash (eg, an LED or xenon lamp, etc.).

The power management module 295 may manage power of the electronic device 201 , for example. According to an embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charger integrated circuit (IC), or a battery or fuel gauge. The PMIC may have a wired and/or wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonance circuit, or a rectifier. have. The battery gauge may measure, for example, the remaining amount of the battery 296 , voltage, current, or temperature during charging. The battery 296 may include, for example, a rechargeable battery and/or a solar battery.

The indicator 297 may display a specific state of the electronic device 201 or a part thereof (eg, the processor 210 ), for example, a booting state, a message state, or a charging state. The motor 298 may convert an electrical signal into mechanical vibration, and may generate vibration or a haptic effect. Although not shown, the electronic device 201 may include a processing unit (eg, GPU) for supporting mobile TV. The processing device for supporting mobile TV may process media data according to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or mediaFloTM, for example.

Each of the components described in this document may be composed of one or more components, and the name of the component may vary depending on the type of the electronic device. In various embodiments, the electronic device may be configured to include at least one of the components described in this document, and some components may be omitted or may further include additional other components. In addition, since some of the components of the electronic device according to various embodiments are combined to form a single entity, the functions of the components prior to being combined may be identically performed.

3 is a block diagram of a program module according to various embodiments of the present disclosure; According to an embodiment, the program module 310 (eg, the program 140 ) is an operating system (OS) and/or an operating system that controls resources related to the electronic device (eg, the electronic device 101 ). It may include various applications (eg, the application program 147) running on the system. The operating system may be, for example, Android, iOS, Windows, Symbian, Tizen, or bada.

The program module 310 may include a kernel 320 , middleware 330 , an application programming interface (API) 360 , and/or an application 370 . At least a portion of the program module 310 may be preloaded on the electronic device or downloaded from an external electronic device (eg, the electronic devices 102 and 104 , the server 106 , etc.).

The kernel 320 (eg, the kernel 141 ) may include, for example, a system resource manager 321 and/or a device driver 323 . The system resource manager 321 may control, allocate, or recover system resources. According to an embodiment, the system resource manager 321 may include a process manager, a memory manager, or a file system manager. The device driver 323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication (IPC) driver. .

The middleware 330 provides, for example, functions commonly required by the applications 370 or provides various functions through the API 360 so that the applications 370 can efficiently use limited system resources inside the electronic device. Functions may be provided to the application 370 . According to an embodiment, the middleware 330 (eg, middleware 143 ) includes a runtime library 335 , an application manager 341 , a window manager 342 , and a multimedia manager. ) (343), resource manager (344), power manager (345), database manager (346), package manager (347), connectivity manager ) 348 , a notification manager 349 , a location manager 350 , a graphic manager 351 , or a security manager 352 . can do.

The runtime library 335 may include, for example, a library module used by the compiler to add a new function through a programming language while the application 370 is being executed. The runtime library 335 may perform input/output management, memory management, or a function for an arithmetic function.

The application manager 341 may manage, for example, a life cycle of at least one application among the applications 370 . The window manager 342 may manage GUI resources used in the screen. The multimedia manager 343 may identify a format required for reproduction of various media files, and may encode or decode the media file using a codec suitable for the format. The resource manager 344 may manage resources such as source code, memory, or storage space of at least one of the applications 370 .

The power manager 345 may, for example, operate in conjunction with a basic input/output system (BIOS), etc. to manage a battery or power, and provide power information required for the operation of the electronic device. The database manager 346 may create, search, or change a database to be used by at least one of the applications 370 . The package manager 347 may manage installation or update of applications distributed in the form of package files.

The connection manager 348 may manage a wireless connection such as WiFi or Bluetooth, for example. The notification manager 349 may display or notify the user of events such as arrival messages, appointments, and proximity notifications in an unobtrusive manner. The location manager 350 may manage location information of the electronic device. The graphic manager 351 may manage graphic effects to be provided to the user or a user interface related thereto. The security manager 352 may provide various security functions necessary for system security or user authentication. According to an embodiment, when the electronic device (eg, the electronic device 101) includes a phone function, the middleware 330 further includes a telephony manager for managing the voice or video call function of the electronic device. can do.

The middleware 330 may include a middleware module that forms a combination of various functions of the aforementioned components. The middleware 330 may provide a specialized module for each type of operating system in order to provide a differentiated function. Also, the middleware 330 may dynamically delete some existing components or add new components.

The API 360 (eg, the API 145 ) is, for example, a set of API programming functions, and may be provided in a different configuration according to an operating system. For example, in the case of Android or iOS, one API set may be provided for each platform, and in the case of Tizen, two or more API sets may be provided for each platform.

The application 370 (eg, the application program 147 ) may include, for example, a home 371 , a dialer 372 , an SMS/MMS 373 , an instant message (IM) 374 , a browser 375 , Camera 376 , Alarm 377 , Contacts 378 , Voice Dial 379 , Email 380 , Calendar 381 , Media Player 382 , Album 383 , or Clock 384 , Health Care It may include one or more applications capable of performing a function such as health care (eg, measuring exercise amount or blood sugar) or providing environmental information (eg, providing information on air pressure, humidity, or temperature).

According to an embodiment, the application 370 is an application that supports information exchange between an electronic device (eg, the electronic device 101) and an external electronic device (eg, the electronic devices 102 and 104) (hereinafter, described below). For convenience, "information exchange application") may be included. The information exchange application may include, for example, a notification relay application for transmitting specific information to an external electronic device or a device management application for managing the external electronic device.

For example, the notification delivery application transmits notification information generated by another application of the electronic device (eg, an SMS/MMS application, an email application, a health management application, or an environment information application) to an external electronic device (eg, the electronic device 102 ). , 104)). Also, the notification delivery application may receive notification information from, for example, an external electronic device and provide the notification information to the user.

The device management application is, for example, the turn-of at least one function (eg, the external electronic device itself (or some component) of the external electronic device (eg, the electronic device 102, 104) that communicates with the electronic device) Manage (eg, install, delete, etc.) applications running on/off or display brightness (or resolution) adjustments), applications running on external electronic devices, or services provided by external electronic devices (eg, call services or message services, etc.) , or update).

According to an embodiment, the application 370 may include an application (eg, a health management application of a mobile medical device, etc.) designated according to a property (eg, the electronic device 102 or 104 ) of the external electronic device (eg, the electronic device 102 or 104 ). According to an embodiment, the application 370 may include an application received from an external electronic device (eg, the server 106 or the electronic devices 102 and 104). may include a preloaded application or a third party application downloadable from a server The names of components of the program module 310 according to the illustrated embodiment depend on the type of the operating system. Therefore, it may vary.

According to various embodiments, at least a portion of the program module 310 may be implemented in software, firmware, hardware, or a combination of at least two or more thereof. At least a portion of the program module 310 may be implemented (eg, executed) by, for example, a processor (eg, the processor 210 ). At least a portion of the program module 310 may include, for example, a module, a program, a routine, sets of instructions, or a process for performing one or more functions.

As used herein, the term “module” may refer to, for example, a unit including one or a combination of two or more of hardware, software, or firmware. The term “module” may be used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit. A “module” may be a minimum unit or a part of an integrally configured component. A “module” may be a minimum unit or a part of performing one or more functions. A “module” may be implemented mechanically or electronically. For example, a “module” may be one of an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs) or programmable-logic device, known or to be developed, that performs certain operations. It may include at least one.

At least a portion of an apparatus (eg, modules or functions thereof) or a method (eg, operations) according to various embodiments is, for example, a computer-readable storage medium in the form of a program module It can be implemented as a command stored in . When the instruction is executed by a processor (eg, the processor 120), the one or more processors may perform a function corresponding to the instruction. The computer-readable storage medium may be, for example, the memory 130 .

Computer-readable recording media include hard disks, floppy disks, magnetic media (eg, magnetic tape), optical media (eg, compact disc read only memory (CD-ROM), DVD ( digital versatile disc), magneto-optical media (such as floppy disk), hardware devices (such as read only memory (ROM), random access memory (RAM), or flash memory, etc.) ), etc. In addition, the program instructions may include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The above-described hardware devices include various It may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

A module or program module according to various embodiments may include at least one or more of the above-described components, some may be omitted, or may further include additional other components. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic manner. Also, some operations may be executed in a different order, omitted, or other operations may be added. In addition, the embodiments disclosed in this document are presented for explanation and understanding of the disclosed and technical content, and do not limit the scope of the technology described in this document. Accordingly, the scope of this document should be construed to include all modifications or various other embodiments based on the technical spirit of this document.

4 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure. The embodiment of Figure 4 will be described in more detail with reference to Figures 5a to 5e. 5A to 5E are conceptual diagrams of an electronic device according to various embodiments of the present disclosure.

In operation 410 , the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. For example, as shown in FIG. 5A , the user may take a picture of himself/herself by holding the electronic device 101 . The electronic device 101 may generate and display a preview image using light incident to a camera module disposed in front. Meanwhile, the user may shake the hand holding the electronic device 101 . According to the user's hand shake, the electronic device 101 may move (501, 502). As the electronic device 101 moves ( 501 , 502 ), a location at which the electronic device 101 captures a user may be changed. For example, the electronic device 101 may capture and store a first image photographed at a first photographing position corresponding to the first movement 501 , and may be stored at a second photographing position corresponding to the second movement 502 . The captured second image may be captured and stored. The position of the user's face region in the first image and the position of the user's face region in the second image may be different, which may be due to the different shooting positions. That is, even when the subject does not move, the subject motion in the image may be caused according to the movement of the electronic device 101 .

In operation 420 , the electronic device 101 may check the first motion amount of the first object and the second motion amount of the second object. In operation 430, the electronic device 101 may apply an image effect based on the first motion amount and the second motion amount. More specifically, as shown in FIG. 5A , the electronic device 101 may acquire the amount of motion of the face region based on a difference between the positions of the face region in the first image and the second image. Here, the amount of motion may be a difference between the position of the face region in the first image and the position of the face region in the second image.

The electronic device 101 may separate the face region from the background region with respect to the first image or the second image based on the amount of motion of the face region, and may separate the person region including the face region and the body region from the background region. have. In addition, the electronic device 101 may apply an image effect based on the separation result. For example, the electronic device 101 may apply a blur effect to a portion excluding the face region or may apply a blur effect to a portion excluding the person region.

According to various embodiments of the present disclosure, the electronic device 101 may segment the person region from the background region by further using at least one of color information and phase information in addition to the amount of motion of the face region. The electronic device 101 may output an image to which the blur effect is applied to the background area based on the segmentation result. Accordingly, even when the electronic device 101 is not a high-performance camera module having an out-focus function, it may output an image processed as out-focused. In another embodiment, the electronic device 101 may obtain a motion amount for each of the face region or another object, and may obtain depth information for each of at least one object in the image. The electronic device 101 may generate and display a depth map or a 3D map based on the acquired depth information. Accordingly, the electronic device 101 may generate and output a 3D map without applying a complex algorithm such as a conventional 3D map algorithm. In addition, even when a phase pixel is not included as in the 2PD system, the electronic device 101 according to various embodiments of the present disclosure may acquire phase information such as depth information.

5B is a conceptual diagram illustrating an operation of an electronic device according to various embodiments of the present disclosure.

As shown in FIG. 5B , the user may take a picture of himself/herself by holding the electronic device 101 . The electronic device 101 may generate and display a preview image using light incident to a camera module disposed in front. Meanwhile, the user may move ( 510 ) the arm while holding the electronic device 101 to move ( 511 ) the photographing position of the electronic device 101 .

The electronic device 101 may move 511 according to the movement of the user's arm. As the electronic device 101 moves 511 , the location at which the electronic device 101 captures the user may be changed. For example, the electronic device 101 may photograph and store the first image 512 photographed at the first photographing position, and may photograph and store the second image 513 photographed at the second photographing position. The position of the user's face region in the first image 512 and the position of the user's face region in the second image 513 may be different, which may be due to the different photographing positions. The electronic device 101 may acquire the amount of motion of the face region based on a difference between the positions of the face region in the first image 512 and the second image 513 . The electronic device 101 may segment the first image 512 or the second image 513 based on the amount of motion of the face region, and separate the person region including the face region and the body region from the background region. can do. In various embodiments of the present disclosure, the electronic device 101 may separate the person region from the background region by further using the motion amount of the face region and at least one of color information and phase information. The electronic device 101 may output an image to which the blur effect is applied to the background area based on the separation result. Accordingly, even when the electronic device 101 is not a high-performance camera module having an out-focus function, it may output an image processed as out-focused. In another embodiment, the electronic device 101 may check the amount of motion for each of the face region or other object, and obtain depth information for each of at least one object in the image. The electronic device 101 may generate and display a depth map or a 3D map based on the acquired depth information. Accordingly, the electronic device 101 may generate and output a 3D map without applying a complex algorithm such as a conventional 3D map algorithm. In addition, even when a phase pixel is not included as in the 2PD system, the electronic device 101 according to various embodiments of the present disclosure may acquire phase information such as depth information.

5C to 5E are conceptual diagrams illustrating photographing of an electronic device according to various embodiments of the present disclosure.

The electronic device 101 may photograph a scene including at least one subject 521 and 522 at the first location 523 ( t1 ). In addition, the electronic device 101 may change the photographing position to photograph a scene including at least one subject 521 and 522 at the second position 524 ( t2 ).

In addition, as described above, the user may grip the electronic device 101 and perform photographing. When the user holds the electronic device 101 , the electronic device 101 may change the photographing position from the first position to the second position due to the user's hand shake. The electronic device 101 may continuously photograph an image after entering the photographing mode to display the preview image. The electronic device 101 may acquire a plurality of images having different photographing positions based on a change in the photographing position due to a user's hand shake while continuously photographing an image in the photographing mode.

In various embodiments of the present disclosure, the user may move the electronic device 101 from the first position 523 to the second position 524 while holding the electronic device 101 . For example, the electronic device 101 may output a location guidance message to the effect of moving the electronic device 101 , so that the user can check this and perform photographing while changing the photographing position of the electronic device 101 . Meanwhile, in the embodiment of FIG. 5C , the distance from the electronic device 101 to the first subject 501 is the first distance d1 , and the distance from the electronic device 101 to the second subject 502 is the It is assumed that is the second distance d2.

5D is a conceptual diagram of an image according to various embodiments of the present disclosure. The first image 530 of FIG. 5D may be an image photographed (t1) at the first location 523 of FIG. 5C, and the second image 540 is photographed (t2) at the second location 524 of FIG. 5C ) may be an image. The first image 530 may include a first object 531 corresponding to the first subject 521 and a second object 532 corresponding to the second subject 522 , and the second image 540 . may include a first object 541 corresponding to the first subject 521 and a second object 542 corresponding to the second subject 522 .

The electronic device 101 may check the amount of motion of each of at least one object in the image. FIG. 5E may be a result of comparing the first image 530 and the second image 540 of FIG. 5D . The electronic device 101 may compare the first image 530 and the second image 540 to determine the amount of motion 551 for the first objects 531 and 541 . For example, the electronic device 101 may compare the pixel position of the first object 531 and the pixel position of the first object 541 to determine a difference in pixel position, and a motion corresponding to the determined pixel position The amount 551 may be determined. That is, the amount of motion 551 may mean the degree of movement of the first objects 531 and 541 in the image, that is, the pixel difference. The electronic device 101 may identify the first object 531 in the first image 530 and may identify the first object 541 in the second image 540 . The electronic device 101 may determine that the identified first objects 531 and 541 are the same object, and accordingly, when comparing the first image 530 and the second image 540 , the difference between the first objects 531 and 541 is A motion amount 551 may be determined. In various embodiments of the present disclosure, the electronic device 101 may store an object recognition algorithm for various objects such as a person and a tree, and accordingly, a first object 531 corresponding to the person in the first image 530 . and a second object 532 corresponding to a tree may be recognized, and a first object 541 corresponding to a person and a second object 542 corresponding to a tree may be recognized in the second image 540 . Meanwhile, in various embodiments of the present disclosure, the electronic device 101 may track a face using a face recognition algorithm, which will be described in more detail later.

The electronic device 101 may separate the object based on the amount of motion of each of the at least one object. As illustrated in FIG. 5E , the magnitude of the motion amount 551 of the first objects 531 and 541 may be greater than the magnitude of the motion amount 552 of the second objects 532 and 542 . This may be due to the fact that the first distance d1 between the electronic device 101 and the first subject 521 is smaller than the second distance d2 between the electronic device 101 and the second subject 522 . have. When the distance to the subject is relatively small, the movement of the object in the image, that is, the amount of motion, may be relatively large according to the change of the shooting position. If the distance to the subject is relatively large, even if the shooting position is changed, The amount of motion of the object in . The electronic device 101 may distinguish and separate the first object 531 and the second object 532 having different motion amounts. In addition, the electronic device 101 may separate the first object 531 and the second object 532 from the background. The electronic device 101 may perform object separation based on the amount of motion for each object. Since the different motion amounts for each object means that the distances to the subjects corresponding to the objects are different, the electronic device 101 may determine that objects having different motion amounts are different and perform segmentation. The electronic device 101 may perform segmentation based on the amount of motion and at least one of color information and depth information, which will be described later in more detail. As described above, the electronic device 101 may separate an object and apply an image effect for each separated object based on the amount of motion for each object using a plurality of images.

Meanwhile, according to various embodiments of the present disclosure, the electronic device 101 may include a dual camera, that is, a camera disposed on one surface with an interval therebetween. The electronic device 101 may calculate the amount of motion of the object, classify the object, and separate the object through the plurality of images acquired through the dual camera.

6 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 610, the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. For example, as shown in FIG. 5C , the electronic device 101 may photograph a scene including at least one subject 521 and 522 at a first location 523 ( t1 ). In addition, the electronic device 101 may change the photographing position to photograph a scene including at least one subject 521 and 522 at the second position 524 ( t2 ). As described above, the electronic device 101 may acquire a plurality of images having different photographing positions based on various location changes, such as a location change due to hand shake or a location change of the electronic device 101 intended by a user.

In operation 620 , the electronic device 101 may identify the amount of motion of each of the at least one object in the image. As described above, the electronic device 101 may compare a plurality of images to determine the amount of motion of each of at least one object in the image.

In operation 630 , the electronic device 101 may determine depth information for each object based on the amount of motion of each of the at least one object. As described above, when the distance to the subject is relatively small, the amount of motion of the object in the image may be relatively large according to the change of the shooting position, and when the distance to the subject is relatively large, even if the shooting position is changed The amount of motion of the object in the image may be relatively small. Accordingly, the electronic device 101 may set a relatively large depth value to an object having a relatively small amount of motion, and may set a relatively small depth value to an object having a relatively large amount of motion. The electronic device 101 may set a depth value to each of the identified objects, and may perform object separation, ie, segmentation, based on the depth information in operation 640 . Meanwhile, as will be described later in more detail, the electronic device 101 according to various embodiments of the present disclosure may generate a depth map or apply a 3D effect based on a depth value.

7 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

In operation 710 , the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. As described above, the electronic device 101 may acquire a plurality of images having different photographing positions based on various location changes, such as a location change due to hand shake or a location change of the electronic device 101 intended by a user.

In operation 720, the electronic device 101 may determine the amount of motion of each of at least one object in the image. As described above, the electronic device 101 may compare a plurality of images to determine the amount of motion of each of at least one object in the image.

In operation 730, the electronic device 101 may analyze color information for each pixel of the target image among the plurality of images. In operation 740 , the electronic device 101 may perform segmentation based on the analyzed color information and the motion amount of each of at least one object in the target image. In various embodiments of the present disclosure, the electronic device 101 may generate a tri-map based on the amount of motion of each object, and classify and separate the object based on the generated tri-map and color information. . Alternatively, the electronic device 101 may generate a trimap based on the amount of motion and color information of each object, and perform object classification and separation based on the generated trimap and color information. In addition, the electronic device 101 may apply an image effect to the separated object. Alternatively, the electronic device 101 may firstly separate the object based on the color information and then secondarily separate the object based on the amount of motion. The above-described order may be changed, and the electronic device 101 may firstly separate the object based on the amount of motion, and then may additionally separate the object secondarily based on color information.

8 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 810 , the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. As described above, the electronic device 101 may acquire a plurality of images having different photographing positions based on various location changes, such as a location change due to hand shake or a location change of the electronic device 101 intended by a user.

In operation 820 , the electronic device 101 may determine the amount of motion of each of at least one object in the image. As described above, the electronic device 101 may compare a plurality of images to determine the amount of motion of each of at least one object in the image.

In operation 830 , the electronic device 101 may analyze color information for each pixel of the target image among the plurality of images. In operation 840, the electronic device 101 may analyze phase information on a phase pixel of the target image. In various embodiments of the present disclosure, the electronic device 101 may include phase pixels such as, for example, two photodiodes (2PDs). A phase pixel such as 2PD may include a plurality of photodiodes in one pixel unit. In this case, the first light-receiving amount measured by the first photodiode of one pixel unit and the second light-receiving amount measured by the second photodiode may be different from each other. The electronic device 101 may analyze the phase information based on the difference in the amount of light received by each of the two photodiodes in one pixel unit. For example, the electronic device 101 may analyze depth information in the corresponding pixel based on the difference in the amount of light received.

In operation 850 , the electronic device 101 may perform segmentation based on the analyzed color information, phase information, and motion amount of each of at least one object in the target image. In various embodiments of the present disclosure, the electronic device 101 may generate a tri map based on the amount of motion of each object, and perform segmentation based on the generated tri map, color information, and phase information. have. Alternatively, the electronic device 101 may generate a trimap based on the amount of motion, phase information, and color information of each object, and perform segmentation based on the generated trimap, color information, and phase information. Alternatively, the electronic device 101 may primarily perform segmentation based on color information and phase information, and then additionally perform segmentation secondarily based on the amount of motion. The above-described order may be changed, and the electronic device 101 may firstly perform segmentation based on the amount of motion, and then may additionally perform segmentation secondarily based on color information and phase information. Meanwhile, according to various embodiments of the present disclosure, the electronic device 101 may perform segmentation based on the amount of motion and the phase information.

9A is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 910 , the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. As described above, the electronic device 101 may acquire a plurality of images having different photographing positions based on various location changes, such as a location change due to hand shake or a location change of the electronic device 101 intended by a user.

In operation 920, the electronic device 101 may select a target image and a comparison image from among the plurality of images. Here, the target image may be an image to which segmentation and additional effects are to be applied. According to an embodiment, the electronic device 101 may select a target image based on a user input through the graphic user interface. Alternatively, the electronic device 101 may select a target image based on a pre-stored algorithm. For example, the electronic device 101 may select an image that satisfies predetermined levels of illuminance information and color information among a plurality of images as a target image. Alternatively, the electronic device 101 selects, as a target image, an image in which the degree of similarity between an object, such as an eye, nose, or mouth, and a predetermined template among a plurality of images including a face of a person exceeds a threshold, that is, satisfies a predetermined value. You can also choose A person skilled in the art will readily understand that there is no limitation on a configuration in which the electronic device 101 determines a target image among a plurality of images.

The electronic device 101 may select an image in which a difference from a target image exceeds a threshold among a plurality of images as a comparison image. As the position difference between each object of the target image and the comparison image increases, the amount of motion may be large. As the amount of motion increases, the electronic device 101 may perform more accurate segmentation. Accordingly, the electronic device 101 may select an image having the greatest difference from the target image as the comparison image. Meanwhile, the electronic device 101 may select an image including the tracking target object of the target image as the comparison image. That is, in one embodiment, the electronic device 101 may select an image having the greatest difference from the target image among images including the tracking target object of the target image as the comparison image.

In operation 930 , the electronic device 101 compares the target image and the comparison image to determine the amount of motion of each of at least one object in the target image. As described above, each of the target image and the comparison image may include a tracking target object, and the tracking target object may correspond to the same subject. In an embodiment, the tracking target object may be a face, and the electronic device 101 may determine the location of the tracking target object in the target image and the location of the tracking target object in the comparison image using a face recognition algorithm. The electronic device 101 may determine the amount of motion by comparing the position of the tracking target object in the target image and the comparison image.

In operation 940 , the electronic device 101 may generate a tri map based on the amount of motion. For example, as shown in FIG. 9B , the electronic device 101 may generate the trimap 980 from the target image 970 . The target image 970 is a photograph of a person and may include a face object 971 and a body object 972 . For example, the electronic device 101 may detect the face object from the comparison image by tracking the face object 971 , and may determine the amount of motion for the face object 971 . The electronic device 101 may generate the trimap 980 based on the amount of motion of the face object 971 . The tri-map 980 may include a background area 981 , an unconfirmed area 982 , and a person area 983 . The electronic device 101 may classify an area having a motion amount corresponding to the motion amount of the face object 971 as a person area 983 . The electronic device 101 may classify a region having the amount of motion of the face object 971 and the amount of motion exceeding the threshold as the background region 981 . The electronic device 101 may classify an area other than the person area 983 and the background area 981 as an unconfirmed area 982 . Meanwhile, the electronic device 101 may generate the trimap 980 by additionally using color information. Alternatively, the electronic device 101 may generate the trimap 980 by additionally using the phase information. Alternatively, the electronic device 101 may generate the trimap 980 by additionally using the color information and the phase information.

In operation 950 , the electronic device 101 may segment the target image based on the trimap 980 . For example, the electronic device 101 may perform segmentation on the unconfirmed area 982 of the trimap 980 . The electronic device 101 may re-classify the pixels of the unconfirmed area 982 into either the person area 983 or the background area 981 based on at least one of color information and phase information of the unconfirmed area 982 . Accordingly, the electronic device 101 may segment the person areas 971 and 972 from the background from the target image 970 .

In operation 960, the electronic device 101 may apply an effect to the background of the target image by using the segmentation result. For example, the electronic device 101 may apply a blurring effect to the background. In the image output from the electronic device 101 , a blurring effect may not be applied to a person area, a blurring effect may be applied to a background, and an out-focused effect may be implemented.

Meanwhile, the above-described tri-map generation is merely exemplary, and as described above, the electronic device 101 according to various embodiments of the present disclosure may separate the tri-map based on the amount of motion of the object. That is, the electronic device 101 may separate an object having a motion amount of the tracking target object and a motion amount less than a predetermined threshold into one object, for example, a person object. The electronic device 101 may apply an image effect, for example, a blur effect, to each object based on the separation result.

10 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1010 , the electronic device 101 may select a target image from among a plurality of images. In various embodiments of the present disclosure, the electronic device 101 may select a target image based on a user's selection or may select a target image based on a predetermined algorithm. In operation 1020 , the electronic device 101 may determine whether to detect a face region in the target image. In the embodiment of FIG. 10 , it is assumed that the electronic device 101 segments a person in an image. If the face region is not detected, the electronic device 101 may end photographing or select another image including the face region from among a plurality of images as the target image. Alternatively, in operation 1090 , the electronic device 101 may apply a second image effect, which is a different image effect, to a case in which the face region is not detected.

In operation 1030 , the electronic device 101 may select a comparison image from among a plurality of images. In operation 1040 , the electronic device 101 may determine whether a face region is detected from the comparison image. In the embodiment of FIG. 10 , it is assumed that the electronic device 101 separates a person in an image. If the face region is not detected, the electronic device 101 may end photographing or select another image including the face region from among a plurality of images as a comparison image.

In operation 1050 , the electronic device 101 may detect the amount of motion by comparing the target image and the comparison image. The electronic device 101 may compare the position of the face region in the target image with the position of the face region in the comparison image. For example, the electronic device 101 may compare the position of the pixel corresponding to the face region in the target image with the position of the pixel corresponding to the face region in the comparison image. The electronic device 101 may detect the amount of motion based on the comparison result. The amount of motion may be for a difference in position of a tracking target object, eg, a face region, in each of the target image and the comparison image. For example, the motion amount may include at least one of a motion degree and a motion direction of the tracking target object.

In operation 1060 , the electronic device 101 may determine whether there is a movement of the tracking target object, for example, a face region. In one embodiment, the electronic device 101 may determine whether there is motion based on whether the magnitude of the motion amount exceeds a predetermined threshold, that is, satisfies a predetermined value. If it is determined that the value is less than or equal to the predetermined threshold, the electronic device 101 may determine that there is no movement. The electronic device 101 may end the operation or select another comparison image. If it is determined that the predetermined threshold is exceeded, the electronic device 101 may determine that there is a movement of the tracking target object, for example, a face region.

In operation 1070 , the electronic device 101 may determine whether the motion between the target image and the comparison image is maximum, that is, whether a predetermined value is satisfied. In an embodiment, the electronic device 101 may sequentially compare a plurality of comparison images with a target image, and may finally determine an image having the greatest motion as the comparison image.

In operation 1080, the electronic device 101 may separate the object based on the amount of motion determined based on the comparison result with the target image and the comparison image and apply an image effect based on the result of separation. As described above, according to an embodiment, the electronic device 101 may perform segmentation by classifying an object having substantially the same amount of motion as that of the face region as a person region. For example, the target image may include a body region connected to a face region. The comparison image may also include a body region connected to the face region. The electronic device 101 may determine that the amount of motion of the face region is substantially the same as that of the body region, and may separate the face region and the body region from the background as one person region. Here, substantially the same may mean that the difference between the two motion amounts is less than a threshold. In addition, as described above, the electronic device 101 may additionally output an image to which a blurring effect is applied to a background or the like.

11 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

Since operations 1110 to 1170 have been described in detail in operations 1010 to 1070 of FIG. 10 , further descriptions herein will be omitted.

In operation 1180, the electronic device 101 may determine whether the magnitude of the movement exceeds a threshold, that is, whether a specified value is satisfied. When the magnitude of the motion is equal to or less than the threshold, the electronic device 101 may generate a trimap in operation 1197 using photodiode data of the target image in operation 1190 . That is, when the amount of motion is not large enough, the electronic device 101 may generate the trimap using photodiode data rather than the amount of motion. For example, the electronic device 101 may generate a trimap based on color information for each pixel. Alternatively, when the electronic device 101 includes a phase pixel such as a 2PD system, the electronic device 101 may generate a trimap based on phase information obtained from the phase pixel. The electronic device 101 may generate a trimap by using the phase information and the color information.

When the magnitude of the motion exceeds the threshold, in operation 1195 , the electronic device 101 may generate a motion map using photodiode data of the target image. In operation 1197, the electronic device 101 may generate a tri-map by using the motion map. For example, the electronic device 101 may generate a trimap having a background area, a foreground area (eg, a person area), and an unconfirmed area based on the amount of motion of each object of the motion map. The electronic device 101 may classify an object corresponding to the amount of motion of the tracking target object of the motion map into one and classify the object as a foreground area. In another embodiment, the electronic device 101 may additionally generate a tri-map using photodiode data based on the motion map. The electronic device 101 may additionally perform object separation based on at least one of color information and phase information based on the generated trimap and applying an image effect using the object separation result. Meanwhile, if the face region is not detected, the electronic device 101 may end the operation or may apply another image effect in operation 1199 .

12 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1210 , the electronic device 101 may determine the target image 1211 and obtain color information, motion amount, and phase information of the target image 1211 . Although not shown, the electronic device 101 may select a comparison image and acquire a motion amount of each of at least one object in the target image according to a comparison result between the target image and the comparison image. In addition, the electronic device 101 may acquire color information of the target image 1211 . When the electronic device 101 includes a phase pixel such as a 2PD system, phase information may be acquired.

In operation 1220 , the electronic device 101 detects the face region 1221 in the target image 1211 . The electronic device 101 may detect the face region 1221 by selecting it as a tracking target object. The electronic device 101 may store a face detection algorithm in advance, and may detect the face region 1221 using this. The electronic device 101 may detect the face region 1221 based on color information of the target image 1211 .

In operation 1230 , the electronic device 101 detects the contour 1231 of the person area 1232 . The electronic device 101 may acquire the amount of motion of the face region 1221 . The electronic device 101 may detect the person area 1232 based on the amount of motion of the face area 1221 , and accordingly, the contour 1231 may be detected. As described above, the electronic device 101 converts a region having a motion amount substantially equal to or less than a threshold difference to the motion amount of the face region 1221 as a person region 1232 together with the face region 1221 . can be classified. The electronic device 101 may detect the boundary of the person area 1232 as the contour 1231 , or determine the area including the boundary as the contour 1231 . Meanwhile, in various embodiments of the present disclosure, the electronic device 101 may detect the contour 1231 of the person area 1232 by further using at least one of color information and phase information in addition to the amount of motion. In operation 1240 , the electronic device 101 may determine the detected contour 1231 as the undecided area 1243 . The electronic device 101 may determine a background area 1241 and a foreground area 1242 .

In operation 1250 , the electronic device 101 may mark the face-corresponding shape 1252 and the body-corresponding shape 1253 to classify them as a person area. The electronic device 101 may also perform marking on the background area 1251 . In operation 1260 , the electronic device 101 may separate the object and apply an image effect using the separation result. The electronic device 101 may perform segmentation on the undetermined area 1243 , and thus may reclassify the pixels of the undetermined area 1243 as either a person area or a background area. The electronic device 101 may perform object separation with respect to the undetermined area 1243 based on at least one of color information and phase information. As the pixels of the undecided area 1243 are reclassified into either the person area or the background area, the electronic device 101 may output an image separated into only the person area and the background area.

13 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1310 , the electronic device 101 may generate a motion map using the target image 1301 and the comparison image 1302 . The electronic device 101 may detect the tracking target object in the target image 1301 and detect the tracking target object in the comparison image 1302 . The tracking target object may be an object of the same subject. The electronic device 101 may generate a motion map by detecting the amount of motion for at least one tracking target object. In various embodiments of the present disclosure, the motion map may be displayed in color to distinguish at least one of a motion size and a motion direction. The electronic device 101 may generate the tri-map 1304 based on the motion map. In addition, the electronic device 101 may determine an initial foreground area and an initial background area based on the motion map. The electronic device 101 may determine an initial foreground area and an initial background area by further using at least one of color information and phase information in addition to the motion map. In addition, the electronic device 101 may generate the trimap 1304 by determining an unconfirmed area that is the remaining area not classified into the initial foreground area and the initial background area.

In operation 1320 , the electronic device 101 may generate a Gaussian Mixture Model (GMM) with respect to a foreground area and an initial background area of an initial motion. A probability that a pixel belongs to a given model may be equal to Equation (1).

In Equation 1, x is a variable value related to a face to be detected, and θ is a mathematical model for expressing the foreground/backview. Meanwhile, i is an index for representing the i-th Gaussian, wi may be a weight, μi may represent an average, and σi may represent a standard deviation.

The electronic device 101 may generate a GMM for each of the background region and the foreground region, and perform segmentation based on each weight calculated from each GMM in operation 1330 . The electronic device 101 may generate a GMM with respect to the phase information 1303 as well as the color information, and may perform segmentation using the GMM. In various embodiments of the present disclosure, the electronic device 101 may perform segmentation using an energy segmentation framework, for example, may perform graph cut. The energy minimization framework may classify pixels into a foreground area and a background area, and the classified pixels from the trimap may act as data components. Meanwhile, the weight of the edge between pixels may correspond to the pixel similarity. For a relatively large difference in pixel color, an edge may have a relatively low weight, which may be used as a smoothness component for minimization. For an unconfirmed pixel, the electronic device 101 may calculate a pixel edge connected to a source and a sink node that is dependent on the probability of a pixel that is a foreground and background area. The minimization function can minimize the cost function of the image to separate source and sink nodes. The cost function may be the sum of weights of edges to be cut. After cutting, a pixel connected to a source node may be classified as a foreground, and a pixel connected to a sink node may be classified as a background.

In operation 1340 , the electronic device 101 may apply an additional effect to the segmented background. For example, the electronic device 101 may apply a blur effect to the background. Accordingly, the electronic device 101 may output an image in which the background is blurred as shown in FIG. 14 .

15 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1501 , the electronic device 101 may acquire a preview image. For example, the electronic device 101 may photograph a user in a self-camera mode. The electronic device 101 may acquire a plurality of preview images, and the plurality of preview images may be images obtained by photographing a user at different photographing positions.

In operation 1503 , the electronic device 101 may detect a face region from the target image. The electronic device 101 may detect a face region from the target image based on a pre-stored face recognition algorithm.

In operation 1505 , the electronic device 101 may determine whether a face region exists in the target image. If it is determined that the face region does not exist in the target image, the electronic device 101 may capture another preview image. If it is determined that the face region exists in the target image, the electronic device 101 may display the face region in operation 1507 . For example, the electronic device 101 may display an additional object capable of indicating a face region on the preview image. In addition, in operation 1509 , the electronic device 101 may track a feature point with respect to the face region. In operation 1511 , the electronic device 101 may store the feature point location.

In operation 1513 , when the number N of the acquired preview images is 1, in operation 1515 , the electronic device 101 may capture and store the preview image as a first image. Meanwhile, when the number N of the acquired preview images exceeds 1, in operation 1517 , the electronic device 101 may acquire the motion amount of the key point, that is, the motion amount of the face region, based on the key point tracking result.

In operation 1519, the electronic device 101 may determine whether the magnitude of the movement exceeds a threshold. When the magnitude of the movement is equal to or less than the threshold, the electronic device 101 may acquire another preview image. When the magnitude of the movement exceeds the threshold, the electronic device 101 may capture and store the second image in operation 1521 . The electronic device 101 may also store the motion amount. Alternatively, in operation 1523 , the electronic device 101 may display a user interface (UI) of completion of obtaining a plurality of images. In operation 1525 , the electronic device 101 may perform object separation using the first image and the second image. The electronic device 101 may perform object separation on the first image or the second image based on the amount of motion.

16 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1601 , the electronic device 101 may capture a plurality of images. The electronic device 101 may capture and store a plurality of images at different photographed locations. In operation 1603 , the electronic device 101 detects a face region from one image among a plurality of images. If it is determined that the face region exists in operation 1605 , the electronic device 101 may select it as a target image in operation 1607 . In operation 1609 , the electronic device 101 may detect an amount of motion between images. In operation 1611 , the electronic device 101 may obtain an optimal motion map. The motion map may include at least one of a motion size and a motion direction, and a criterion for determining the quality of the motion map may be preset. In operation 1613 , the electronic device 101 may determine whether the quality of the motion map satisfies a predetermined value. When it is determined that the quality of the motion map satisfies the predetermined value, the electronic device 101 may perform object separation on the target image based on the amount of motion of the motion map.

17 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

Since operations 1701 and 1703 and operations 1715 to 1719 have been described in detail in operations 1601 and 1603 and operations 1609 to 1613 of FIG. 16 , further descriptions thereof will be omitted.

If it is determined in operation 1705 that the face region exists, in operations 1707 , 1709 , and 1711 , the electronic device 101 determines whether the face region of the image has a smiling shape, does not blink, or is not a blurred image. can judge When it is determined that the face region of the image has a smiling shape, eyes do not blink, and it is not a blurred image, the electronic device 101 may select the corresponding image as the target image.

18 is a flowchart illustrating a method of controlling an electronic device according to various embodiments of the present disclosure.

In operation 1810, the electronic device 101 may acquire a plurality of images. Here, the plurality of images may be images captured at different photographing locations. As described above, the electronic device 101 may acquire a plurality of images having different photographing positions based on various location changes, such as a location change due to hand shake or a location change of the electronic device 101 intended by a user.

In operation 1820, the electronic device 101 may determine the amount of motion of each of the at least one object in the image. As described above, the electronic device 101 may compare a plurality of images to determine the amount of motion of each of at least one object in the image.

In operation 1830, the electronic device 101 may determine depth information for each object based on the amount of motion of each of the at least one object. As described above, when the distance to the subject is relatively small, the movement of the object in the image may be relatively large according to the change of the shooting position, and when the distance to the subject is relatively large, even if the shooting position is changed The movement of the object in the image may be relatively small. Accordingly, the electronic device 101 may set a relatively large depth value to an object having a relatively small amount of motion, and may set a relatively small depth value to an object having a relatively large amount of motion.

In operation 1840, the electronic device 101 may generate a depth map based on a depth value of each object. The depth map may be a map including a depth value of each object. The electronic device 101 may reconstruct and display the 3D image based on the depth map.

19 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

In operation 1910 , the electronic device 101 may capture and obtain a scene including one object. The electronic device 101 may acquire a first image and a second image of one object.

In operation 1920, the electronic device 101 may determine the amount of motion corresponding to the object by comparing the first image and the second image. The electronic device 101 may check whether the checked motion amount satisfies a specified value.

In operation 1930, the electronic device 101 may apply a specified effect to the image area corresponding to the object based at least on the result of the check. For example, the electronic device 101 may apply an image effect based on the amount of motion. In more detail, when the amount of motion satisfies a predetermined value, the electronic device 101 may perform object separation, and may perform image processing such as blur processing using the separation result.

As described above, image processing may be performed through image analysis including one object.

20 is a flowchart illustrating a method for controlling an electronic device according to various embodiments of the present disclosure.

In operation 2010, the electronic device 101 may detect a face from the image. In operation 2020, the electronic device 101 may determine whether a face is detected in the image. When a face is detected, in operation 2030 , the electronic device 101 may enter a background separation photographing mode in which the face is separated from the background through the above-described process. If no face is detected, the electronic device 101 may enter a general photographing mode in operation 2040 .

Even when face detection is successful, if the number of detected faces is equal to or greater than a certain number (eg, 5) or the area occupied by (plural) face(s) is equal to or greater than a certain percentage of the entire image, the electronic device 101 may not use the background separation shooting mode in the corresponding scene, and thus the amount of computation may be reduced.

In various embodiments of the present disclosure, a method for controlling an electronic device may include acquiring a first image and a second image with respect to the scene including a first object and a second object, and using the first image and the second image a first motion amount corresponding to the first object and a second motion amount corresponding to the second object using Different effects may be applied to the first image region and the second image region corresponding to the second object.

In various embodiments of the present disclosure, the first amount of motion corresponds to a difference between a position of the first object in the first image and a position of the first object in the second image, wherein the second The amount of motion may correspond to a difference between a position of the second object in the first image and a position of the second object in the second image.

In various embodiments of the present disclosure, the method of controlling the electronic device may select a pair of images having a difference in motion amount equal to or greater than a specified value among a plurality of preview images for the scene as the first image and the second image.

In various embodiments of the present disclosure, the control method of the electronic device includes comparing the first image and the second image, and obtaining at least one of the first amount of motion or the second amount of motion based on a comparison result. can

In various embodiments of the present disclosure, the control method of the electronic device compares the first image and the second image, and determines whether a difference between the first motion amount and the second motion amount satisfies a specified value; Based on the determination, the first image area and the second image area may be determined.

In various embodiments of the present disclosure, the method of controlling the electronic device may separate the first object from the second object in the first image based on the first amount of motion and the second amount of motion. The first image area may correspond to a first depth for the scene, and the second image area may correspond to a second depth for the scene.

In various embodiments of the present disclosure, the control method of the electronic device classifies the first image region based on specified color information, the first motion amount, and a motion of a region to be classified around the first image region determine the difference in quantity, and if the difference satisfies the specified value, determine the region of the classification target as a part of the corresponding first image region; if the difference does not satisfy the specified value, the region of the classification target may be determined not to belong to the first image area.

In various embodiments of the present disclosure, the first object includes a person and the second object includes a background, and the control method of the electronic device includes blurring the background, and processing the person and the blurring. An image including the background image may be output as an image corresponding to the scene. The first image may be obtained by photographing the electronic device at a first location, and the second image may be obtained by photographing the electronic device at a second location. The first motion amount and the second motion amount may be generated by a position difference between the first position and the second position.

In various embodiments of the present disclosure, the method of controlling the electronic device includes setting a first depth value in the first object and a second depth in the second object based on the first motion amount and the second motion amount. value can be set.

In various embodiments of the present disclosure, the method of controlling an electronic device includes acquiring a plurality of images for the scene, checking whether a motion amount corresponding to the object in the plurality of images satisfies a specified value, and A specified effect may be applied to the image region corresponding to the object based at least on the result of the check. The amount of motion may correspond to a difference between a position of the object in a first image among the plurality of images and a position of the object in a second image among the plurality of images.

In various embodiments of the present disclosure, the method of controlling the electronic device may select a pair of images having a difference in motion amount equal to or greater than a specified value among a plurality of preview images for the scene as the first image and the second image. The first image area may correspond to a first depth for the scene, and the second image area may correspond to a second depth for the scene.

In various embodiments of the present disclosure, the object includes a person, the control method of the electronic device blurs a background excluding the person, and displays an image including the person and the blurred background into the scene It can be output as an image corresponding to . Each of the plurality of images may be obtained by photographing the electronic device at different locations.

In various embodiments of the present disclosure, a method of controlling an electronic device includes obtaining a first image and a second image, comparing the first image and the second image, and each of at least one object of the first image an operation of determining the amount of motion of , an operation of acquiring depth information of each of the at least one object based on the amount of motion of each of the at least one object, and an operation of obtaining the depth information of each of the at least one object based on the acquired depth information for the first image It may include an operation of generating a depth map. Each of the first image and the second image may be obtained by photographing the electronic device at different locations.

Each of the aforementioned components of the electronic device may be composed of one or more components, and the name of the corresponding component may vary depending on the type of the electronic device. In various embodiments, the electronic device may be configured to include at least one of the above-described components, and some components may be omitted or may further include additional other components. In addition, since some of the components of the electronic device according to various embodiments are combined to form a single entity, the functions of the components prior to being combined may be identically performed.

According to various embodiments of the present disclosure, in a storage medium storing instructions, the instructions are configured to cause the at least one processor to perform at least one operation when executed by the at least one processor, wherein the at least one An operation of obtaining a first image and a second image with respect to the scene including the first object and the second object, identifying a first motion amount and a second motion amount corresponding to the second object, and a first image area corresponding to the first object and the second motion amount based on at least the first motion amount and the second motion amount It may include an operation of applying different effects to the second image area corresponding to the object.

In addition, the embodiments disclosed in this document are provided for description and understanding of the disclosed, technical content, and do not limit the scope of the present disclosure. Accordingly, the scope of the present disclosure should be construed to include all modifications or various other embodiments based on the technical spirit of the present disclosure.

Claims (20)

In an electronic device,
camera; and
an image processing circuit operatively coupled to the camera;
The image processing circuit,
Obtaining a first image and a second image including the first object and the second object,
Using the first image and the second image, a first amount of motion corresponding to the first object and a second amount of motion corresponding to the second object having a difference greater than or equal to a specified value from the first amount of motion are identified, and ,
A first effect is applied to a first image area corresponding to the first object based at least in part on the confirmation of the first motion amount and the second motion amount, and a second effect is applied to a second image area corresponding to the second object. an electronic device configured to apply two effects, wherein the first effect is different from the second effect. The method of claim 1,
wherein the first amount of motion corresponds to a difference between a position of the first object in the first image and a position of the first object in the second image, wherein the second amount of motion includes: an electronic device corresponding to a difference between a position in the first image of and a position in the second image of the second object. ◈Claim 3 was abandoned when paying the registration fee.◈ The method of claim 1,
The image processing circuit,
An electronic device for selecting a pair of images having a difference in motion amount equal to or greater than a specified value among a plurality of preview images as the first image and the second image. The method of claim 1,
The image processing circuit,
The electronic device compares the first image and the second image, and obtains at least one of the first motion amount and the second motion amount based on a comparison result. The method of claim 1,
The image processing circuit,
comparing the first image and the second image;
determining whether a difference between the first motion amount and the second motion amount satisfies the specified value or more,
The electronic device determines the first image area and the second image area based on the determination. 6. The method of claim 5,
The image processing circuit,
The electronic device separates the first object from the second object in the first image based on the first amount of motion and the second amount of motion. The method of claim 1,
The first image area corresponds to a first depth, and the second image area corresponds to a second depth. The method of claim 1,
The image processing circuit,
classifying the first image area based on specified color information;
determining a difference between the first motion amount and a motion amount of a region to be classified around the first image region,
If the difference satisfies a specified value or more, determining the classification target area as a part of the corresponding first image area;
If the difference does not satisfy the specified value or more, the electronic device determines that the region to be classified does not belong to the first image region. The method of claim 1,
The first object includes a person and the second object includes a background,
The image processing circuit,
blur the background,
An electronic device for outputting an image including the person and the blurred background. ◈Claim 10 was abandoned when paying the registration fee.◈ The method of claim 1,
The first image is obtained by photographing at a first location, and the second image is obtained by photographing at a second location. ◈Claim 11 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The first motion amount and the second motion amount are generated by a position difference between the first position and the second position. 2. The method of claim 1
The image processing circuit is configured to set a first depth value to the first object and a second depth value to the second object based on the first motion amount and the second motion amount. A method for controlling an electronic device, comprising:
acquiring a first image and a second image including the first object and the second object;
Using the first image and the second image to identify a first motion amount corresponding to the first object and a second motion amount corresponding to the second object having a difference between the first motion amount and the specified value or more movement; and
A first effect is applied to a first image area corresponding to the first object based at least in part on the confirmation of the first motion amount and the second motion amount, and a second effect is applied to a second image area corresponding to the second object. and applying two effects, wherein the first effect is different from the second effect. ◈Claim 14 was abandoned when paying the registration fee.◈ 14. The method of claim 13,
the first amount of motion corresponds to a difference between a position of the first object in the first image and a position of the first object in the second image, wherein the second amount of motion is A control method of an electronic device corresponding to a difference between a position in the first image of and a position in the second image of the second object. ◈Claim 15 was abandoned when paying the registration fee.◈ 14. The method of claim 13,
and selecting, as the first image and the second image, a pair of images having a difference in motion amount equal to or greater than a specified value among a plurality of preview images. ◈Claim 16 has been abandoned at the time of payment of the registration fee.◈ 14. The method of claim 13,
The first image area corresponds to a first depth, and the second image area corresponds to a second depth. ◈Claim 17 was abandoned when paying the registration fee.◈ 14. The method of claim 13,
The first object includes a person, the second object includes a background,
The method is
blurring the background except for the person; and
The method of controlling an electronic device further comprising outputting an image including the person and the blurred background. ◈Claim 18 was abandoned when paying the registration fee.◈ 14. The method of claim 13,
The method of controlling an electronic device, wherein the first image is obtained by photographing it at a first location, and the second image is obtained by photographing it at a second location. delete delete

Images