KR20180055708A - Device and method for image processing - Google Patents

Abstract

The present disclosure relates to an artificial intelligence (AI) system and its application that simulate functions such as recognition and judgment of a human brain using a machine learning algorithm such as deep learning.
More particularly, the present disclosure relates to an image processing apparatus, comprising a display unit for displaying an image, a memory for storing one or more instructions, and a processor for executing one or more instructions stored in the memory, Extracting one object, determining an object to be deleted among the extracted at least one object based on the identity between the extracted at least one object and at least one object included in the user data, and deleting the determined object from the image Discloses an electronic device.

Description

[0001] DEVICE AND METHOD FOR IMAGE PROCESSING [0002]

Various embodiments relate to an electronic device and method for processing an image, and more particularly to an electronic device and method for deleting unnecessary objects among objects extracted from an image.

Artificial intelligence (AI) system is a computer system that implements human-level intelligence. Unlike existing Rule-based smart systems, AI is a system in which machines learn, judge and become smart. Artificial intelligence systems are increasingly recognized and improving their understanding of user preferences as they are used, and existing rule-based smart systems are gradually being replaced by deep-run-based artificial intelligence systems.

Artificial intelligence technology consists of element technologies using deep learning and machine learning.

Machine learning is an algorithm technology that classifies / learns the characteristics of input data by itself. Element technology is a technology that simulates the functions of human brain such as recognition and judgment using machine learning algorithms such as deep learning. Understanding, reasoning / prediction, knowledge representation, and motion control.

The various fields in which artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing, applying, and processing human language / characters, including natural language processing, machine translation, dialogue system, query response, speech recognition / synthesis, and the like. Visual understanding is a technology for recognizing and processing objects as human vision, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, and image enhancement. Inference prediction is a technique for judging and logically inferring and predicting information, including knowledge / probability based reasoning, optimization prediction, preference base planning, and recommendation. Knowledge representation is a technology for automating human experience information into knowledge data, including knowledge building (data generation / classification) and knowledge management (data use). The motion control is a technique for controlling the autonomous travel of the vehicle and the motion of the robot, and includes motion control (navigation, collision, traveling), operation control (behavior control), and the like.

Meanwhile, as multimedia technology and computer technology are developed, users can receive various services using electronic devices. Particularly, as the image processing technology advances, the user can use the image processing apparatus to receive an image in which unnecessary objects included in the image are deleted.

However, conventionally, in order to determine an unnecessary object to the user in the image, it is necessary to use a plurality of images taken at different times for the same screen, or to select an unnecessary object by using functions provided by the image processing apparatus directly There was a problem. In addition, even when an unnecessary object is determined by a user, it is difficult to repeatedly apply the image to a plurality of images. Accordingly, there is a demand for a technique in which an electronic device automatically determines an unnecessary object to a user through machine learning, and enables repeated application of a plurality of images.

Various embodiments are intended to provide an electronic device and method for automatically determining which of objects extracted from an image are to be deleted and deleting the determined object.

An electronic device according to an embodiment includes a display for displaying an image, a memory for storing one or more instructions, and a processor for executing one or more instructions stored in a memory, the processor executing one or more instructions, Extracting at least one object, determining an object to be deleted among the extracted at least one object based on the identity between the extracted at least one object and at least one object included in the user data, and deleting the determined object from the image .

A method of processing an image in an electronic device, the method comprising: extracting at least one object from the image; based on the identity between the extracted at least one object and at least one object included in the user data, Determining an object to be deleted among the extracted at least one object, and deleting the determined object from the image.

A method of processing an image using a learning model using a neural network, the method comprising: extracting at least one object from an image; extracting at least one object from at least one object and at least one Determining an object to be deleted of the extracted at least one object based on the identity between the objects in the image and deleting the determined object from the image.

1 is a schematic diagram illustrating an example in which an electronic device according to one embodiment processes an image.
2 is a flowchart illustrating an operation method of an electronic device according to an embodiment.
3 is a diagram for explaining a method for an electronic device to determine an object to be deleted using previously stored phone book data, according to an embodiment.
4 is a flow diagram illustrating a method by which an electronic device determines an object to delete based on an entry frequency, in accordance with one embodiment.
5 is a diagram for describing a method of generating a probability database in which an electronic device is known from user data, according to an embodiment.
6 is a diagram illustrating a method by which an electronic device determines an object to be deleted based on an occurrence frequency in an image related to a specific event, according to an embodiment.
7 is a flowchart illustrating a method for an electronic device to determine an object to be deleted using an object having a previously deleted history according to an exemplary embodiment.
8 is a diagram for explaining a method of determining an object to be deleted by using an object having a history of a previously deleted electronic device according to an embodiment.
9 is a diagram illustrating a user interface screen provided by an electronic device according to an embodiment.
10 and 11 are block diagrams showing the configuration of an electronic device according to an embodiment.
12 is a block diagram of a processor in accordance with one embodiment.
13 is a block diagram of a data learning unit according to an embodiment.
14 is a block diagram of a data recognition unit according to an embodiment.
15 is a diagram showing an example of learning and recognizing data by interlocking with an electronic device and a server according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram illustrating an example in which an electronic device according to one embodiment processes an image.

Referring to FIG. 1, in accordance with one embodiment, electronic device 100 may determine an object to be deleted within image 101.

The electronic device 100 may extract at least one object from the image 101 to determine the object to be deleted within the image 101. [ Here, the object refers to one object included in the image, which is distinguished from other objects in the image 101. For example, the image 101 may include a plurality of objects 110 to 116, and the electronic device 100 may extract a plurality of objects 110 to 116 from the image 101. For example,

In one embodiment, the electronic device 100 may utilize a learning model using a neural network to extract at least one object within the image 101. Here, the learning model may be a model in which the electronic device 100 uses a plurality of images as learning data to extract objects in the image. However, the present invention is not limited to this, and the electronic device 100 may receive a learning model for extracting at least one object in the image 101 from an external device. The electronic device 100 can extract at least one object included in the image 101 by using the image 101 as the input data to the learning model using the neural network.

In addition, the electronic device 100 may determine the object to be deleted in the image 101 among the extracted objects. For example, the image 101 may include first through fifth objects 110 through 114 corresponding to a person, and a sixth object 115 and seventh object 116 corresponding to the environment. If the electronic device 100 determines that the fourth object 113 and the fifth object 114 are known, the electronic device 100 can recognize the first object 110, the second object 110, 111, and the third object 112 as objects to be deleted. Also, for example, if the electronic device 100 determines that the seventh object 116 (e.g., smoke or cloud) corresponding to the environment is a previously deleted historical object, May determine the seventh object 116 as an object to be deleted.

Accordingly, the electronic device 100 can determine the object to be deleted in the image based on the user data based analysis result such as the analysis of the image data generated by the user, such as the judgment of the user's acquaintance, or the analysis result of the user's deletion history have.

According to one embodiment, the electronic device 100 may determine an object to be deleted within the image 101, based on the learning model 102 using the neural network. For example, the electronic device 100 may determine, based on the learning model 102, whether each of the plurality of objects 110 to 116 included in the image corresponds to an object to be deleted. Specifically, the electronic device 100 may determine, based on the learning model 102, the identity between each of the plurality of objects 110 to 116 and at least one object included in the user data. In addition, the electronic device 100 may determine an object to be deleted within the image 101, based on the identity determined based on the learning model 102.

The learning model 102 may be a set of algorithms that identify and / or determine objects included in an image by extracting and using various attributes contained in the image, using the results of statistical machine learning. For example, the learning model 102 may determine whether the first to fifth objects 110 to 114 corresponding to the person are acquainted, as in block 104, and determine whether the sixth object It is possible to determine whether there is a previously deleted history for the first object 115 and the seventh object 116. Further, in one embodiment, the learning model 102 may be constructed based on learning using a neural network. For example, the learning model 102 may be learned about the criteria for determining the identity between objects using the user data previously stored in the electronic device 100 as learning data. Also, the learning model 102 may include one or more layers. In addition, the learning model 102 may include one or more input channels or one or more output channels.

According to one embodiment, the electronic device 100 may delete the determined object in the image 101 once the object to be deleted is determined. For example, the electronic device 100 may replace the portion 120,121 corresponding to the object to be deleted in the image 101 with the background portion of the image, mosaic, or replace with a specific icon or predetermined image, etc. You can delete the object with. In one embodiment, the electronic device 100 uses a learned learning model (not shown) to reproduce the portion corresponding to the object to be deleted in the image 101 using a neural network, (120, 121) with the background portion of the image.

According to one embodiment, the electronic device 100 may provide the user with an image 103 in which some objects in the image 101 have been deleted.

In one embodiment, the electronic device 100 extracts at least one object in the image 101, determines which of the extracted at least one object to delete, and for each process that deletes the determined object from the image 101 A learning model using one or more corresponding neural networks can be used. However, the disclosed embodiment is not so limited, and the electronic device 100 may use at least one object in the image 101, using a learned neural network learning model to delete unnecessary objects in the image Extracts the extracted object, determines an object to be deleted from the extracted at least one object, and deletes the determined object from the image (101).

The electronic device 100 may be a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a micro server, But are not limited to, broadcast terminals, navigation, kiosks, MP3 players, digital cameras, consumer electronics, and other mobile or non-mobile computing devices. The electronic device 100 may also be a wearable device, such as a watch, glasses, hair bands, and rings, having display and data processing capabilities. However, not limited to this, the electronic device 100 may include any kind of device capable of processing images and providing processed images to the user.

2 is a flow chart illustrating a method of operating an electronic device 100 in accordance with one embodiment.

Referring to FIG. 2, at step 202, electronic device 100 extracts at least one object from an image.

According to one embodiment, the electronic device 100 may obtain an image. For example, the electronic device 100 may execute a photo album application or the like to obtain a pre-stored image. Alternatively, the electronic device 100 may obtain an image from the outside. In another example, the electronic device 100 may obtain an image from a cloud server or other electronic device. In another example, the electronic device 100 may capture an image using a camera provided in the electronic device 100. For example, the electronic device 100 may execute a camera application to capture an image.

The electronic device 100 may extract at least one object from the acquired image. In a case where a plurality of objects are included in an image, the electronic device 100 can extract a plurality of objects as individual objects by using a first object, a second object, or the like.

For example, the electronic device 100 may extract at least one object by analyzing pixels included in the image.

In another example, the electronic device 100 may extract at least one object from an image using a foreground and background recognition method. Specifically, the electronic device 100 can extract an object corresponding to the foreground or background from the image using the foreground and background recognition methods. In addition, the electronic device 100 can distinguish objects corresponding to the foreground as objects corresponding to figures, animals, objects, or the like, and extract them. If there are a plurality of objects corresponding to the persons, And the like.

According to one embodiment, the electronic device 100 may utilize a learning model using a neural network to extract at least one object from the image.

In step 204, the electronic device 100 determines, based on the identity between the extracted at least one object (first object) and at least one object (second object) contained in the user data, Decide which object to delete. Here, the identity between the first object and the second object means whether or not the first object and the second object are regarded as the same object.

The fact that the first object and the second object are the same may include not only when the first object and the second object correspond to each other but also when the first object and the second object are similar. For example, even if the characteristics of the first object and the characteristics of the second object coincide with each other within a preset range, the first object and the second object may be determined to be the same object. However, the present invention is not limited thereto.

According to one embodiment, when determining that the first object and the second object are the same object, the electronic device 100 may not delete the first object from the image. For example, when the first object and the second object included in the user data are objects corresponding to the person, the electronic device 100 determines that the first object is the user's acquaintance, and deletes the first object from the image .

According to another embodiment, when determining that the first object and the second object are the same object, the electronic device 100 may determine the first object as an object to be deleted. For example, if the second object included in the user data has a history of previously deleted, the electronic device 100 may determine the first object, which is the same as the second object, as an object to be deleted in the image.

Here, the user data may include data in which a user's input is input to a process for at least one of the steps of generating, storing, and processing data.

For example, the user data may be data directly input to the electronic device 100 by the user of the electronic device 100, data received from the external device via the input controlling the electronic device 100, ) May be data uploaded to an external device.

The user data may also include phone book data stored in the electronic device 100, images included in the photo book, data processed by the electronic device 100, and user's usage history data for the electronic device 100 . Here, the data processed by the electronic device 100 may include at least one object extracted from the image.

In another embodiment, the user data may be obtained from an external server (e.g., an SNS server, a cloud server, a content provision server, etc.). For example, the electronic device 100 may obtain user data included in the user's SNS account from the SNS server.

In one embodiment, the electronic device 100 may use a learning model with a neural network to determine which objects to delete in the image. The electronic device 100 can determine an object to be deleted in the image based on the extracted learning model constructed by using at least one object and user data as learning data. For example, the learning model may be a learned model to determine an object to be deleted in the image, based on the identity between at least one object extracted from the image and at least one object contained in the user data. The learning model can determine that at least one object extracted from the image is an object to be deleted, which is not the same as at least one object included in the user data. The learning model constructed using the user data as the learning data may be a model optimized for the user of the electronic device 100. [ Accordingly, in the case of determining an object to be deleted in the image based on the learning model based on the user data, it is possible to increase the accuracy of the determination of the object to be deleted in the image, as compared with the case of using the learning model based on the general- have.

Specifically, the electronic device 100 can learn a learning model using a neural network using user data. For example, the electronic device 100 can learn a learning model through phonebook data previously stored in the electronic device 100, image data included in a photo album, and usage history data of a user. In another example, the electronic device 100 may learn a learning model using at least one object included in user data. Also, the electronic device 100 can determine the identity between the object extracted from the image and the user's acquaintance object, using the learning result.

In another example, the electronic device 100 may utilize a learning model received from an external device. For example, the electronic device 100 may transmit user data to an external device. The external device can learn the learning model using the user data received from the electronic device 100 and transmit the learned learning model to the electronic device 100. [ In addition, the electronic device 100 may determine, based on the received learning model, of the at least one object extracted from the image to be deleted.

In step 206, the electronic device 100 deletes the determined object from the image. The electronic device 100 may delete, from the image, an object determined as an object to be deleted among at least one object extracted from the image. For example, the electronic device 100 may delete the determined object from the image by filling the portion corresponding to the object to be deleted in the image with another portion. Alternatively, when the object to be deleted is an object corresponding to a person, the electronic device 100 may delete the object by a method of displaying the object to be deleted with a mosaic.

According to one embodiment, the electronic device 100 may use a learning model with a neural network to delete the determined object from the image. The electronic device 100 can correct the deleted portion of the object using the learning model. For example, the electronic device 100 may use the learning model to generate pixel information to be substituted for the deleted portion of the object. Alternatively, the electronic device 100 can generate pixel information to be substituted in the deleted portion of the object, using a peripheral pixel information-based data playback method using a learning model.

According to one embodiment, when there are a plurality of objects determined to be deleted, the electronic device 100 may delete each determined object in different ways depending on each determined object. For example, the first object to be deleted can be deleted by a method of replacing based on the pixel information of the surrounding portion, and the second object to be deleted can be deleted by a method of displaying with a mosaic.

3 is a diagram for explaining how the electronic device 100 determines an object to be deleted using previously stored phone book data, according to an embodiment.

According to one embodiment, the user data may include phone book data 301 pre-stored in electronic device 100. For example, the electronic device 100 may determine an object to delete based on the identity between at least one object extracted from the image and at least one object contained in the phonebook data 301.

3, the electronic device 100 may generate a knowledge database 302 from the phonebook data 301, according to one embodiment. Electronic device 100 may create an acquaintance database 302 that includes an object corresponding to at least one person (e.g., object 312 in FIG. 3).

For example, the electronic device 100 may extract an object 310 corresponding to a person from the image 311 included in the phone book data 301. [ Here, the embodiments described in step 202 of FIG. 2 may be applied to a method by which the electronic device 100 extracts an object corresponding to a person. In addition, the electronic device 100 may include the object 310 corresponding to the person in the acquaintance database 302 as the object 312 corresponding to the acquaintance.

If the electronic device 100 determines that the object extracted from the image is not the same as the object included in the database 302 based on the acquaintance database 302 generated from the phone book data 301, The extracted object can be determined as an object to be deleted. Here, the method of determining the identity between at least two objects may correspond to the embodiments described in step 204 of FIG.

According to one embodiment, the electronic device 100 can update the database 302, which is where the phone book data 301 is updated.

4 is a flow diagram illustrating a method by which an electronic device 100 determines an object to delete based on an entry frequency, in accordance with one embodiment.

Referring to FIG. 4, in step 402, the electronic device 100 extracts at least one object from an image. Step 402 may correspond to step 202 of FIG. 2, and therefore detailed description is omitted.

In step 404, the electronic device 100 determines an entry frequency corresponding to each of at least one object included in the user data. The electronic device 100 may determine an occurrence frequency for each of the at least one object extracted from the at least one image included in the user data.

In one embodiment, the electronic device 100 may determine an occurrence frequency corresponding to each of the objects contained in the user data, based on the determination of identity between at least two objects included in the user data.

For example, when the electronic device 100 determines two objects extracted from different images included in the user data as the same object, the electronic device 100 transmits the user data Can determine the number of occurrences corresponding to each of the objects included in the object.

Here, the method of determining the identity between at least two objects can be applied to the embodiments described in step 204 of FIG.

In step 406, the electronic device 100 determines the object to delete based on the identity between the extracted at least one object and at least one object included in the user data and the determined occurrence frequency.

For example, if the first one of the at least one object extracted at step 402 is determined to be the same object as the second object at which the determined frequency of occurrence is determined at step 404, You can choose not to delete the object. Conversely, if the third of the at least one object extracted at step 402 is not the same as any one of the objects whose determined frequency at step 404 meets a predetermined criterion, then the electronic device 100 sends the third object You can decide to delete the object.

In one embodiment, the predetermined criterion may be the minimum value of the frequency of occurrence to be determined by the user's acquaintance. In addition, the predetermined criteria may be a default value set in the electronic device 100. [ Alternatively, the predetermined criteria may be a value determined by user input. For example, the electronic device 100 may provide a user interface for determining a predetermined reference value. Alternatively, the predetermined criteria can be determined based on the learning model using the neural network.

Further, in step 406, the embodiments described in step 204 of FIG. 2 may be applied.

In step 408, the electronic device 100 deletes the determined object from the image. Step 408 may correspond to step 206 of FIG. 2, so a detailed description will be omitted.

5 is a diagram for describing a method of generating a probability database in which the electronic device 100 is known from user data, according to one embodiment.

Referring to FIG. 5, in accordance with one embodiment, electronic device 100 may generate a knowing probability database 500 using user data. For example, the electronic device 100 uses the image data included in the phone book data 501 and the photo album 502 previously stored in the electronic device 100, and calculates the appearance frequency 510 and the background probability 511). ≪ / RTI >

According to one embodiment, the electronic device 100 may determine an occurrence frequency of each of the objects corresponding to a person included in the user data. Specifically, the electronic device 100 can extract at least one object corresponding to the person from the image data included in the phone book data 501 and the photo album 502. [ In addition, the electronic device 100 may determine an occurrence frequency 510 for each of the objects based on the identity between the extracted objects.

According to one embodiment, the electronic device 100 may determine the probability of an acquaintance 511 for each of the objects, based on predetermined criteria from the entropy frequency 510. Here, the preset reference may be a default value set in the electronic device 100. [ Alternatively, the predetermined criteria may be a value determined by user input.

For example, the electronic device 100 can determine the probability of acquiring an object included in the phonebook data 501 as 99%. Alternatively, the electronic device 100 may store the ratio of the frequency of appearance of each specific object in the entire object included in the user data to the probability 511 of the recognition probability. Alternatively, the electronic device 100 may determine the recognition probability 511 by classifying the appearance frequency into predetermined intervals and adding additional weights to the intervals. The electronic device 100 may also provide a user interface for setting criteria based on user input.

According to one embodiment, the acquaintance probability 511 may be stored as a percentage (%) value as shown in FIG. 5, or may be stored as a value between 0 and 1, but is not limited thereto.

6 is a diagram illustrating a method by which an electronic device 100 determines an object to be deleted based on an occurrence frequency in an image associated with a specific event, according to an embodiment.

According to one embodiment, the electronic device 100 may use a plurality of images associated with a particular event to determine an object in the image that is not associated with a particular event to be deleted.

According to one embodiment, the electronic device 100 may obtain a plurality of images associated with a particular event from user data. The electronic device 100 may obtain images related to a particular event, in a manner that classifies images that satisfy certain conditions from user data. For example, the electronic device 100 may obtain images associated with an event specified on the date or period of time that the image was taken. Alternatively, the electronic device 100 may obtain images associated with the event specified on condition that it includes a common object.

Referring to FIG. 6, for example, the electronic device 100 may obtain a plurality of images 601 associated with a "chorus" based on a user input of "chorus. &Quot; Specifically, the electronic device 100 may obtain images 601 taken from the user data on the date the "telescope" occurred.

According to one embodiment, the electronic device 100 may determine an entry frequency corresponding to each of at least one object included in a plurality of images 601 associated with a particular event. The electronic device 100 may apply the embodiments described in step 404 of FIG. 4 to determine an occurrence frequency by analyzing a plurality of images associated with a particular event.

According to one embodiment, the electronic device 100 may determine an object to delete based on the identity between the extracted at least one object and at least one object included in the plurality of images associated with the particular event, and the determined occurrence frequency .

For example, the electronic device 100 may generate a "portrait" related person database 600 by analyzing a plurality of images 601 associated with " The "person-to-person" related person database 600 can be generated by a method corresponding to the generation method of the recognition probability database 500 described in FIG.

Further, for example, the electronic device 100 may determine an object 603 to be deleted in one of the plurality of images 601 (602). Specifically, the electronic device 100 can use the generated related personality probability database 600 to determine objects 603 whose objects are less than 10% probability of being related persons to be deleted.

According to one embodiment, the electronic device 100 may provide the user with an object 603 to delete. For example, the electronic device 100 may display the object 603 to be deleted, which is determined in one image 602, on the display unit.

In addition, the electronic device 100 may provide the user with images 601 and an image 604 from which the determined object has been deleted. For example, electronic device 100 may display images 601 and an image 604 from which the determined object has been deleted through an external display device connected to electronic device 100 or electronic device 100. [

7 is a flowchart illustrating a method for an electronic device 100 to determine an object to be deleted using an object having a previously deleted history according to an embodiment.

Referring to FIG. 7, in step 702, the electronic device 100 extracts at least one object from an image. Step 702 may correspond to step 202 of FIG. 2, and thus a detailed description will be omitted.

In step 704, the electronic device 100 may determine the object to delete based on the identity between the extracted at least one object and the object having the deleted history. For example, when the first object is deleted from the first image and the second object is determined to be the same object as the first object in the second image, the electronic device 100 can determine the second object as an object to be deleted have.

In one embodiment, an object having a previously deleted history may be an object deleted based on user input.

In step 704, the embodiments described in step 204 of FIG. 2 may be applied to how the electronic device 100 determines the identity.

According to one embodiment, the electronic device 100 may use the user data to determine a previously deleted historical object.

In step 706, the electronic device 100 deletes the determined object from the image. Step 706 may correspond to step 206 of FIG. 2, and therefore a detailed description will be omitted.

FIG. 8 is a diagram for explaining in detail a method of determining an object to be deleted using an object having a history of a previously deleted electronic device 100 according to an embodiment.

According to one embodiment, the electronic device 100 is configured to generate at least one (1, 2, 3, 4, 5, 6, Can be determined as an object to be deleted.

8, the first image 801, the second image 802, and the third image 803 are images in which the same object 804 is photographed at various angles.

The electronic device 100 may obtain a user input to delete the first object 811 in the first image 801. [ In addition, the electronic device 100 may store the deleted history of the first object 811 in a storage device included in the electronic device 100 or an external storage device. In addition, the electronic device 100 may extract the second object 812 from the second image 802. The electronic device 100 may determine the second object 812 as the same object as the first object 811 and may determine the second object 812 identical to the first object 811 as the object to be deleted. The electronic device 100 may determine the third object 813 as an object to be deleted, for the third image 803 in the same manner as in the second image.

According to the disclosed embodiments, the electronic device 100 automatically deletes the object having the deleted history from a plurality of images, thereby making it unnecessary for the user to repeatedly delete the same object included in the plurality of images Can be reduced.

9 is a view showing a user interface screen provided by the electronic device 100 according to an embodiment.

9, in accordance with one embodiment, electronic device 100 displays a first image 901 and a second image 904 from which objects 902, 903 determined in the first image have been deleted, on the display . In addition, the electronic device 100 may store at least one of the first image 901 and the second image 904 based on user input.

For example, the electronic device 100 may display 'save' objects 911 and 913 and 'delete' objects 912 and 914 on the user interface 900 screen. The electronic device 100 may store the first image 901 when receiving an input to select a 'save' object 911. In addition, the electronic device 100 may delete the first image 901 when receiving an input to select the 'delete' object 912. Electronic device 100 may store a second image 904 when receiving an input to select a 'save' object 913. [ In addition, the electronic device 100 may delete the second image 904 when receiving an input to select the 'delete' object 914.

 Meanwhile, according to one embodiment, when the object is deleted, the electronic device 100 may include the deleted object in the user data. For example, if the second image 904 with the objects 902 and 903 deleted is stored, the electronic device 100 may include the deleted objects 902 and 903 in the user data. Accordingly, the electronic device 100 can determine the object to be deleted from the image, based on the identity between the extracted object and the deleted object 902, 903 from the acquired image.

10 and 11 are block diagrams showing the configuration of an electronic device according to an embodiment.

10, an electronic device 100 according to one embodiment may include a memory 1100, a display portion 1210, and a processor 1300. [

However, not all of the components shown in Fig. 10 are essential components of the electronic device 100. Fig. The electronic device 100 may be implemented by more components than the components shown in Fig. 10, or the electronic device 100 may be implemented by fewer components than those shown in Fig.

11, the electronic device 100 according to one embodiment includes an output unit 1200, a communication unit 1500, a memory unit 1100, a display unit 1210, and a processor 1300, A sensing unit 1400, an A / V input unit 1600, and a user input unit 1700.

The memory 1100 may store a program for processing and controlling the processor 1300 and may store data input to or output from the electronic device 100. [ The memory 1100 may also store user data.

In one embodiment, the memory 1100 may store at least one instruction for processing an image acquired by the electronic device 100. The memory 1100 extracts at least one object from the image, determines an object to be deleted among at least one object extracted based on the identity between the at least one object and the object included in the user data, and deletes the determined object For example.

In one embodiment, the memory 1100 may store a learning model based on one or more neural networks for extracting at least one object from the image.

In one embodiment, the memory 1100 may store a learning model based on one or more neural networks for determining objects to delete in the image.

In one embodiment, the memory 1100 may store a learning model based on one or more neural networks for replacing the portion corresponding to the object to be deleted in the image with the portion corresponding to the background of the image.

The memory 1100 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a RAM (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , An optical disc, and the like.

The programs stored in the memory 1100 may be classified into a plurality of modules according to their functions, for example, a UI module 1110, a touch screen module 1120, a notification module 1130, .

The UI module 1110 can provide a specialized UI, a GUI, and the like, which are interlocked with the electronic device 100 for each application. The touchscreen module 1120 may detect a touch gesture on the user's touch screen and may pass information to the processor 1300 about the touch gesture. The touch screen module 1120 according to one embodiment can recognize and analyze the touch code. The touch screen module 1120 may be configured as separate hardware including a controller.

The notification module 1130 may generate a signal for notifying the occurrence of an event of the electronic device 100. [ Examples of events generated in the electronic device 100 include call signal reception, message reception, key signal input, schedule notification, and the like. The notification module 1130 may output a notification signal in the form of a video signal through the display unit 1210 or may output a notification signal in the form of an audio signal through the sound output unit 1220, It is possible to output a notification signal in the form of a vibration signal.

The output unit 1200 may output an audio signal or a video signal or a vibration signal and the output unit 1200 may include a display unit 1210, an acoustic output unit 1220, and a vibration motor 1230 have.

The display unit 1210 displays and outputs information processed in the electronic device 100. [ Specifically, the display unit 1210 can display and output an image for processing in the electronic device 100. [ Also, the display unit 1210 may be determined as an object to be deleted in the processor 1300, and the determined object may display and output the deleted image.

In addition, the display unit 1210 can display, in response to a user's input, a user interface for executing operations related to the response.

The audio output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1100. Also, the sound output unit 1220 outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, alarm sound) performed in the electronic device 100. [

The processor 1300 typically controls the overall operation of the electronic device 100. For example, the processor 1300 may include a user input unit 1700, an output unit 1200, a sensing unit 1400, a communication unit 1500, an A / V input unit 1700 ) Can be generally controlled. The processor 1300 may also perform the functions of the electronic device 100 described in Figures 1-9 by executing programs stored in the memory 1100. [ The processor 1300 may also process the image acquired by the electronic device 100 by executing at least one instruction stored in the memory 1100. [ Processor 1300 may include one or more processors.

Specifically, the processor 1300 may extract at least one object from the image displayed on the display unit 1210.

In one embodiment, the processor 1300 may extract at least one object from the image by analyzing the pixels contained in the image.

In another embodiment, the processor 1300 may extract at least one object from the image using a learning model using a neural network.

In addition, the processor 1300 may determine an object to be deleted from at least one extracted object. In one embodiment, the processor 1300 may determine which of the extracted at least one object to delete based on the identity between the extracted at least one object and at least one object contained in the user data. The processor 1300 may utilize the user data stored in the memory 1100. In another embodiment, the processor 1300 may determine the object to delete based on the identity between the extracted at least one object and at least one object contained in the phonebook data. For example, the processor 1300 may extract at least one object from the image data included in the phone book data stored in the memory 1100, and use the extracted object to determine an object to be deleted from the image. In another embodiment, the processor 1300 may determine the object to delete based on the identity between the extracted at least one object and the object having the previously deleted history. Here, the object having the history to be deleted may be deleted based on the user input. For example, the processor 1300 may obtain the deleted object based on the user input received via the user input 1700 and determine the object to delete based on the identity between the extracted at least one object and the acquired object .

The processor 1300 may delete the object determined as the object to be deleted from the image.

In one embodiment, the processor 1300 may delete the determined object from the image by filling the portion corresponding to the object to be deleted in the image with another portion. According to one embodiment, the processor 1300 may use a learned neural network learning model to delete the determined object from the image.

In addition, the processor 1300 can control the deleted object in the image to be included in the user data. For example, the processor 1300 may store the deleted objects in the image in memory 1100. [

In one embodiment, the processor 1300 can learn a learning model using a neural network with user data as learning data. The learning model may include one or more layers. The learning model may also include one or more input and output channels. The processor 1300 may determine an identity between at least one object extracted and at least one object included in the user data based on a learning model using a neural network. In addition, the processor 1300 can determine an object to be deleted from at least one object extracted from the image, based on the learning result using the learning model.

In one embodiment, the processor 1300 may determine an occurrence frequency for each of at least one object included in the user data. For example, the processor 1300 may determine an occurrence frequency based on the identity between at least two objects included in the user data. In addition, the processor 1300 may determine an object to delete based on the identity and the occurrence frequency between the at least one object extracted and the at least one object contained in the user data.

In one embodiment, the processor 1300 may obtain a plurality of images associated with a particular event. In addition, the processor 1300 may determine an occurrence frequency for each of at least one object included in a plurality of images associated with a particular event. In addition, the processor 1300 may determine the object to delete based on the identity and the occurrence frequency between the at least one object extracted and the at least one object included in the plurality of images associated with the particular event.

In one embodiment, the processor 1300 controls the display unit 1210 to display the image and the determined object are deleted, and controls the display unit 1210 to display the image based on the user input received by the communication unit 1500 or the user input unit 1700. [ And an image in which the determined object has been deleted.

The sensing unit 1400 may sense a state of the electronic device 100 or a state around the electronic device 100 and may transmit sensed information to the processor 1300. [

The sensing unit 1400 includes a magnetism sensor 1410, an acceleration sensor 1420, an on / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, (GPS) 1460, an air pressure sensor 1470, a proximity sensor 1480, and an RGB sensor (illuminance sensor) 1490, for example. The function of each sensor can be intuitively deduced from the name by those skilled in the art, so a detailed description will be omitted.

The communication unit 1500 may include one or more components that allow the electronic device 100 to communicate with other devices (not shown) and a server (not shown). Other devices (not shown) may be computing devices, such as electronic device 100, or may be sensing devices, but are not limited thereto. For example, the communication unit 1500 may include a local communication unit 1510, a mobile communication unit 1520, and a broadcast receiving unit 1530.

The short-range wireless communication unit 1510 includes a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a Near Field Communication unit, a WLAN communication unit, a Zigbee communication unit, IrDA, an infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, an UWB (ultra wideband) communication unit, an Ant + communication unit, and the like.

The mobile communication unit 1520 transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The broadcast receiving unit 1530 receives broadcast signals and / or broadcast-related information from outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The electronic device 100 may not include the broadcast receiver 1530 according to an embodiment.

In addition, the communication unit 1500 can send and receive information required for executing an operation related to a response message to another apparatus (not shown) and a server (not shown) by requesting a response message to the user's voice input.

The A / V (Audio / Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610, a microphone 1620, and the like. The camera 1610 can obtain an image frame such as a still image or a moving image through the image sensor in a video communication mode or a shooting mode. The image captured through the image sensor may be processed through the processor 1300 or a separate image processing unit (not shown). The image photographed by the camera 1610 can be used as user data.

The microphone 1620 receives an external acoustic signal and processes it as electrical voice data. For example, the microphone 1620 may receive acoustic signals from an external device or user. The microphone 1620 may receive the user's voice input. The microphone 1620 may use various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 1700 means means for the user to input data for controlling the electronic device 100. [ For example, the user input unit 1700 may include a key pad, a dome switch, a touch pad (contact type capacitance type, pressure type resistive type, infrared ray detection type, surface ultrasonic wave conduction type, A tension measuring method, a piezo effect method, etc.), a jog wheel, a jog switch, and the like, but is not limited thereto.

12 is a block diagram of a processor 1300 in accordance with one embodiment.

12, the processor 1300 according to an embodiment may include a data learning unit 1310 and a data recognition unit 1320.

The data learning unit 1310 can learn a criterion for determining the identity between objects in order to determine an object to be deleted in the image. The data learning unit 1310 can learn what data is used to determine the identity between the object extracted from the image and the object included in the user data and how to determine the identity between the objects using the data . The data learning unit 1310 may acquire data to be used for learning and apply the obtained data to a data recognition model to be described later so as to learn an identity determination criterion between objects for determining an object to be deleted.

The data recognition unit 1320 can determine the identity between at least two objects. The data recognition unit 1320 can determine the identity between at least two objects using the learned data judgment model. The data recognition unit 1320 analyzes the characteristics of each object according to a preset reference based on learning and uses the data decision model as the input value to determine the identity between at least two objects have. Also, the resultant value output by the data judgment model with the analyzed data as an input value can be used to update the data judgment model.

At least one of the data learning unit 1310 and the data recognizing unit 1320 may be manufactured in the form of at least one hardware chip and mounted on the electronic device. For example, at least one of the data learning unit 1310 and the data recognition unit 1320 may be fabricated in the form of a dedicated hardware chip for artificial intelligence (AI) or a conventional general purpose processor Or application processor) or a graphics-only processor (e.g., a GPU), and may be mounted on various electronic devices as described above.

In this case, the data learning unit 1310 and the data recognizing unit 1320 may be mounted on one electronic device or on separate electronic devices, respectively. For example, one of the data learning unit 1310 and the data recognizing unit 1320 may be included in the electronic device, and the other may be included in the server. The data learning unit 1310 and the data recognition unit 1320 may provide the model information constructed by the data learning unit 1310 to the data recognition unit 1320 via a wire or wireless communication, 1320 may be provided to the data learning unit 1310 as additional learning data.

At least one of the data learning unit 1310 and the data recognition unit 1320 may be implemented as a software module. When at least one of the data learning unit 1310 and the data recognition unit 1320 is implemented as a software module (or a program module including instructions), the software module may be read by a computer, And may be stored in non-transitory computer readable media. Also, in this case, the at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an Operating System (OS), and some of the software modules may be provided by a predetermined application.

13 is a block diagram of a data learning unit 1310 according to an embodiment.

13, a data learning unit 1310 according to an embodiment includes a data acquisition unit 1310-1, a preprocessing unit 1310-2, a learning data selection unit 1310-3, a model learning unit 1310 -4) and a model evaluation unit 1310-5.

The data acquisition unit 1310-1 may acquire data necessary for determining the identity between objects in order to determine objects to be deleted in the image. The data acquisition unit 1310-1 may acquire data necessary for learning for determining the identity between objects.

The data acquisition unit 1310-1 may acquire an image. . For example, the data acquisition unit 1310-1 may acquire an image through the camera of the electronic device 100 including the data learning unit 1310. [ Alternatively, the data acquisition unit 1310-1 may acquire data from at least one external device capable of communicating with the electronic device 100. [ The image acquired by the data acquisition unit 1310-1 may include at least one object.

In addition, the data acquisition unit 1310-1 may acquire user data for recognizing at least one object in the acquired image. Here, the user data may be data associated with a user of the electronic device 100, and the user's input to the process for at least one of the steps of generating, storing, and processing the data.

For example, the user data may be data directly input to the electronic device 100 by the user of the electronic device 100, data received from the external device via the input controlling the electronic device 100, ) May be data uploaded to an external device.

More specifically, the user data includes the phone book data stored in the electronic device 100, the images contained in the photo book, the data processed by the electronic device 100, and the user's usage history data for the electronic device 100 . Here, the data processed by the electronic device 100 may include at least one object extracted from the image.

The preprocessing unit 1310-2 can preprocess the acquired data so that the acquired data can be used for learning for determining the identity between objects. The preprocessing unit 1310-2 can process the acquired data into a predetermined format so that the model learning unit 1310-4, which will be described later, can utilize the data acquired for learning for the determination of equality.

For example, the preprocessing unit 1310-2 may extract at least one object included in the acquired image and process the data into data having a predetermined format. In addition, the preprocessing unit 1310-2 may generate a database for determining the acquaintance of the user among at least one object included in the acquired image, based on the user data. For example, the preprocessing unit 1310-2 extracts an object corresponding to a person from an image included in at least one of phone book data and photo album of the electronic device 100, Lt; / RTI > Also, the preprocessing unit 1310-2 can determine the frequency of occurrence of each object according to the frequency of the extracted object in the phonebook data and the photo album. Also, the preprocessing unit 1310-2 can determine the probability that each extracted object is the user's identity based on the frequency of appearance.

The learning data selection unit 1310-3 can select data necessary for learning from the preprocessed data. The selected data may be provided to the model learning unit 1310-4. The learning data selection unit 1310-3 can select data necessary for learning from the preprocessed data according to a predetermined criterion for determining the identity between objects. The learning data selection unit 1310-3 can also select data according to a predetermined criterion by learning by the model learning unit 1310-4, which will be described later.

For example, the learning data selection unit 1310-3 may include at least one object extracted from the image generated by the preprocessing unit 1310-2, Can be used.

The model learning unit 1310-4 can learn a criterion on how to determine the identity between objects based on the learning data. Also, the model learning unit 1310-4 may learn a criterion about which learning data should be used for determining the identity between objects.

For example, the model learning unit 1310-4 can analyze characteristics of each object to learn the determination of identity between at least one object extracted from the acquired image and an object included in the knowledge database. In addition, if the two objects are identical, the model learning unit 1310-4 can learn how the analyzed data of the characteristics of the respective objects coincide with each other.

Also, the model learning unit 1310-4 can learn the data recognition model used for the determination of the identity using the learning data. In this case, the data recognition model may be a pre-built model. For example, the data recognition model may be a pre-built model that receives basic learning data (e.g., a sample image, etc.).

The data recognition model can be constructed considering the application field of the judgment model, the purpose of learning, or the computer performance of the apparatus. The data recognition model may be, for example, a model based on a neural network. For example, models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Bidirectional Recurrent Deep Neural Network (BRDNN) may be used as a data recognition model, but the present invention is not limited thereto.

According to various embodiments, when there are a plurality of data recognition models that are built in advance, the model learning unit 1310-4 may use a data recognition model to learn a data recognition model having a large relation between input learning data and basic learning data You can decide. In this case, the basic learning data may be pre-classified according to the type of data, and the data recognition model may be pre-built for each data type. For example, the basic learning data may be pre-classified by various criteria such as an area where the learning data is generated, a time at which the learning data is generated, a size of the learning data, a genre of the learning data, a creator of the learning data, .

In addition, the model learning unit 1310-4 can learn a data recognition model using, for example, a learning algorithm including an error back-propagation method or a gradient descent .

Also, the model learning unit 1310-4 can learn the data recognition model through, for example, supervised learning using the learning data as an input value. In addition, the model learning unit 1310-4 can learn, for example, the type of data necessary for the determination of identity without any guidance, by performing unsupervised learning that finds a criterion for determining the identity, The data recognition model can be learned. Also, the model learning unit 1310-4 can learn the data recognition model through reinforcement learning using, for example, feedback as to whether or not the result of the determination of conformity according to learning is correct.

Further, when the data recognition model is learned, the model learning unit 1310-4 can store the learned data recognition model. In this case, the model learning unit 1310-4 can store the learned data recognition model in the memory of the electronic device including the data recognition unit 1320. [ Alternatively, the model learning unit 1310-4 may store the learned data recognition model in a memory of the electronic device including the data recognition unit 1320 to be described later. Alternatively, the model learning unit 1310-4 may store the learned data recognition model in the memory of the server connected to the electronic device and the wired or wireless network.

In this case, the memory in which the learned data recognition model is stored may also store, for example, instructions or data associated with at least one other component of the electronic device. The memory may also store software and / or programs. The program may include, for example, a kernel, a middleware, an application programming interface (API), and / or an application program (or "application").

The model evaluating unit 1310-5 inputs the evaluation data to the data recognition model, and when the judgment result outputted from the evaluation data does not satisfy the predetermined criterion, the model evaluating unit 1310-5 can allow the model learning unit 1310-4 to learn again have. In this case, the evaluation data may be predetermined data for evaluating the data recognition model.

For example, the evaluation data may include data regarding at least two objects and whether or not they are the same between respective objects.

For example, the model evaluating unit 1310-5 determines whether or not a predetermined criterion is satisfied when the number or the ratio of the evaluation data for which the determination result is not correct out of the determination results of the learned data recognition model for the evaluation data exceeds a predetermined threshold value It can be evaluated as unsatisfactory. For example, when the predetermined criterion is defined as a ratio of 2%, when the learned data recognition model outputs a wrong determination result to the evaluation data exceeding 20 out of 1000 evaluation data, the model evaluation unit 1310-5 outputs, Can be estimated that the learned data recognition model is not suitable.

According to one embodiment, the model evaluation unit 1310-5 may evaluate the data recognition model using the result of the determination of the identity provided in the recognition result provider 1320-4 included in the data recognition unit 1320 have. For example, the model evaluating unit 1310-5 can evaluate the data recognition model using the result of the determination of the identities determined by applying the data selected from the user data to the data recognition model in the recognition result providing unit 1320-4 .

On the other hand, when there are a plurality of learned data recognition models, the model evaluation unit 1310-5 evaluates whether each of the learned data recognition models satisfies a predetermined criterion, Can be determined as a model. In this case, when there are a plurality of models satisfying the predetermined criterion, the model evaluating unit 1310-5 can determine any one or a predetermined number of models previously set in the order of higher evaluation scores as the final data recognition model.

The data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-4 in the data learning unit 1310, -5) may be fabricated in at least one hardware chip form and mounted on an electronic device. For example, at least one of the data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 One may be made in the form of a dedicated hardware chip for artificial intelligence (AI), or may be fabricated as part of a conventional general purpose processor (e.g., a CPU or application processor) or a graphics dedicated processor (e.g., GPU) And may be mounted on various electronic devices.

The data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, Or may be mounted on separate electronic devices, respectively. For example, some of the data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 May be included in the electronic device, and the rest may be included in the server.

At least one of the data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5 Software module. At least one of the data acquisition unit 1310-1, the preprocessing unit 1310-2, the learning data selection unit 1310-3, the model learning unit 1310-4, and the model evaluation unit 1310-5, (Or a program module including an instruction), the software module may be stored in a computer-readable, readable non-transitory computer readable media. Also, in this case, the at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an Operating System (OS), and some of the software modules may be provided by a predetermined application.

FIG. 14 is a block diagram of a data recognition unit 1320 according to an embodiment.

14, a data recognition unit 1320 according to an embodiment includes a data acquisition unit 1320-1, a preprocessing unit 1320-2, a recognition data selection unit 1320-3, 1320-4 and a model updating unit 1320-5.

The data acquiring unit 1320-1 may acquire data for determining the identity between objects in order to determine objects to be deleted in the image, and the preprocessing unit 1320-2 may acquire data The acquired data can be preprocessed so that it can be used. The preprocessing unit 1320-2 can process the acquired data into a predetermined format so that the recognition result providing unit 1320-4, which will be described later, can use the data acquired for the determination of the inter-object identity.

The recognition data selection unit 1320-3 can select data required for determining the identity between objects among the preprocessed data. The selected data may be provided to the recognition result provider 1320-4. The recognition data selection unit 1320-3 can select some or all of the preprocessed data according to a predetermined criterion for determining the identity between objects. The recognition data selection unit 1320-3 can also select data according to a predetermined criterion by learning by the model learning unit 1310-4, which will be described later. For example, the recognition data selection unit 1320-3 can select a database extracted from the acquired image and a user's knowledge as data necessary for determining the identity between objects.

The recognition result providing unit 1320-4 can determine the identity between the objects by applying the selected data to the data recognition model. The recognition result providing unit 1320-4 can provide a determination result according to the purpose of data determination. The recognition result providing unit 1320-4 can apply the selected data to the data recognition model by using the data selected by the recognition data selecting unit 1320-3 as an input value. In addition, the recognition result can be determined by the data recognition model. For example, the recognition result providing unit 1320-4 may use a database for determining the object extracted from the image and the user's knowledge as an input value. The recognition result providing unit 1320-4 may determine the same object as at least one object included in the database for determining the acquaintance of the user among the at least one object extracted from the image as the acquaintance of the user. In addition, the recognition result providing unit 1320-4 may determine at least one object extracted from the image as an object to be deleted, which is not the same as at least one object included in the database for determining the user's acquaintance . In addition, the recognition result providing unit 1320-4 may provide an object determined by the user's knowledge and a result of the object to be deleted, among at least one object extracted from the image. For example, the recognition result providing unit 1320-4 can distinguish objects to be deleted from objects determined by the user's knowledge among at least one object extracted from the image. In addition, the recognition result providing unit 1320-4 can provide information on an object determined by the user's knowledge and at least one of the at least one object extracted from the image in a text format. In addition, the recognition result providing unit 1320-4 can provide an object determined by the user's knowledge and an object to be deleted in the image format among at least one object extracted from the image. The recognition result providing unit 1320-4 may be provided in the form of a command for controlling deletion of an object determined as an object to be deleted among at least one object extracted from the image (for example, application execution command, module function execution command, etc.) can do.

The model updating unit 1320-5 can update the data recognition model based on the evaluation of the determination result provided by the recognition result providing unit 1320-4. For example, the model updating unit 1320-5 provides the determination result provided by the recognition result providing unit 1320-4 to the model learning unit 1310-4 so that the model learning unit 1310-4 The data recognition model can be updated. The data acquisition unit 1320-1, the preprocessing unit 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-4 in the data recognizing unit 1320 1320-5 may be fabricated in at least one hardware chip form and mounted on an electronic device. For example, among the data acquisition unit 1320-1, the preprocessing unit 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-5 At least one may be made in the form of a dedicated hardware chip for artificial intelligence (AI) or as part of a conventional general purpose processor (e.g. CPU or application processor) or a graphics dedicated processor (e.g. GPU) It may be mounted on one electronic device.

The data acquiring unit 1320-1, the preprocessing unit 1320-2, the recognition data selecting unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-5 May be mounted on an electronic device, or may be mounted on separate electronic devices, respectively. For example, among the data acquisition unit 1320-1, the preprocessing unit 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-5 Some may be included in the electronic device, and the rest may be included in the server.

At least one of the data acquisition unit 1320-1, the preprocessing unit 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-5 May be implemented as a software module. At least one of the data acquisition unit 1320-1, the preprocessing unit 1320-2, the recognition data selection unit 1320-3, the recognition result providing unit 1320-4, and the model updating unit 1320-5, When implemented in a module (or program module including an instruction), the software module may be stored in a computer-readable, non-transitory computer readable medium. Also, in this case, the at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of the at least one software module may be provided by an Operating System (OS), and some of the software modules may be provided by a predetermined application.

15 is a diagram showing an example of learning and recognizing data by interlocking with the electronic device 1000 and the server 2000 according to the embodiment.

Referring to FIG. 15, the server 2000 can learn the criteria for determining the identity between objects, and the electronic device 1000 can determine the identity between objects based on the learning result by the server 2000. [

In this case, the model learning unit 2340 of the server 2000 can perform the function of the data learning unit 1310 shown in FIG. The model learning unit 2340 of the server 2000 can learn what data to use for determining the identity between objects and how to determine the identity between objects using the data. The model learning unit 2340 can acquire data to be used for learning and apply the obtained data to a data recognition model to be described later to learn a criterion for determining the identity between objects.

In addition, the recognition result provider 1320-4 of the electronic device 1000 applies the data selected by the recognition data selector 1320-3 to the data recognition model generated by the server 2000 to determine the identity between the objects It can be judged. For example, the recognition result providing unit 1320-4 transmits the data selected by the recognition data selecting unit 1320-3 to the server 2000. When the server 2000 selects the recognition data selecting unit 1320-3, May be applied to the data recognition model to request the determination of identity between objects. In addition, the recognition result providing unit 1320-4 may receive from the server 2000 information about the identities of the objects determined by the server 2000. [

Alternatively, the recognition result provider 1320-4 of the electronic device 1000 receives the data recognition model generated by the server 2000 from the server 2000, and uses the received data recognition model to determine the identity between the objects It can be judged. In this case, the recognition result providing unit 1320-4 of the electronic device 1000 applies the data selected by the recognition data selecting unit 1320-3 to the data recognition model received from the server 2000 and deletes it in the image The object can be judged.

One embodiment may also be implemented in the form of a recording medium including instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

Also, in this specification, the term "part" may be a hardware component such as a processor or a circuit, and / or a software component executed by a hardware component such as a processor.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

Claims (20)

An electronic device for processing an image,
A display unit for displaying the image;
A memory for storing one or more instructions; And
And a processor executing the one or more instructions stored in the memory,
The processor executing the one or more instructions,
Extracting at least one object from the image,
Determining an object to be deleted among the extracted at least one object based on the identity between the extracted at least one object and at least one object included in the user data,
And deletes the determined object from the image. The method according to claim 1,
Wherein the processor, by executing the one or more instructions,
And determines an identity between the extracted at least one object and at least one object included in the user data based on a result of learning the identity determination using the user data. The method according to claim 1,
Wherein the user data includes phonebook data previously stored in the electronic device,
Wherein the processor, by executing the one or more instructions,
Determine an object to delete based on an identity between the extracted at least one object and at least one object contained in the phonebook data. The method according to claim 1,
The processor executing the one or more instructions,
Determining an appearance frequency for each of at least one object included in the user data,
Determine an object to delete based on the identity between the extracted at least one object and at least one object contained in the user data and the occurrence frequency. The method according to claim 1,
The processor executing the one or more instructions,
Acquiring a plurality of images associated with a specific event,
Determine an appearance frequency for each of at least one object included in the plurality of images associated with the specific event,
And determine the object to delete based on the identity between the extracted at least one object and at least one object contained in a plurality of images associated with the specific event and the occurrence frequency. The method according to claim 1,
Wherein the display unit displays the image and the image from which the determined object is deleted,
The processor executing the one or more instructions,
And wherein based on the user input, the image and the determined object store at least one of the deleted images. The method according to claim 1,
The processor executing the one or more instructions,
And to control that the deleted object in the image is included in the user data. The method according to claim 1,
The processor executing the one or more instructions,
And determine an object to delete based on an identity between the extracted at least one object and an object having a previously deleted history. 9. The method of claim 8,
And the object having the previously deleted history is deleted based on the input of the user. A method for an electronic device to process an image,
Extracting at least one object from the image;
Determining an object to be deleted among the extracted at least one object based on the identity between the extracted at least one object and at least one object included in the user data; And
And deleting the determined object from the image. 11. The method of claim 10,
The method comprises:
Further comprising the step of learning the identity determination using the user data,
Wherein the step of determining the object to be deleted comprises:
Determining an identity between the extracted at least one object and at least one object included in the user data based on the result of the learning. 11. The method of claim 10,
Wherein the user data includes phone book data previously stored in the electronic device,
Wherein the step of determining the object to be deleted comprises:
Determining the object to be deleted based on the identity between the extracted at least one object and at least one object contained in the phone book data. 11. The method of claim 10,
The method comprises:
Further comprising determining an occurrence frequency for each of at least one object included in the user data,
Wherein the step of determining the object to be deleted comprises:
Determining the object to be deleted based on the identity between the extracted at least one object and at least one object included in the user data and the occurrence frequency. 11. The method of claim 10,
The method comprises:
Acquiring a plurality of images associated with a specific event; And
Further comprising determining an occurrence frequency for each of at least one object included in the plurality of images associated with the specific event,
Wherein the step of determining the object to be deleted comprises:
Determining the object to be deleted based on the identity between the extracted at least one object and at least one object included in the plurality of images associated with the specific event and the occurrence frequency. 11. The method of claim 10,
The method comprises:
Displaying the image and the image on which the determined object has been deleted on a display; And
Further comprising, based on user input, storing at least one of the image and the deleted image of the determined object. 11. The method of claim 10,
The method comprises:
Further comprising: storing the deleted object in the image as the user data. 11. The method of claim 10,
Wherein the step of determining the object to be deleted comprises:
Determining an object to delete based on an identity between the extracted at least one object and an object having a previously deleted history. 18. The method of claim 17,
Wherein the object having the previously deleted history is deleted based on a user's input. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the method according to any one of claims 10 to 18. A method of processing an image using an electronic device using a learning model using a neural network,
Extracting at least one object from the image;
Determining an object to be deleted among the extracted at least one object based on the identity between the extracted at least one object and at least one object included in the user data; And
And deleting the determined object from the image.

Images