JP2024546819A - Electronic device value assessment method and device and deep learning model training method - Google Patents

Abstract

An electronic device value assessment method is disclosed, in which an appearance condition assessment is performed on an electronic device based on a plurality of images acquired by photographing the electronic device and a plurality of deep learning assessment models, and a value of the electronic device is determined based on a result of the appearance condition assessment and a result of an internal condition assessment of the electronic device.
[Selected figure] Figure 7

Description

The following embodiments relate to an electronic device value assessment method and device and a deep learning model training method.

Current methods for calculating the purchase price of used mobile phones include one where the operator directly operates the client's used mobile phone or checks it with the naked eye and determines the price of the used mobile phone based on the operator's own criteria, and another where the price of the used mobile phone is calculated through automated evaluation.

The above-mentioned background art was held or acquired by the inventors in the process of deriving the contents of this specification, and cannot necessarily be considered publicly known art that was disclosed to the general public prior to the filing of this application.

Currently used automated appraisals involve installing a linked app on a used mobile phone via wired or wireless connection, automatically acquiring information about the used mobile phone and inspecting its internal functions, and sending photos of the used mobile phone's appearance to an appraisal center, which then appraises the used mobile phone. A lot of resources and time are spent on appraising the appearance of used mobile phones. There is a need for an artificial intelligence value appraisal system and method that can quickly and accurately evaluate the appearance of used mobile phones and predict the optimal purchase price.

An electronic device value evaluation method according to one embodiment includes a step of evaluating the appearance condition of the electronic device based on a plurality of images acquired by photographing the electronic device and a plurality of deep learning evaluation models, and a step of determining the value of the electronic device based on the results of the appearance condition evaluation and the results of the internal condition evaluation of the electronic device.

The step of evaluating the appearance condition includes a step of generating a mask that predicts the defect state of each evaluation area of the electronic device from the image through the deep learning evaluation model, determining a grade for the defect of each evaluation area based on each of the generated masks, and determining a final grade for the appearance condition of the electronic device based on each of the determined grades.

The step of determining a grade for defects in each of the evaluation areas includes a step of, when a first deep learning evaluation model receives an input of a front image obtained by photographing the front of the electronic device, generating a first mask that predicts the defect state of the front through the front image, and determining a grade for the defect of the front based on the generated first mask; and, when a second deep learning evaluation model receives an input of a rear image obtained by photographing the rear of the electronic device, generating a second mask that predicts the defect state of the rear through the rear image, and determining a grade for the defect of the rear based on the generated second mask; Among the deep learning evaluation models, when a third deep learning evaluation model receives an input of a side image obtained by photographing the side of the electronic device, a step of generating a third mask that predicts a defect state of at least one of the sides through the side image, and determining a grade for the defect of the side based on the generated third mask; and among the deep learning evaluation models, when a fourth deep learning evaluation model receives an input of a screen image obtained by photographing the screen of the electronic device, a step of generating a fourth mask that predicts a defect state of the screen of the electronic device through the screen image, and determining a grade for the defect of the screen of the electronic device based on the generated fourth mask.

The step of evaluating the appearance condition may further include, when the shape of the electronic device is changed, generating a mask that predicts the defect state of the evaluation area of the changed shape from an image acquired by photographing the changed shape through an additional deep learning evaluation model other than the deep learning evaluation model, and determining a grade for the defect of the evaluation area of the changed shape based on the mask that predicts the defect state of the evaluation area of the changed shape.

The step of determining the final grade may include a step of determining the minimum grade among the determined grades as the final grade.

The step of determining the final grade may include the steps of applying a weighting value to each of the determined grades, and determining the final grade using the grade to which the angular weighting value has been applied.

The step of determining the value may include a step of determining a first amount based on the results of the appearance condition evaluation, a step of determining a second amount based on the results of the interior condition evaluation, and a step of calculating a price of the electronic device by subtracting the determined first amount and the determined second amount from a base price of the electronic device.

The defect state may include at least one of the defect position, the defect type, and the defect severity of each of the evaluation areas.

The electronic device value assessment method may further include a step of determining whether each of the images contains one or more objects that are erroneously recognized as defects, and, if any of the images contains the objects, a step of performing processing on the objects.

The object may include at least one of an object corresponding to a floating icon on the screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device.

An electronic device value evaluation method according to one embodiment includes the steps of: determining whether or not each of a plurality of images obtained by photographing an electronic device contains one or more objects that are erroneously recognized as defects in the electronic device; if any of the images contains the objects, processing the objects; evaluating the appearance condition of the electronic device based on the images in which the objects have been processed, the remaining images without the objects, and a deep learning evaluation model; and determining the value of the electronic device based on the results of the appearance condition evaluation and the results of the internal condition evaluation of the electronic device.

The step of evaluating the appearance condition may include a step of generating a mask predicting the defect condition of each evaluation area of the electronic device from the image in which the object is processed through the deep learning evaluation model and the remaining image, determining a grade for the defect of each evaluation area based on each of the generated masks, and determining a final grade for the appearance condition of the electronic device based on each of the determined grades.

The processing step may include performing a masking process on the object.

The object may include at least one of an object corresponding to a floating icon on the screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device.

An electronic device value evaluation device according to one embodiment includes a memory that stores multiple deep learning evaluation models, an appearance condition evaluation module that performs an appearance condition evaluation of the electronic device based on multiple images acquired by photographing the electronic device and the deep learning evaluation model, and a value determination module that determines the value of the electronic device based on the results of the appearance condition evaluation and the results of the internal condition evaluation of the electronic device.

The appearance condition evaluation module generates a mask that predicts the defect condition of each evaluation area of the electronic device from the image through the deep learning evaluation model, determines a grade for the defect of each evaluation area based on each generated mask, and determines a final grade for the appearance condition of the electronic device based on each determined grade.

Among the deep learning evaluation models, the first deep learning evaluation model, when receiving an input of a front image acquired by photographing the front of the electronic device, generates a first mask that predicts the defect state of the front through the front image, and can determine a grade for the defect of the front based on the generated first mask.

When the second deep learning evaluation model of the deep learning evaluation models receives an input of a rear image acquired by photographing the rear of the electronic device, the second deep learning evaluation model generates a second mask that predicts the defect state of the rear through the rear image, and determines a grade for the defect of the rear based on the generated second mask.

Among the deep learning evaluation models, the third deep learning evaluation model, when receiving an input of a side image obtained by photographing the side of the electronic device, generates a third mask that predicts a defect state of at least one of the sides through the side image, and can determine a grade for the defect of the side based on the generated third mask.

Among the deep learning evaluation models, the fourth deep learning evaluation model, when receiving an input of a screen image obtained by photographing the screen of the electronic device, generates a fourth mask that predicts the defect state of the screen of the electronic device through the screen image, and can determine a grade for the defect of the screen of the electronic device based on the generated fourth mask.

When the shape of the electronic device is changed, the appearance condition evaluation module generates a mask that predicts the defect state of the evaluation area of the changed shape from an image acquired by photographing the changed shape through an additional deep learning evaluation model other than the deep learning evaluation model, and can determine a grade for the defect of the evaluation area of the changed shape based on the mask that predicts the defect state of the evaluation area of the changed shape.

The appearance condition evaluation module can determine the minimum grade among the determined grades as the final grade.

The appearance condition evaluation module can apply a weighting value to each of the determined grades and determine the final grade using the grade to which the angle weighting value has been applied.

The value determination module can determine a first amount based on the results of the appearance condition evaluation, determine a second amount based on the results of the internal condition evaluation, and calculate the price of the electronic device by subtracting the determined first amount and the determined second amount from the base price of the electronic device.

The defect state may include at least one of the defect position, the defect type, and the defect severity of each of the evaluation areas.

The electronic device value assessment device may further include a pre-processing module that determines whether each of the images contains one or more objects that may be mistaken for defects, and performs processing on the objects if any of the images contain the objects.

The object may include at least one of an object corresponding to a floating icon on the screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device.

An electronic device value evaluation device according to one embodiment includes a pre-processing module that determines whether each of a plurality of images obtained by photographing an electronic device contains one or more objects that may be mistaken for defects in the electronic device; a step of processing the object if an image containing the object is found; an appearance condition evaluation module that performs an appearance condition evaluation of the electronic device based on the image in which the object has been processed, the remaining image without the object, and a deep learning evaluation model; and a value determination module that determines the value of the electronic device based on the results of the appearance condition evaluation and the results of the internal condition evaluation of the electronic device.

The appearance condition evaluation module generates masks predicting the defect condition of each evaluation area of the electronic device from the image in which the object is processed through the deep learning evaluation model and the remaining image, determines a grade for the defect of each evaluation area based on each of the generated masks, and determines a final grade for the appearance condition of the electronic device based on each of the determined grades.

A training method performed by a computing device according to one embodiment includes inputting a learning image of a defect into a deep learning model, generating a mask that predicts the state of the defect from the learning image through the deep learning model, calculating a similarity between the generated mask and a labeled mask for the defect, updating at least one parameter in the deep learning model if the calculated similarity is less than a threshold, and terminating training of the deep learning model if the calculated similarity is equal to or greater than the threshold.

The step of generating the mask includes, when a first learning image is input to a first deep learning model, a step of generating a first mask that predicts a defect state of the front side of the electronic device from the first learning image using the first deep learning model; when a second learning image is input to a second deep learning model, a step of generating a second mask that predicts a defect state of the rear side of the electronic device from the second learning image using the second deep learning model; when a third learning image is input to a third deep learning model, a step of generating a third mask that predicts a defect state of the side of the electronic device from the third learning image using the third deep learning model; and when a fourth learning image is input to a fourth deep learning model, a step of generating a fourth mask that predicts a defect state of the screen of the electronic device from the fourth learning image using the fourth deep learning model.

In one embodiment, images of the exterior of a used mobile phone can be analyzed quickly, allowing the exterior of the used mobile phone to be evaluated quickly and accurately.

1 and 2 are diagrams illustrating an unmanned purchase device and a server according to an embodiment. 1 and 2 are diagrams illustrating an unmanned purchase device and a server according to an embodiment. 3 to 6 are diagrams for explaining the operation of the electronic device value assessment device according to an embodiment. 3 to 6 are diagrams for explaining the operation of the electronic device value assessment device according to an embodiment. 3 to 6 are diagrams for explaining the operation of the electronic device value assessment device according to an embodiment. 3 to 6 are diagrams for explaining the operation of the electronic device value assessment device according to an embodiment. FIG. 7 is a flowchart illustrating an electronic device value assessment method according to an embodiment. FIG. 8 is a block diagram illustrating the configuration of a computing device for training a deep learning model according to one embodiment. 9A to 9C are diagrams illustrating a target mask and a predicted mask according to one embodiment. 9A to 9C are diagrams illustrating a target mask and a predicted mask according to one embodiment. 9A to 9C are diagrams illustrating a target mask and a predicted mask according to one embodiment. FIG. 10 is a flowchart illustrating a method for training a deep learning model in a computing device according to one embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concept.

Although terms such as first or second may be used to describe multiple components, such terms should be construed as being only for the purpose of distinguishing one component from the other components. For example, a first component may be named a second component, and similarly, a second component may be named a first component.

When a component is referred to as being "connected" to another component, it is understood that it is directly connected or connected to the other component, but that there may be other components in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "include" or "have" indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

All terms used herein, including technical or scientific terms, unless otherwise defined, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains. Terms commonly used and predefined should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined in this specification.

Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. When describing the embodiments with reference to the drawings, the same components are given the same reference numerals regardless of the drawing numerals, and duplicate descriptions thereof will be omitted.

Figures 1 and 2 are diagrams illustrating an unmanned purchase device and a server according to one embodiment.

Referring to Figures 1 and 2, an unmanned purchase device 110 and a server 120 are shown.

The unmanned purchase device 110 can purchase electronic devices (or used electronic devices) (e.g., smartphones, tablet PCs, wearable devices, etc.) from users and/or sell electronic devices (or used electronic devices) to users. The types of electronic devices are classified into, for example, bar type, rollable type, foldable type, etc., depending on the shape.

The unmanned purchase device 110 may be, for example, in the form of a kiosk, but is not limited to this.

The unmanned purchasing device 110 includes a photography box and a control unit.

When the door of the photography box of the unmanned purchase device 110 is opened, the user can connect the electronic device to the cable of the unmanned purchase device 110 (e.g., USB Type-C cable, Lightning cable, etc.) and place the electronic device in the photography box. The electronic device is connected to the control unit of the unmanned purchase device 110 via the cable. According to an embodiment, the electronic device is connected to the control unit of the unmanned purchase device 110 wirelessly (e.g., Bluetooth (registered trademark), BLE (Bluetooth Low Energy), etc.).

The control unit of the unmanned purchase device 110 may install a first application on the electronic device to inspect the internal state of the electronic device and collect information about the electronic device (e.g., model name, serial number, operating system version, etc.). Without being limited thereto, the user may pre-install the first application on the electronic device before inserting the electronic device into the unmanned purchase device 110.

The first application is executed on the electronic device to collect information about the electronic device and evaluate (or analyze) the internal state of the electronic device (e.g., hardware operating state, etc.). The hardware operating state indicates, for example, the state of whether the hardware of the electronic device (e.g., a sensor, a camera, etc.) is operating normally. The first application can evaluate (or determine) whether the hardware of the electronic device is operating normally.

The shooting box may be provided with multiple cameras and multiple lights. A first camera in the shooting box may capture the front of the electronic device to obtain one or more front images of the electronic device. A second camera in the shooting box may capture the rear of the electronic device to obtain one or more rear images of the electronic device. Each of the multiple third cameras in the shooting box may capture a side (or corner) of the electronic device to obtain one or more side images (or corner images).

The first camera photographs the screen of the electronic device to obtain one or more images (hereinafter referred to as "screen images"). According to an embodiment, the first application may cause the electronic device to display a monochromatic (e.g., white, black, red, blue, green, etc.) screen. When the monochromatic screen is displayed on the electronic device, the first camera photographs the monochromatic screen of the electronic device to obtain an image (hereinafter referred to as "monochromatic screen image"). For example, when a white screen is displayed on the electronic device, the first camera photographs the white screen of the electronic device to obtain a first monochromatic screen image. When a black screen is displayed on the electronic device, the first camera photographs the black screen of the electronic device to obtain a second monochromatic screen image. When a monochromatic screen other than white and black (e.g., red, blue, green, etc.) is displayed on the electronic device, the first camera may photograph the other monochromatic screen of the electronic device to obtain a third monochromatic screen image.

The electronic device value assessment device 130 can perform an appearance condition assessment of an electronic device based on images acquired by photographing the electronic device (e.g., one or more front images, one or more rear images, one or more side images, one or more monochrome screen images) and a deep learning assessment model.

In the example shown in FIG. 1, the electronic device value evaluation device 130 is included in the server 120. In the example shown in FIG. 1, the server 120 receives an image obtained by photographing an electronic device from the unmanned buy-back device 110 and transmits the received image to the electronic device value evaluation device 130. As described above, the first application in the electronic device can perform an internal state evaluation of the electronic device and transmit the result of the internal state evaluation of the electronic device to the server 120 via the unmanned buy-back device 110. In another example, the first application may cause the electronic device to be connected to the server 120 and transmit the result of the internal state evaluation of the electronic device to the server 120 via the electronic device. The electronic device value evaluation device 130 can determine the value (e.g., price) of the electronic device based on the result of the appearance state evaluation of the electronic device and the result of the internal state evaluation of the electronic device (e.g., the result of the first application performing the internal state evaluation of the electronic device). The electronic device value evaluation device 130 transmits the value of the electronic device to the unmanned buy-back device 110, and the unmanned buy-back device 110 transmits the value of the electronic device to the user. The user communicates to the unmanned purchasing device 110 that they will accept the value (e.g., price) of the electronic device and sell the electronic device, and if the user decides to sell the electronic device, the unmanned purchasing device 110 can move the electronic device placed in the photography box to a collection box (or a storage box). Depending on the embodiment, the collection box may be located inside or outside the unmanned purchasing device 110.

In the example shown in FIG. 2, the electronic device value evaluation device 130 may be included in the unmanned purchase device 110. In the example shown in FIG. 2, the electronic device value evaluation device 130 may receive an image obtained by photographing an electronic device with a camera in a photography box. The electronic device value evaluation device 130 receives the result of the internal state evaluation of the electronic device from the first application. The electronic device value evaluation device 130 can determine the value (e.g., price) of the electronic device based on the result of the appearance state evaluation of the electronic device and the result of the internal state evaluation of the electronic device. The electronic device value evaluation device 130 communicates the value of the electronic device to the user. The user communicates to the unmanned purchase device 110 that they will accept the value (e.g., price) of the electronic device and sell the electronic device, and the unmanned purchase device 110 can move the electronic device placed in the photography box to a collection box (or a storage box) if the user decides to sell the electronic device.

Figures 3 to 6 are diagrams explaining the operation of an electronic device value assessment device according to one embodiment.

Referring to FIG. 3, the electronic device evaluation device 130 includes a memory 310, an appearance condition evaluation module 320, and a value determination module 330.

In one embodiment, the appearance condition assessment module 320 and the value determination module 330 may be implemented in a single processor.

In one embodiment, each of the appearance condition assessment module 320 and the value determination module 340 may be implemented in a separate processor. For example, a first processor may implement the appearance condition assessment module 320 and a second processor may implement the value determination module 340.

The memory 310 stores a plurality of deep learning evaluation models. For example, the memory 310 includes a first deep learning evaluation model that detects defects in a first evaluation area (e.g., front surface) of the electronic device and determines a grade of the detected defect (or the first evaluation area), a second deep learning evaluation model that detects defects in a second evaluation area (e.g., rear surface) of the electronic device and determines a grade of the detected defect (or the second evaluation area), a third deep learning evaluation model that detects defects in a third evaluation area (e.g., side (or corner)) of the electronic device and determines a grade of the detected defect (or the third evaluation area), and a fourth deep learning evaluation model that detects defects in a fourth evaluation area (e.g., screen) of the electronic device and determines a grade of the detected defect (or the fourth evaluation area). Table 1 below shows examples of defect types and grades for each evaluation area (e.g., screen, front surface, side (or corner), rear surface).

In Table 1 above, intermediate image retention refers to, for example, a phenomenon in which, although the electronic device is displaying a white screen, the user sees a specific area of the screen (for example, the status display area at the top of the screen) as a color other than white, and an icon is seen in the specific area. Strong image retention refers to, for example, a phenomenon in which, although the electronic device is displaying a white screen, the user sees the entire screen as a color other than white. LCD-class image retention is a state in which the degree of image retention is more severe than strong image retention, and refers to, for example, a phenomenon in which, although the electronic device is displaying a white screen, the user sees the entire screen as a color other than white, and an icon is seen on the screen.

Each of the first to fourth deep learning evaluation models can perform image segmentation on a given input image.

Figure 4 shows the schematic structure of the deep neural network that is the basis of each deep learning evaluation model. For ease of explanation, the following will use the structure of a deep neural network as an example, but this is not necessarily limited to this, and neural networks of various structures may be used in the deep learning evaluation model.

A deep neural network includes multiple layers as a method for realizing a neural network. A deep neural network includes, for example, an input layer 410 to which input data is applied, an output layer 440 that outputs a result value derived through prediction based on the input data based on training, and multiple hidden layers 420 and 430 between the input layer and the output layer.

Deep neural networks are classified into convolutional neural networks and recurrent neural networks depending on the algorithm used to process information. In the following, in accordance with common practice in the field of neural networks, the input layer is called the lowest layer and the output layer is called the highest layer, and the layers are named in order from the highest layer, the output layer, to the lowest layer, the input layer. In FIG. 4, hidden layer 2 is a layer higher than hidden layer 1 and the input layer, and a layer lower than the output layer.

A relatively higher layer among adjacent layers in a deep neural network can multiply the output value of a relatively lower layer by a weighted value, apply a biased value, and output a predetermined calculation result. Here, the output calculation result may be applied to a higher layer adjacent to the corresponding layer in a similar manner.

The method of training a neural network is called, for example, deep learning, and as mentioned above, deep learning uses various algorithms such as convolutional neural networks and recurrent neural networks.

"Training a neural network" can be understood to encompass all of the following: determining and updating weights and biases between layers, and/or determining and updating weights and biases between multiple neurons that belong to different layers among adjacent layers.

The multiple layers, the hierarchical structure between the multiple layers, the weights between neurons, and the biases may all be collectively referred to as the "connectivity" of the neural network. Therefore, "training a neural network" may be understood as building and training the connectivity.

In a neural network, each of the layers contains multiple nodes. A node corresponds to a neuron in the neural network. The term "neuron" is used synonymously with the term "node."

In the deep neural network shown in FIG. 4, it can be seen that connections are formed between all combinations of nodes included in layers adjacent to a plurality of nodes included in one layer. When all combinations of nodes included in adjacent layers of a neural network are connected to each other in this way, it is called "fully-connected." Node 3-1 of hidden layer (2) 430 shown in FIG. 4 is connected to all nodes in hidden layer (1) 420, i.e., nodes 2-1 to 2-4, and a value obtained by multiplying the output value of each node by a predetermined weighting value can be input.

Data input to the input layer 410 is processed through multiple hidden layers 420, 430, and an output value is output through the output layer 440. Here, the larger the weight value multiplied to the output value of each node, the stronger the connectivity between the corresponding two nodes, and the smaller the weight value, the weaker the connectivity between the two nodes. A weight value of 0 means that there is no connectivity between the two nodes.

Returning to FIG. 3, the appearance condition evaluation module 320 may perform an appearance condition evaluation for the electronic device based on a plurality of images acquired by photographing the electronic device and a deep learning evaluation model. For example, the appearance condition evaluation module 320 may generate a mask that predicts the defect state of each of the first to fourth evaluation areas of the electronic device from the image through the deep learning evaluation model. The appearance condition evaluation module 320 may determine a defect grade for each of the first to fourth evaluation areas based on each of the generated masks. The appearance condition evaluation module 320 may determine a final grade for the appearance condition of the electronic device based on each of the determined grades.

In the example shown in FIG. 5, the first deep learning evaluation model 510 may receive a front image. The first deep learning evaluation model 510 generates a first mask that predicts a defect state (e.g., at least one of a defect position, a defect type, and a defect degree) of the front of the electronic device through the front image. Here, the defect degree is related to the type of defect. For example, the first deep learning evaluation model 510 may perform image segmentation on the front image to classify each pixel of the front image into one of the first classes, and generate a first mask according to such classification. Table 2 below shows examples of the first classes.

The first camera in the aforementioned shooting box can capture not only the front of the electronic device but also the area around the front, so that the front image can include parts that are not the electronic device.

The first deep learning evaluation model 510 classifies some pixels of the front image into a 1-1 class and classifies the remaining pixels into a 1-2 class, a 1-3 class, or a 1-4 class. Through such classification, the first deep learning evaluation model 510 can generate a first mask.

An example of a visual representation of the first mask is shown in Figure 6.

In the example shown in FIG. 6, black color areas 610-1, 610-2, 610-3, and 610-4 indicate the result of the first deep learning evaluation model 510 classifying some pixels of the front image into class 1-3 (or the result of the first deep learning evaluation model 510 predicting that some pixels of the front image do not correspond to an electronic device). Area 620 indicates the result of the first deep learning evaluation model 510 classifying some pixels of the front image into class 1-2 (or the result of the first deep learning evaluation model 510 predicting that there is damage on the front of the electronic device from the front image). Area 630 indicates the result of the first deep learning evaluation model 510 classifying some pixels of the front image into class 1-1 (or the result of the first deep learning evaluation model 510 predicting that there is a scratch on the front of the electronic device from the front image). Area 640 shows the result of the first deep learning evaluation model 510 classifying some pixels of the front image into classes 1-4 (or the result of the first deep learning evaluation model 510 predicting the front of the electronic device from the front image).

The first deep learning evaluation model 510 may determine a grade for the defect on the front surface based on the first mask. For example, if the first deep learning evaluation model 510 predicts that the front surface of the electronic device has at least one of damage and floating liquid crystal through the front image, the first deep learning evaluation model 510 may determine the grade of the defect on the front surface of the electronic device as a C grade (e.g., the C grade in Table 1 above). The first deep learning evaluation model 510 outputs a score 5 corresponding to the C grade. If the first deep learning evaluation model 510 predicts that the front surface of the electronic device has damage and scratches through the front image, the first deep learning evaluation model 510 may determine the grade of the defect on the front surface of the electronic device as a C grade (e.g., the C grade in Table 1 above). The first deep learning evaluation model 510 outputs a score 5 corresponding to the C grade. When the first deep learning evaluation model 510 predicts that the front of the electronic device has at least one of scratches and scratches of a front damage level through the front image, the first deep learning evaluation model 510 may determine the grade of defects on the front of the electronic device as grade B (e.g., grade B in Table 1 above). The first deep learning evaluation model 510 outputs a score 3 corresponding to grade B. When the first deep learning evaluation model 510 predicts that the front of the electronic device is clean (or has no defects on the front) through the front image, the first deep learning evaluation model 510 may determine the grade of defects on the front of the electronic device as grade A (e.g., grade A in Table 1 above). The first deep learning evaluation model 510 outputs a score 1 corresponding to grade A.

The second deep learning evaluation model 520 may receive a rear image. The second deep learning evaluation model 520 may generate a second mask that predicts a defect state (e.g., at least one of a defect position, a defect type, and a defect degree) of the rear of the electronic device through the rear image. For example, the second deep learning evaluation model 520 may perform image segmentation on the rear image to classify each pixel of the rear image into one of the second classes, and generate the second mask through such classification. Table 3 below shows examples of the second classes.

The second deep learning evaluation model 520 may determine a grade for the defect on the rear surface based on the second mask. For example, if the second deep learning evaluation model 520 predicts that the rear surface of the electronic device has at least one of damage, rear surface lift, and camera lens damage through the rear surface image, the second deep learning evaluation model 520 may determine the grade of the defect on the rear surface of the electronic device as grade C (e.g., grade C in Table 1 above). The second deep learning evaluation model 520 outputs a score 5 corresponding to grade C. If the second deep learning evaluation model 520 predicts that the rear surface of the electronic device is clean through the rear surface image, the second deep learning evaluation model 520 may determine the grade of the defect on the rear surface of the electronic device as grade A (e.g., grade A in Table 1 above). The second deep learning evaluation model 520 outputs a score 1 corresponding to grade A.

The third deep learning evaluation model 530 may receive a side image (or a corner image). The third deep learning evaluation model 530 may generate a third mask that predicts a defect state (e.g., at least one of a defect position, a defect type, and a defect degree) of the side (or corner) of the electronic device through the side image (or the corner image). For example, the third deep learning evaluation model 530 may perform image segmentation on the side image (or the corner image) to classify each pixel of each side image into one of the third classes, and generate a third mask through such classification. Table 4 below shows examples of the third classes.

The third deep learning evaluation model 530 may determine a grade of the defect of the side (or corner) based on the third mask. For example, if the third deep learning evaluation model 530 predicts that the first side (or first corner) of the electronic device is scratched through the side image (or corner image), the third deep learning evaluation model 530 may determine the grade of the defect of the side (or corner) of the electronic device as a B+ grade (e.g., the B+ grade in Table 1 above). The third deep learning evaluation model 530 outputs a score 2 corresponding to the B+ grade. If the third deep learning evaluation model 530 predicts that the side (or corner) of the electronic device is clean through the side image (or corner image), the third deep learning evaluation model 530 may determine the grade of the defect of the side (or corner) of the electronic device as an A grade (e.g., the A grade in Table 1 above). The third deep learning evaluation model 530 outputs a score 1 corresponding to the A grade.

The fourth deep learning evaluation model 540 receives a screen image (e.g., a monochrome screen image) input to the electronic device. The fourth deep learning evaluation model 540 may generate a fourth mask that predicts a defect state (e.g., at least one of a defect position, a defect type, and a defect degree) of the screen of the electronic device through the screen image. For example, the fourth deep learning evaluation model 540 may perform image segmentation on the screen image to classify each pixel of the screen image into one of the fourth classes, and generate a fourth mask through such classification. Table 5 below shows examples of the fourth classes.

The fourth deep learning evaluation model 540 may determine a grade for the defect of the screen of the electronic device based on the fourth mask. For example, if the fourth deep learning evaluation model 540 predicts that the screen of the electronic device has at least one of three or more white spots, screen lines, black spots, and bullet damage through the screen image, the grade of the defect of the screen of the electronic device may be determined as a D grade (e.g., the D grade in Table 1 above). The fourth deep learning evaluation model 540 outputs a score 7 corresponding to the D grade. If the fourth deep learning evaluation model 540 predicts that the screen of the electronic device has at least one of LCD-grade image retention and LCD-grade white spots through the screen image, the grade of the defect of the screen of the electronic device may be determined as a DL grade (e.g., the DL grade in Table 1 above). The fourth deep learning evaluation model 540 outputs a score 6 corresponding to the DL grade. The fourth deep learning evaluation model 540 may determine a grade for the defect of the screen of the electronic device as a CL grade (e.g., CL grade in Table 1 above) when predicting that at least one of two or less of strong afterimage and whitening is present on the screen of the electronic device through the screen image. The fourth deep learning evaluation model 540 outputs a score 4 corresponding to the CL grade. When the fourth deep learning evaluation model 540 predicts that there is an intermediate afterimage on the screen of the electronic device through the screen image, it may determine a grade for the defect of the screen of the electronic device as a B grade (e.g., B grade in Table 1 above). The fourth deep learning evaluation model 540 outputs a score 3 corresponding to the B grade. When the fourth deep learning evaluation model 540 predicts that the screen of the electronic device is clean through the screen image, it may determine a grade for the defect of the screen of the electronic device as an A grade (e.g., A grade in Table 1 above). The fourth deep learning evaluation model 540 outputs a score 1 corresponding to the A grade.

Returning to FIG. 3, the value determination module 330 can determine the value of the electronic device based on the results of the external condition assessment of the electronic device and/or the results of the internal condition assessment of the electronic device.

In one embodiment, the value determination module 330 may determine the minimum grade among the grades determined by each of the first to fourth deep learning evaluation models 510 to 540 as the final grade for the appearance condition of the electronic device. Grade A is the highest, and grade B+ is lower than grade A and higher than grade B. Grade CL is lower than grade B and higher than grade C. Grade D is the lowest. As an example, the grade determined by the first deep learning evaluation model 510 is grade C, the grade determined by the second deep learning evaluation model 520 is grade B+, the grade determined by the third deep learning evaluation model 530 is grade C, and the grade determined by the fourth deep learning evaluation model 540 is grade CL. Among the grades determined by each of the first to fourth deep learning evaluation models 510 to 540, the grade C determined by the first deep learning evaluation model 510 is the minimum grade, and the value determination module 330 may determine the final grade for the appearance condition of the electronic device as the grade C. According to an embodiment, the lower the grade, the higher the score output by each of the first to fourth deep learning evaluation models 510 to 540. The value determination module 330 may determine the maximum score among the scores output by each of the first to fourth deep learning evaluation models 510 to 540 as the final score for the appearance evaluation of the electronic device.

In one embodiment, the value determination module 330 may apply weights to the grades (or scores) determined by each of the first to fourth deep learning evaluation models 510 to 540, and determine a final grade (or final score) for the appearance condition of the electronic device using the grades (or scores) to which each weight is applied. As an example, the value determination module 330 may apply a first weight to the grade (or score) determined by the first deep learning evaluation model 510, apply a second weight to the grade (or score) determined by the second deep learning evaluation model 520, apply a third weight to the grade (or score) determined by the third deep learning evaluation model 530, and apply a fourth weight to the grade determined by the fourth deep learning evaluation model 540. Here, each of the first to fourth weights is greater than 0 and less than 1. The value determination module 330 can sum the grades (or scores) to which the first to fourth weighting values are applied to determine a final grade (or final score) for the appearance condition of the electronic device.

The value determination module 330 determines a first amount based on the result of the appearance condition evaluation of the electronic device (e.g., the final grade (or final score) for the appearance condition of the electronic device) and determines a second amount based on the result of the internal condition evaluation of the electronic device. The value determination module 330 can calculate the price of the electronic device by subtracting the first amount and the second amount from the reference price of the electronic device (e.g., the highest used price of an electronic device of the same type as the electronic device). As an example, the value determination module 330 may obtain the reference price of the electronic device in conjunction with a database of used market prices. The value determination module 330 can obtain a first amount mapped to the final grade for the appearance condition of the electronic device from a first table in which the grade of the appearance condition and the amount are mapped to each other. The value determination module 330 can obtain a second amount mapped to the result of the internal condition evaluation of the electronic device from a second table in which the grade of the internal condition and the amount are mapped to each other. The value determination module 330 can calculate the price of the electronic device by subtracting the first amount and the second amount from the reference amount.

In the example shown in FIG. 1, the value determination module 330 can transmit the value (e.g., price) of the electronic device to the unmanned purchase device 110. The unmanned purchase device 110 can display the value (e.g., price) of the electronic device to the user via a display.

In the example shown in FIG. 2, the value determination module 330 can display the value (e.g., price) of the electronic device on the display of the unmanned purchase device 110.

In one embodiment, the electronic device value assessment device 130 includes a pre-processing module. The pre-processing module determines whether each of the images (e.g., a front image, a rear image, a side image, and a screen image) includes one or more objects that are erroneously recognized as defects. Here, the objects may include at least one of an object corresponding to a floating icon on the screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device. The object corresponding to the floating icon indicates an object included in the image by photographing the floating icon on the screen of the electronic device. The object corresponding to the sticker attached to the electronic device indicates an object included in the image by photographing the sticker attached to the electronic device. The object corresponding to the foreign object attached to the electronic device indicates an object included in the image by photographing the foreign object attached to the electronic device. The floating icon includes, for example, a floating icon for assistive touch, a floating icon for triggering a specific task, and the like, but is not limited thereto.

If an image includes an object that is erroneously recognized as a defect, the pre-processing module may process the object. As an example, the pre-processing module may perform a masking process on the object, but is not limited thereto. The appearance condition evaluation module 320 may perform an appearance condition evaluation based on the image in which the object has been processed, the remaining image in which the object is not included, and the deep learning evaluation models 510 to 540. The appearance condition evaluation module 320 may generate masks that predict the defect state of each evaluation area of the electronic device from the image in which the object has been processed and the remaining image in which the object is not included through the deep learning evaluation models 510 to 540, determine a grade for the defect of each evaluation area based on each generated mask, and determine a final grade for the appearance condition of the electronic device based on each determined grade.

The pre-processing module may determine that none of the images captured by photographing the electronic device contain the above-mentioned objects. In this case, as described above, the appearance condition evaluation module 320 may perform an appearance condition evaluation based on the images and the deep learning evaluation models 510 to 540.

The pre-processing module determines whether any of the images obtained by photographing the electronic device are images that cannot be analyzed by one or more of the deep learning evaluation models (hereinafter referred to as "images that cannot be analyzed by model"). As an example, the pre-processing module may determine that any of the images obtained by photographing the electronic device are images that have a certain level of light reflection or an image where the camera is not in focus, as images that cannot be analyzed by model. If any of the images cannot be analyzed by model, the pre-processing module may request the operator to evaluate the appearance condition of the electronic device.

In one embodiment, the electronic device value assessment device 130 may assess the value of a bar-type electronic device. In this case, as described above, the electronic device value assessment device 130 (or the appearance condition assessment module 320) may perform an appearance condition assessment of the bar-type electronic device based on a plurality of images acquired by photographing the bar-type electronic device and the first to fourth deep learning assessment models 510 to 540.

The electronic device value evaluation device 130 can evaluate the value of an electronic device whose shape can be changed (e.g., foldable, rollable, etc.). The electronic device whose shape can be changed may have a first shape (e.g., an unfolded shape or a contraction shape) and may be changed to a second shape (e.g., a folded shape or an expansion shape) by an operation. As an example, a foldable electronic device may be in an unfolded shape and may be changed to a folded shape in response to an operation. A rollable electronic device may be in a contracted shape and may be changed to an expanded shape in response to an operation. The contracted shape indicates a state in which the rollable display is rolled in to the device, and the expanded shape indicates a state in which the rollable display is rolled out from the device.

For example, the electronic device value assessment device 130 (or the appearance condition assessment module 320) may determine a defect grade for each assessment area of the foldable electronic device in the unfolded state based on multiple images acquired by photographing the foldable electronic device in the unfolded state and the first to fourth deep learning assessment models 510 to 540.

The unmanned purchase device 110 may change the foldable electronic device in the photography box from an unfolded form to a folded form. Alternatively, the unmanned purchase device 110 may request the user to change the foldable electronic device from an unfolded form to a folded form and then re-insert the folded electronic device into the unmanned purchase device 110. When the foldable electronic device is changed from an unfolded form to a folded form, the folded part forms a side and the sub-screen of the foldable electronic device is activated. The unmanned purchase device 110 may obtain an image (hereinafter, an image of the folded side) by photographing the side corresponding to the folded part of the foldable electronic device through one or more of the multiple third cameras in the photography box. The unmanned purchase device 110 may obtain an image (hereinafter, a sub-screen image) by photographing the sub-screen of the foldable electronic device through the first camera in the photography box.

The electronic device value assessment device 130 (or the appearance state assessment module 320) may assess the fifth assessment area (e.g., the side corresponding to the folded portion) of the foldable electronic device based on the image of the folded side and the fifth deep learning assessment model. Here, the fifth deep learning assessment model may be a deep learning assessment model that detects defects in the fifth assessment area of the foldable electronic device and determines a grade of the detected defects (or the fifth assessment area). As an example, the electronic device value assessment device 130 (or the appearance state assessment module 320) may input the image of the folded side to the fifth deep learning assessment model. The fifth deep learning assessment model generates a fifth mask that predicts the defect state (e.g., at least one of the defect position, defect type, and defect degree) of the fifth assessment area of the foldable electronic device through the image of the folded side. The fifth deep learning assessment model may determine a grade for the defect in the fifth assessment area of the foldable electronic device based on the fifth mask.

The electronic device value assessment device 130 (or the appearance state assessment module 320) may assess the sixth assessment area (e.g., the sub-screen) of the foldable electronic device based on the sub-screen image and the sixth deep learning assessment model. Here, the sixth deep learning assessment model may be a deep learning assessment model that detects defects in the sixth assessment area of the foldable electronic device and determines a grade of the detected defects (or the sixth assessment area). As an example, the electronic device value assessment device 130 (or the appearance state assessment module 320) may input the sub-screen image to the sixth deep learning assessment model. The sixth deep learning assessment model generates a sixth mask that predicts the defect state (e.g., at least one of the defect position, defect type, and defect degree) of the sixth assessment area of the foldable electronic device via the sub-screen image. The sixth deep learning assessment model may determine a grade for the defect in the sixth assessment area of the foldable electronic device based on the sixth mask. According to an embodiment, the electronic device value assessment device 130 (or the appearance condition assessment module 320) can determine a defect grade for the sixth assessment area (e.g., the sub-screen) of the electronic device based on the sub-screen image and the fourth deep learning assessment model 540 described above.

The electronic device value assessment device 130 (or the value determination module 330) can determine the value of the foldable electronic device based on the results of the appearance condition assessment of the foldable electronic device (e.g., the grades determined by each of the first to sixth deep learning assessment models) and/or the results of the internal condition assessment of the foldable electronic device.

As another example, the electronic device value assessment device 130 (or the appearance condition assessment module 320) may determine a defect grade for each assessment area of the reduced-size rollable electronic device based on multiple images acquired by photographing the reduced-size rollable electronic device and the first to fourth deep learning assessment models 510 to 540.

The unmanned purchase device 110 may change the rollable electronic device in the photography box from a reduced form to an expanded form. Alternatively, the unmanned purchase device 110 may request the user to change the rollable electronic device from the reduced form to the expanded form and then re-insert the electronic device in the expanded form into the unmanned purchase device 110. When the rollable electronic device is changed from the reduced form to the expanded form, the screen and sides may be expanded. The unmanned purchase device 110 may obtain an image (hereinafter, an image of the expanded side) by photographing the expanded side through one or more of the multiple third cameras in the photography box. The unmanned purchase device 110 may obtain an image (hereinafter, an image of the expanded screen) by photographing the expanded screen of the electronic device through the first camera in the photography box.

The electronic device value assessment device 130 (or the appearance condition assessment module 320) may assess the seventh assessment region (e.g., the extended side) of the rollable electronic device based on the extended side image and the seventh deep learning assessment model. Here, the seventh deep learning assessment model may be a deep learning assessment model that detects defects in the seventh assessment region of the rollable electronic device and determines a grade of the detected defects (or the seventh assessment region). As an example, the electronic device value assessment device 130 (or the appearance condition assessment module 320) may input the extended side image to the seventh deep learning assessment model. The seventh deep learning assessment model generates a seventh mask that predicts the defect state (e.g., at least one of the defect position, defect type, and defect degree) of the seventh assessment region of the rollable electronic device through the extended side image. The seventh deep learning assessment model may determine a grade for the defect in the seventh assessment region of the rollable electronic device based on the seventh mask. According to an embodiment, the electronic device value assessment device 130 (or the appearance condition assessment module 320) can assess a seventh assessment area (e.g., the extended side) of the rollable electronic device based on the image of the extended side and the third deep learning assessment model 530.

The electronic device value assessment device 130 (or the appearance state assessment module 320) may assess the eighth assessment region (e.g., the extended screen) of the rollable electronic device based on the image of the extended screen and the fourth deep learning assessment model 540. As an example, the electronic device value assessment device 130 (or the appearance state assessment module 320) may input the image of the extended screen to the fourth deep learning assessment model 540. As described above, the fourth deep learning assessment model 540 may be a deep learning assessment model that generates a mask predicting a defect state of the screen in a given screen image and determines a grade of the defect of the screen based on the generated mask. The fourth deep learning assessment model 540 generates an eighth mask predicting a defect state (e.g., at least one of a defect position, a defect type, and a defect degree) of the eighth assessment region of the rollable electronic device through the image of the extended screen. The fourth deep learning assessment model 540 may determine a grade for the defect of the eighth assessment region of the rollable electronic device based on the eighth mask.

The electronic device value evaluation device 130 (or the value determination module 330) can determine the value of the rollable electronic device based on the results of the appearance condition evaluation of the rollable electronic device (e.g., the grades determined by each of the first to fourth deep learning evaluation models and the seventh deep learning evaluation model) and/or the results of the internal condition evaluation of the rollable electronic device.

In one embodiment, a user can insert a wearable device (e.g., a smart watch) into the unmanned buy-back device 110. The electronic device value assessment device 130 stores a deep learning assessment model that can assess the appearance (e.g., front, back, side, screen) of the wearable device. The electronic device value assessment device 130 can evaluate the appearance state of the wearable device based on an image acquired by photographing the wearable device and the deep learning assessment model. The electronic device value assessment device 130 can determine the value of the wearable device based on the results of the appearance state assessment of the wearable device and the results of the internal state assessment of the wearable device.

Figure 7 is a flowchart illustrating an electronic device value evaluation method according to one embodiment.

Referring to FIG. 7, in step 710, the electronic device value assessment device 130 performs an appearance condition assessment for the electronic device based on a plurality of images acquired by photographing the electronic device and a plurality of deep learning assessment models. The electronic device value assessment device 130 can generate masks predicting the defect state of each evaluation area of the electronic device from the image via the deep learning assessment models 510 to 540, and determine a grade for the defect of each evaluation area of the electronic device based on each generated mask. The electronic device value assessment device 130 can determine a final grade for the appearance condition of the electronic device based on each determined grade.

In one embodiment, the shape of the electronic device may be changed. As described above, the electronic device value assessment device 130 may change the electronic device from a first form (e.g., an unfolded form or a contracted form) to a second form (e.g., a folded form or an extended form). Alternatively, the electronic device value assessment device 130 may request the user to change the electronic device from the first form to the second form and then re-input the electronic device in the second form into the unmanned buy-back device 110. When the shape of the electronic device is changed, the electronic device value assessment device 130 may generate a mask (e.g., at least one of the fifth mask, the sixth mask, and the seventh mask described above) that predicts a defect state of the evaluation area of the changed shape of the electronic device from an image obtained by photographing the changed shape of the electronic device through an additional deep learning evaluation model other than the deep learning evaluation models 510 to 540 (e.g., at least one of the fifth deep learning evaluation model, the sixth deep learning evaluation model, and the seventh deep learning evaluation model described above). The electronic device value assessment device 130 can determine a grade for defects in the evaluation area of the modified form of the electronic device based on the mask that predicts the defect state of the evaluation area of the modified form.

Each of the fifth to seventh deep learning evaluation models described above can perform image segmentation on a given input image.

In step 720, the electronic device value assessment device 130 determines the value of the electronic device based on the results of the external appearance assessment and the results of the internal condition assessment of the electronic device.

The matters described with reference to Figures 1 to 6 may be applied to the electronic device value evaluation method shown in Figure 7.

Figure 8 is a block diagram illustrating the configuration of a computing device for training a deep learning model according to one embodiment.

Referring to FIG. 8, a computing device 800 for training a deep learning model includes a memory 810 and a processor 820.

The memory 810 stores one or more deep learning models. The deep learning models are based on the deep neural networks described with reference to FIG. 4. The deep learning models can perform image segmentation on a given input image.

The processor 820 trains the deep learning model.

The processor 820 can input a learning image of the defect into a deep learning model and generate a mask that predicts the state of the defect from the learning image via the deep learning model.

The processor 820 calculates the similarity between the generated mask and a labeled mask for the defect.

Figures 9A to 9C show examples of a generated mask and a Label mask, respectively. In Figure 9A, the target mask shows the Label mask, and the predicted mask shows the mask generated by the deep learning model.

The processor 820 may update at least one parameter in the deep learning model if the calculated similarity is less than the threshold. The processor 820 may terminate training for the deep learning model if the calculated similarity is greater than or equal to the threshold.

According to an embodiment, when the processor 820 inputs a first learning image (e.g., a front image in which a front surface having a first defect is captured) into the first deep learning model, the processor 820 can generate a first mask that predicts a defect state of the front surface of the electronic device from the first learning image using the first deep learning model. The first deep learning model performs image segmentation on the first learning image to generate a first mask that predicts a defect state of the front surface of the electronic device from the first learning image. The processor 820 calculates a first similarity between the first mask and a label mask for the first defect. If the calculated first similarity is less than a threshold, the processor 820 updates at least one parameter in the first deep learning model. If the calculated first similarity is equal to or greater than the threshold, the processor 820 terminates training for the first deep learning model. The first deep learning model for which training has been completed can be loaded into the electronic device value assessment device 130 as the first deep learning assessment model 510.

When the processor 820 inputs a second learning image (e.g., a rear image of a rear surface having a second defect) into the second deep learning model, the processor 820 can generate a second mask that predicts the defect state of the rear surface of the electronic device from the second learning image using the second deep learning model. The second deep learning model performs image segmentation on the second learning image to generate a second mask that predicts the defect state of the rear surface of the electronic device from the second learning image. The processor 820 calculates a second similarity between the second mask and a label mask for the second defect. If the calculated second similarity is less than a threshold, the processor 820 updates at least one parameter in the second deep learning model. If the calculated second similarity is equal to or greater than the threshold, the processor 820 terminates training for the second deep learning model. The second deep learning model for which training has been completed can be installed in the electronic device value assessment device 130 as the second deep learning assessment model 520.

When the processor 820 inputs a third learning image (e.g., a side (or corner) image in which a side (or corner) having a third defect is photographed) into the third deep learning model, the processor 820 can generate a third mask that predicts the defect state of the side (or corner) of the electronic device from the third learning image using the third deep learning model. The third deep learning model performs image segmentation on the third learning image to generate a third mask that predicts the defect state of the side (or corner) of the electronic device from the third learning image. The processor 820 calculates a third similarity between the third mask and a label mask for the third defect. If the calculated third similarity is less than a threshold, the processor 820 updates at least one parameter in the third deep learning model. If the calculated third similarity is equal to or greater than the threshold, the processor 820 terminates training for the third deep learning model. The third deep learning model for which training has been completed can be mounted in the electronic device value assessment device 130 as the third deep learning assessment model 530.

When the processor 820 inputs a fourth learning image (e.g., a screen image of a screen having a fourth defect) into the fourth deep learning model, the processor 820 can generate a fourth mask that predicts the defect state of the screen of the electronic device from the fourth learning image using the fourth deep learning model. The fourth deep learning model performs image segmentation on the fourth learning image to generate a fourth mask that predicts the defect state of the screen of the electronic device from the fourth learning image. The processor 820 calculates a fourth similarity between the fourth mask and a label mask for the fourth defect. If the calculated fourth similarity is less than a threshold, the processor 820 updates at least one parameter in the fourth deep learning model. If the calculated fourth similarity is equal to or greater than the threshold, the processor 820 terminates training for the fourth deep learning model. The fourth deep learning model that has completed training can be installed in the electronic device value evaluation device 130 as the fourth deep learning evaluation model 540.

In one embodiment, when the processor 820 inputs a fifth learning image (e.g., an image of a folded side having a fifth defect) into the fifth deep learning model, the processor 820 can generate a fifth mask that predicts the defect state of the side corresponding to the folded portion of the foldable electronic device from the fifth learning image using the fifth deep learning model. The processor 820 calculates a fifth similarity between the fifth mask and a label mask for the fifth defect. If the calculated fifth similarity is less than a threshold, the processor 820 updates at least one parameter in the fifth deep learning model. If the calculated fifth similarity is equal to or greater than the threshold, the processor 820 terminates training for the fifth deep learning model. The fifth deep learning model that has completed training can be loaded into the electronic device value assessment device 130 as a fifth deep learning assessment model.

In one embodiment, when the processor 820 inputs a sixth learning image (e.g., an image of a sub-screen having a sixth defect) into the sixth deep learning model, the processor 820 can generate a sixth mask that predicts the defect state of the sub-screen of the foldable electronic device from the sixth learning image using the sixth deep learning model. The processor 820 calculates a sixth similarity between the sixth mask and a label mask for the sixth defect. If the calculated sixth similarity is less than a threshold, the processor 820 updates at least one parameter in the sixth deep learning model. If the calculated sixth similarity is equal to or greater than the threshold, the processor 820 terminates training for the sixth deep learning model. The sixth deep learning model that has completed training can be loaded into the electronic device value assessment device 130 as a sixth deep learning assessment model.

In one embodiment, when the processor 820 inputs a seventh learning image (e.g., an image of an extended side having a seventh defect) into the seventh deep learning model, the seventh mask can be generated using the seventh deep learning model to predict the defect state of the extended side of the rollable electronic device from the seventh learning image. The processor 820 calculates a seventh similarity between the seventh mask and a label mask for the seventh defect. If the calculated seventh similarity is less than a threshold, the processor 820 updates at least one parameter in the seventh deep learning model. If the calculated seventh similarity is equal to or greater than a threshold, the processor 820 terminates training for the seventh deep learning model. The seventh deep learning model that has completed training can be loaded into the electronic device value assessment device 130 as a seventh deep learning assessment model. In one embodiment, the processor 820 trains the third deep learning model through the seventh learning image so that the third deep learning model generates the seventh mask.

In one embodiment, the processor 820 can train each deep learning model based on learning images of a wearable device with visual defects, similar to the training method described above, and generate a deep learning evaluation model that evaluates the appearance (e.g., front, back, sides, screen) of the wearable device.

Figure 10 is a flowchart illustrating a method for training a deep learning model on a computing device according to one embodiment.

Referring to FIG. 10, in step 1010, the computing device 800 inputs training images for defects into a deep learning model.

In step 1020, the computing device 800 generates a mask that predicts the defect state from the training image via the deep learning model.

In step 1030, the computing device 800 calculates the similarity between the generated mask and the label mask for the defect.

In step 1040, the computing device 800 determines whether the calculated similarity is less than a threshold value.

If the calculated similarity is less than the threshold, the computing device 800 updates at least one parameter in the deep learning model in step 1050. The computing device 800 repeats steps 1010 to 1040.

If the calculated similarity is greater than or equal to the threshold, the computing device 800 terminates training of the deep learning model in step 1060.

The contents described based on Figures 1 to 9 can be applied to the deep learning model training method shown in Figure 10.

The embodiments described above may be implemented with hardware components, software components, or a combination of hardware and software components. For example, the devices and components described in the present embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or a different device that executes and responds to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For ease of understanding, the description may assume that a single processing device is used, but one skilled in the art will appreciate that a processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

Software may include computer programs, codes, instructions, or any combination of one or more thereof, to configure or instruct a processing device to operate as desired, either independently or in combination. The software and/or data may be embodied permanently or temporarily in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, to be interpreted by or provide instructions or data to a processing device. The software may be distributed across computer systems coupled to a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to the present invention is embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, and the like, either alone or in combination. The recording medium and program instructions may be specially designed and constructed for the purposes of the present invention, or may be known and available to those skilled in the art of computer software technology. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations described in the present invention, and vice versa.

Although the embodiments have been described above with reference to limited drawings, a person having ordinary skill in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and/or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, and may be replaced or substituted with other components or equivalents to achieve suitable results.

Therefore, other realizations, other embodiments, and equivalents to the claims are also within the scope of the claims described below.

Claims (26)

A method for evaluating the value of an electronic device, comprising:
Evaluating an appearance state of the electronic device based on a plurality of images acquired by photographing the electronic device and a plurality of deep learning evaluation models;
determining a value of the electronic device based on a result of the appearance condition evaluation and a result of the internal condition evaluation of the electronic device;
Including,
The step of evaluating the appearance state includes:
generating a mask predicting a defect state of each evaluation area of the electronic device from the image through the deep learning evaluation model, and determining a grade for the defect of each evaluation area based on the generated mask;
determining a final grade for the appearance condition of the electronic device through each of the determined grades;
The electronic device value evaluation method includes: The step of determining a grade for each defect in each of the evaluation areas includes:
When a first deep learning evaluation model of the deep learning evaluation models receives an input of a front image acquired by photographing a front surface of the electronic device, a first mask is generated by predicting a defect state of the front surface through the front image, and a grade for the defect of the front surface is determined based on the generated first mask;
When a second deep learning evaluation model of the deep learning evaluation models receives an input of a rear image acquired by photographing a rear of the electronic device, a second mask is generated by predicting a defect state of the rear through the rear image, and a grade for the defect of the rear is determined based on the generated second mask;
When a third deep learning evaluation model of the deep learning evaluation models receives an input of a side image obtained by photographing a side of the electronic device, a third mask is generated that predicts a defect state of at least one of the sides through the side image, and a grade for the defect of the side is determined based on the generated third mask;
When a fourth deep learning evaluation model of the deep learning evaluation models receives an input of a screen image acquired by photographing the screen of the electronic device, a fourth mask is generated by predicting a defect state of the screen of the electronic device through the screen image, and a grade of the defect of the screen of the electronic device is determined based on the generated fourth mask;
The electronic device value evaluation method according to claim 1 , The electronic device value evaluation method according to claim 1, wherein the step of evaluating the appearance condition further includes a step of generating a mask that predicts the defect state of the evaluation area of the changed form from an image obtained by photographing the changed form through an additional deep learning evaluation model other than the deep learning evaluation model when the form of the electronic device is changed, and determining a grade for the defect of the evaluation area of the changed form based on the mask that predicts the defect state of the evaluation area of the changed form. The electronic device value evaluation method according to claim 1, wherein the step of determining the final grade includes a step of determining the minimum grade among the determined grades as the final grade. The step of determining the final grade comprises:
applying a weighting to each of the determined classes;
determining the final grade using the angular weighted grade;
The electronic device value evaluation method according to claim 1 , The step of determining the value comprises:
determining a first monetary amount based on the results of the exterior condition evaluation and determining a second monetary amount based on the results of the interior condition evaluation;
calculating a price of the electronic device by subtracting the determined first amount and the determined second amount from a base price of the electronic device;
The electronic device value evaluation method according to claim 1 , The electronic device value evaluation method according to claim 1, wherein the defect state includes at least one of the defect position, the defect type, and the defect degree of each of the evaluation areas. determining whether each of the images contains one or more objects that are misidentified as defects;
performing processing on the object if there is an image including the object;
The electronic device value assessment method according to claim 1 , further comprising: The electronic device value evaluation method according to claim 8, wherein the object includes at least one of an object corresponding to a floating icon on the screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device. A method for evaluating the value of an electronic device, comprising:
determining whether each of a plurality of images acquired by photographing an electronic device includes one or more objects that are misidentified as defects in the electronic device;
performing processing on the object if there is an image including the object;
performing an appearance condition assessment on the electronic device based on the processed image of the object, the remaining image without the object, and a deep learning assessment model;
determining a value of the electronic device based on a result of the appearance condition evaluation and a result of the internal condition evaluation of the electronic device;
The electronic device value evaluation method includes: The step of evaluating the appearance state includes:
generating a mask predicting a defect state of each evaluation area of the electronic device from the image in which the object is processed through the deep learning evaluation model and the remaining image, and determining a grade for the defect of each evaluation area based on each of the generated masks;
determining a final grade for the appearance condition of the electronic device through each of the determined grades;
The electronic device value assessment method according to claim 10, comprising: The electronic device value evaluation method according to claim 10, wherein the processing step includes a step of performing a masking process on the object. The electronic device value evaluation method according to claim 10, wherein the object includes at least one of an object corresponding to a floating icon on a screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device. An electronic device value assessment device,
A memory for storing a plurality of deep learning evaluation models;
An appearance state evaluation module that performs an appearance state evaluation for the electronic device based on a plurality of images acquired by photographing the electronic device and the deep learning evaluation model;
a value determination module for determining a value of the electronic device based on a result of the appearance condition evaluation and a result of the internal condition evaluation of the electronic device;
Including,
The appearance condition evaluation module generates a mask that predicts a defect state of each evaluation area of the electronic device from the image via the deep learning evaluation model, determines a grade for the defect of each evaluation area based on each generated mask, and determines a final grade for the appearance condition of the electronic device based on each determined grade. Among the deep learning evaluation models, a first deep learning evaluation model generates a first mask predicting a defect state of the front surface through the front image when a front image obtained by photographing the front surface of the electronic device is input, and determines a grade for the defect of the front surface based on the generated first mask;
Among the deep learning evaluation models, a second deep learning evaluation model generates a second mask predicting a defect state of the rear surface through the rear surface image when a rear surface image obtained by photographing a rear surface of the electronic device is input, and determines a grade for the defect of the rear surface based on the generated second mask;
A third deep learning evaluation model among the deep learning evaluation models generates a third mask predicting a defect state of at least one of the sides through the side image when a side image obtained by photographing a side of the electronic device is input, and determines a grade for the defect of the side based on the generated third mask;
The electronic device value evaluation device of claim 14, wherein a fourth deep learning evaluation model among the deep learning evaluation models, when receiving an input of a screen image obtained by photographing the screen of the electronic device, generates a fourth mask that predicts a defect state of the screen of the electronic device through the screen image, and determines a grade for the defect of the screen of the electronic device based on the generated fourth mask. The electronic device value evaluation device according to claim 14, wherein the appearance condition evaluation module, when the shape of the electronic device is changed, generates a mask predicting a defect state of the evaluation area of the changed shape from an image acquired by photographing the changed shape through an additional deep learning evaluation model other than the deep learning evaluation model, and determines a grade for defects in the evaluation area of the changed shape based on the mask predicting the defect state of the evaluation area of the changed shape. The electronic device value assessment device according to claim 14, wherein the appearance condition assessment module determines the minimum grade among the determined grades as the final grade. The electronic device value assessment device according to claim 14, wherein the appearance condition assessment module applies a weighting value to each of the determined grades, and determines the final grade using the grade to which the angular weighting value has been applied. The electronic device value evaluation device according to claim 14, wherein the value determination module determines a first amount based on the result of the appearance condition evaluation, determines a second amount based on the result of the internal condition evaluation, and calculates the price of the electronic device by subtracting the determined first amount and the determined second amount from the base price of the electronic device. The electronic device value evaluation device according to claim 14, wherein the defect state includes at least one of the defect position of each of the evaluation areas, the defect type, and the defect degree. The electronic device value assessment device according to claim 14, further comprising a pre-processing module that determines whether each of the images contains one or more objects that are erroneously recognized as defects, and performs processing on the objects if any of the images contains the objects. The electronic device value evaluation device according to claim 21, wherein the object includes at least one of an object corresponding to a floating icon on a screen of the electronic device, an object corresponding to a sticker attached to the electronic device, and an object corresponding to a foreign object attached to the electronic device. An electronic device value assessment device,
a pre-processing module for determining whether each of a plurality of images acquired by photographing an electronic device includes one or more objects that are likely to be mistaken for defects in the electronic device;
If there is an image including the object, a step of processing the object; and an appearance state evaluation module that performs an appearance state evaluation for the electronic device based on the image in which the object has been processed, the remaining image without the object, and a deep learning evaluation model.
a value determination module for determining a value of the electronic device based on a result of the appearance condition evaluation and a result of the internal condition evaluation of the electronic device;
An electronic device value assessment device comprising: The electronic device value evaluation device according to claim 23, wherein the appearance condition evaluation module generates masks predicting the defect condition of each evaluation area of the electronic device from the image in which the object is processed through the deep learning evaluation model and the remaining image, determines a grade for the defect of each evaluation area based on each of the generated masks, and determines a final grade for the appearance condition of the electronic device based on each of the determined grades. 1. A training method performed by a computing device, comprising:
inputting training images of defects into a deep learning model;
generating a mask that predicts the state of the defect from the training image via the deep learning model;
computing a similarity between the generated mask and a labeled mask for the defect;
if the calculated similarity is less than a threshold, updating at least one parameter in the deep learning model;
if the calculated similarity is greater than or equal to the threshold, terminating training of the deep learning model;
A training method performed by a computing device, comprising: The step of generating the mask comprises:
When a first learning image is input to a first deep learning model, generating a first mask predicting a defect state of a front surface of an electronic device from the first learning image using the first deep learning model;
When a second learning image is input to a second deep learning model, generating a second mask predicting a defect state of the rear surface of the electronic device from the second learning image using the second deep learning model;
When a third learning image is input to a third deep learning model, generating a third mask that predicts a defect state of a side surface of the electronic device from the third learning image using the third deep learning model;
When a fourth learning image is input to a fourth deep learning model, generating a fourth mask that predicts a defective state of a screen of the electronic device from the fourth learning image using the fourth deep learning model;
26. A training method performed by the computing device of claim 25, comprising:

Images