KR102231239B1 - Eyeglasses try-on simulation method - Google Patents

Abstract

The present invention relates to a spectacle wearing simulation method capable of showing the appearance of a real spectacled face by synthesizing a 3D spectacle model of spectacles according to a face of a photographed image based on an accurately measured face dimension, and The face and the reference article are photographed together in accordance with the facial contour 141 and the reference article contour 142 output according to the following, and the reference article dimension in the image obtained by photographing is corrected with the value of the face position, and the corrected reference article dimension, Real face dimensions are obtained according to the dimensions of the face on the image and the actual dimensions of the reference item, and the 3D glasses model is synthesized according to the 3D user's face model reflecting the actual face dimensions.

Description

Eyeglasses try-on simulation method}

The present invention relates to a spectacle wearing simulation method capable of showing the appearance of an actual spectacled face by synthesizing a 3D spectacle model of spectacles according to a face of a photographed image based on an accurately measured face dimension.

Recently, online store sales have decreased, online store sales have increased rapidly, and the types of items that can be purchased online are also on the trend of becoming more diversified. Accordingly, it is possible to purchase most items online.

Taking advantage of this trend, it has become possible to purchase desired glasses from among various types of glasses online.

However, in order to check whether it fits well in online purchase, there is a tendency to avoid online purchase of eyeglasses, which is recognized as a fashion product, in particular, because it is not possible to actually try on them.

Accordingly, a simulation technology was developed that overlaps (or synthesizes) the photographed face image with the glasses image to resemble a face wearing the actual glasses, and online sales using this technology were also started, so that various types of glasses were sequentially sold. It is now possible to select and purchase glasses that match while looking at the virtual image worn by him. In addition, in order to focus the glasses on the pupils, a technology that allows the purchase of glasses that match the PM value, which is the interval between the two pupils, has also been developed.

However, since the dimensions of each part of the glasses are determined at the manufacturing stage, the glasses image must be reduced or enlarged according to the dimensions of the face in the image to be synthesized. Accordingly, it is necessary to obtain the face dimensions. In the prior art, the dimensions can be obtained as a face image in the image by allowing the user to input the face dimensions or taking pictures with a constant distance between the camera and the face. In addition, it was possible to obtain the dimensions of the face image using the image dimensions of the object by photographing an auxiliary object for knowing the dimensions in the face image.

However, few people know the dimensions of their faces, and it is difficult to shoot according to the shooting distance, and an error due to the difference in the shooting direction occurs only with the photographed image of an auxiliary object that knows the dimensions, so that the facial dimensions can be accurately obtained. Therefore, an error may occur in the obtained face dimension, and eventually, the glasses image combined with the face image may not match the face and may become awkward.

Accordingly, there is a need for a technology that allows anyone to easily measure the face dimension and show the wearing of glasses accurately matched to the shape of the face.

KR 10-2001-0008011 A 2001.02.05. KR 10-2004-0097349 A 2004.11.17. KR 10-2017-0010985 A 2017.02.02.

Accordingly, an object of the present invention is to provide a spectacle wearing simulation method capable of simply and accurately measuring a face dimension and showing a virtual spectacle wearing state in which a spectacle image having a predetermined dimension is accurately matched to the face dimension.

In order to achieve the above object, the present invention generates a virtual face wearing glasses by the image processing unit 230 using an image obtained by photographing a user's face with a camera 130 and a 3D glasses model of glasses whose actual dimensions are determined. In the spectacle wear simulation method shown by the display unit 140, the center of the front face and the reference article should be aligned with the face contour 141 and the reference article contour 142 output to the display unit 140 according to a preset arrangement. A photographing step (S100) of acquiring a front image captured at the time and an activity image photographed while turning the face through the camera 130; A face model acquisition step of obtaining a 3D user face model by recognizing a face from an activity image (S200); A reference article dimension correction step (S300) of obtaining a correction reference article dimension obtained by correcting the dimension of the reference article by the value of the face position in the front image; A face dimension acquisition step (S400) of determining a dimension of a 3D user's face model based on a ratio between the face dimension and the correction reference article dimension in the front image, and a preset actual dimension of the reference article; A simulation step (S600) of synthesizing a 3D glasses model whose size is adjusted according to a ratio between the dimensions of the face in the image and the dimensions of the 3D user's face model when photographing a face, and displaying it on the display unit 140; Includes.

According to an embodiment of the present invention, the reference article is a credit card whose actual size is known, and the photographing step (S100) recognizes both pupils or both ears, and the face contour 141 is symmetrical to the center of the front face. Aligned to the center, the reference article is photographed when it meets the reference article outline 142 to obtain a front image.

According to an embodiment of the present invention, in the photographing step (S100), the reference item outline 142 is superimposed on either side of the left and right of the face outline 141, so that the front face in which the reference item is disposed on the outer side of one eye. Obtaining a front image photographed, and in the step of obtaining face size (S400), 3D according to the relative ratio between the PD value in the front image and the size of the reference product, and the actual PD value obtained based on the preset actual size of the reference product. Determine the dimensions of the user's face model.

According to an embodiment of the present invention, the photographing step (S100) further acquires a complementary image photographed together by different photographing directions of a face and a reference item when photographing a front image, and the facial dimension obtaining step (S400) Obtains a depth difference between the face and the reference article in the complementary image, and corrects the relative ratio between the face dimension on the front image and the correction reference article dimension according to the depth difference.

According to an embodiment of the present invention, the complementary image is a side image taken from the side in a state in which the reference item is placed in front of the face so that the face is not covered from the side.

According to an embodiment of the present invention, a glasses model correction step of generating a corrected 3D glasses model by varying a deformable portion of a 3D glasses model according to a 3D user's face model having the dimensions determined in the step of obtaining the face dimensions (S400). (S500); It further includes, and applies the corrected 3D glasses model to the simulation step (S600).

According to an embodiment of the present invention, in the correction of the 3D glasses model in the glasses model calibration step (S500), in order to wear the glasses according to the 3D user's face model, a bridge connecting both rims is bent. Bending a joint that is fixed to a rim so that a hinge can be installed at the end, adjusting the angle of the temple by means of a hinge, and adjusting the temple It includes at least one of bending, and bending a tip provided at an end of a temple so as to be hung on the ear.

According to an embodiment of the present invention, in the step of correcting the dimensions of the reference article (S300), the angle between the direction 132 of the reference article and the optical axis of the camera 131 is the ratio between the corrected reference article dimension and the reference article dimension in the still image. After setting the equation expressed as in advance, obtaining the direction 132 of the reference article according to the position of the reference article in the still image, and obtaining the angle between the direction 132 of the reference article and the optical axis of the camera 131, the angle By applying the ratio corresponding to the calibration standard article dimensions are obtained.

The present invention made as described above is an image dimension that can be obtained by photographing a reference item at a face position by a simple pre-operation of photographing a front face and a reference item together in accordance with the face outline and the reference item outline at which the position on the screen is fixed. Can be obtained by using a reference item of a certain size and a preset arrangement of outlines, and the numerical value of the front face can also be accurately measured.Based on this, the dimensions of the 3D user's face model can be accurately obtained, and glasses with fixed dimensions can be obtained. It can be shown as if it was worn on a real face.

According to an embodiment of the present invention, even if the face size is different, a method of fitting a reference article to a reference article outline, a method of photographing with the same depth of the eyes and the reference article, or an image taken at a different angle from the front By utilizing the method of obtaining the difference in depth between the reference item and the face, it is possible to obtain the face dimensions more easily and more accurately.

According to an embodiment of the present invention, since the 3D glasses model is corrected according to the user's face, it can be shown as if wearing the glasses according to the actual shape of the face.

1 is a block diagram of an exemplary spectacle wearing simulation system for realizing a spectacle wearing simulation method according to an embodiment of the present invention.
2 is a flow chart of a method for simulating wearing glasses according to an embodiment of the present invention.
Fig. 3 is a screen (a) for obtaining a front image, and a screen (b) for obtaining a front image.
4 is a screen while obtaining a complementary image.
5 is a screen while an activity image is being acquired.
6 is a view showing a difference in image dimensions in a camera image appearing according to an arrangement of a reference article for explanation of the reference article dimension correction step (S300).
7 is a perspective view and a top view image (a) obtained from a 3D glasses model before wearing and a top view image (b) obtained from a 3D glasses model worn on a 3D user's face model for explanation of the glasses model correction step (S500). A drawing showing.
8 is a screen of the simulation step (S600).
9 is a screen for photographing a still image for photographing a reference article according to the depth of the eye.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. In describing an embodiment of the present invention, if it is determined that a detailed description of a related known function or a known configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

The spectacle wear simulation method according to the present invention is a method for showing a virtual spectacle wearing state by synthesizing the 3D spectacle model 30 of the spectacles to the image of the user's face 10, using a reference article 20 with known dimensions. Thus, a 3D user face model accurately matched to the dimensions of the user's face 10 is obtained, and the image 30' of the 3D glasses model 30 adjusted in size and direction to fit the user's face 10 in the captured image is overlapped. It shows the closest appearance to the face 10 wearing glasses.

Here, the glasses of the 3D glasses model 30 are glasses whose dimensions are determined, for example, glasses that have dimensions already determined as ready-made products or products to be sold, glasses that do not change the dimensions of major parts even if they are made to order, or made to order. Even if it is, it may be glasses with a limited change in size.

The present invention uses a front image obtained by photographing the front face 10 and the reference article 20 together, but the arrangement of the front face 10 and the reference article 20 is specified in the outlines 141 and 142 in advance. By allowing them to shoot in a matched state, the dimensions of the face can be accurately obtained using the front image. Based on this, the exact actual dimensions of the 3D user's face model 40 are obtained, and the 3D glasses model 30 of the glasses whose actual dimensions are determined can be matched with the 3D user's face model 40, and based on this, The synthesized image can be displayed by adjusting the size and direction of the 3D glasses model 30 according to the dimensions of the face.

A system for simulating wearing glasses for realizing such a specific embodiment of the present invention will be described first by way of example.

1 is a block diagram of a system for simulating wearing glasses.

The spectacle wearing simulation system illustrated in FIG. 1 is an example of a system for online purchase by connecting to a server 200 built for online sales through a network through a user terminal 100, and a spectacle wearing simulation method according to the present invention Equipped with the components for realizing the glasses, it is possible to select and purchase glasses while experiencing virtual wearing of glasses.

The user terminal 100 can access the server 200 through a network as illustrated, and may be a terminal equipped with a photographing function and a user interface function. To this end, the communication module 120, the camera 130, and the display The unit 140 and the control unit 110 for controlling the operation of the user terminal 100 may include a smart phone, a computer, a laptop, and the like.

According to an embodiment of the present invention, the user terminal 100 displays a list of glasses sold by accessing the server 200 on the display unit 140, and performs a pre-operation required when receiving a virtual glasses wearing service. After performing, receive virtual glasses wearing service, and choose glasses.

Here, the preliminary operation is a front image obtained by photographing the user's front face and a reference item together, a supplementary image obtained by additional photographing to know the difference in depth, which is the difference in the distance between the front face and the reference item from the camera, and 3D of the face. In order to obtain a model, an activity image obtained by photographing while turning a face is acquired through the camera 130 and transmitted to the server 200.

The virtual glasses wearing service selects glasses to be worn virtually and informs the server 200, acquires motion information of the user's face 10 photographed by the camera 130 in real time, transmits it to the server 200, and transmits it to the server 200. A function of displaying the synthesized and generated image on the real-time display unit 140 is performed.

Here, the motion information of the user's face 10 is obtained as a result of comparing the 3D user's face model with the 3D user's face model by receiving the 3D user's face model from the server 200 and then performing facial recognition processing on the image captured by the camera 130. However, the present invention is not limited thereto, and may be obtained by transmitting the camera 130 image to the server 200 and performing facial recognition processing in the server 200.

Of course, it is possible to receive virtual glasses wearing service by changing glasses.

Such a user terminal 100 is a smartphone or tablet PC installed with an application capable of performing the above-described pre-operation and virtual glasses wearing service operation, or the pre-operation and virtual wearing service operation under connection with the server 200. It can be implemented as a computer or laptop equipped with a camera capable of performing.

The server 200 includes an online sales unit 210 that provides a conventional online purchase service, and further includes a glasses model storage unit 220 and an image processing unit 230 for the implementation of the present invention.

The online sales unit 210 shows a list of glasses to be sold to the user through the user terminal 100, shows detailed information of the glasses selected through the user terminal 100, and performs payment and delivery processing of the glasses determined to be purchased. Since it is a typical configuration to be performed, detailed descriptions are omitted. However, according to the present invention, there is a difference in that the virtual glasses wearing service of the selected glasses is implanted.

The glasses model storage unit 220 stores 3D glasses models each generated for various types of glasses whose actual dimensions are determined. The stored 3D glasses model includes not only the 3D data of the glasses, but also dimension data of each part.

The image processing unit 210 includes a face recognition unit 231 that processes a photographed image or image of a user's face with a conventional facial recognition algorithm, and a 3D user face model of the user's face with the help of the face recognition unit 231. The face model acquisition unit 232 that acquires the image, or a single front image, or both the front image and the complementary image, are processed by the face recognition unit 231 to obtain the dimensions within the image of the reference product and based on the A dimension calculation unit 233 that determines dimensions and determines the dimensions of the 3D user's face model, and a glasses model correction unit 234 that generates a corrected 3D glasses model by varying the deformable portion of the 3D glasses model according to the 3D user's face model. ) And a virtual experience support unit 235 showing an image of a 3D glasses model synthesized according to the movement of a user's face in an image captured in real time through the user terminal 100, that is, an image of a face wearing virtual glasses through the user terminal 100 Includes.

The present invention consisting of a simulation system for wearing glasses configured as described above will be described with reference to FIGS. 2 to 9.

In the following, a detailed description of the online sales process will be omitted, and a method of simulating glasses wearing for providing a virtual glasses wearing service during the online sales process will be mainly described, and the user terminal 100 will be described as being configured with a smartphone.

2 is a flowchart of a method for simulating wearing glasses according to an embodiment of the present invention.

Fig. 3 is a diagram illustrating a screen (a) for obtaining a front image and a screen (b) during obtaining a front image in order to explain the front image acquisition step (S120) in the photographing step (S100).

4 is a diagram illustrating a screen while obtaining a supplementary image in order to explain the step S130 of obtaining a supplementary image in the photographing step S100.

FIG. 5 is a diagram illustrating a screen while obtaining an activity image in order to explain the activity image acquisition step S110 of the photographing step S100. FIG.

6 is a view showing the difference in dimensions of the reference article image in the camera image according to the arrangement of the reference article in order to explain the reference article dimension correction step (S300).

7 is a perspective view and a top view (a) of the 3D glasses model before wearing and a top view (b) of the 3D glasses model worn on the 3D user face model after being partially deformed in order to explain the glasses model correction step (S500). It is an illustrated drawing.

8 is a diagram illustrating a screen of the simulation step S600 in order to explain the simulation step S600.

As shown in FIG. 2, in the method of simulating wearing glasses according to an embodiment of the present invention, a photographing step (S100), a face model acquisition step (S200), and a reference product dimension correction step performed as a pre-action for modeling a user's face. (S300) and facial dimension acquisition step (S400), and glasses model calibration step (S500) and simulation step (S600) performed when a virtual glasses wearing service is requested by selecting glasses from the user terminal 100 (S10). Including, and when the user terminal 100 selects to change the glasses (S20), the glasses model calibration step (S500) and the simulation step (S600) are performed on the changed glasses.

The photographing step (S100) is a step of receiving an activity image, a front image, and a supplemental image obtained by shooting with the camera 130 through the user terminal 100 to the server 200, and thus, the activity image acquisition step (S110) , A front image acquisition step (S120) and a supplementary image acquisition step (S130), and in order to obtain a necessary image and image, the user must guide the camera 130 to align the direction or follow an appropriate photographing method. To this end, how to use each step is guided through the display unit 140 or a speaker (not shown).

In the activity image acquisition step (S110), as shown in FIG. 5, the face 10 is photographed with the camera 130 (front camera of the smartphone) and the head is turned left and right while the photographed image is displayed on the display unit 140. This is the step of obtaining an activity video shot while turning your face by guiding.

The front image acquisition step (S120) is a step of obtaining a front image by photographing the front face 10 and the reference article 20 together, and the front face 10 and the reference article 20 are preset on the front image. It is obtained by shooting when placed in a position according to.

Referring to FIG. 3(a), in the screen displayed by the display unit 140 at the beginning of the front image acquisition step (S120), a face outline 141 and a reference item outline 142 are designated positions on the screen according to a preset arrangement. Are indicated in each. Of course, when the zoom function of the camera 130 is used, the outlines 141 and 142 are also enlarged or reduced, and the angle of view according to the zoom may be reflected in each step to be described later, but it will be described that the zoom function is not used.

In order to obtain the front image, first, while displaying the image captured by the camera 130 on the display unit 140, align the user's front face 10 with the center of the face contour 141, and then set the reference article 20 to the reference article contour. Instruct them to align to the center of (142). Accordingly, the user will try to align the front face 10 with the center of the facial contour 141 by changing the direction of the camera 130, and although not shown in the drawing, the position of the reference item 20 held in one hand is changed to We will try to center (20) the reference article contour 142.

As shown in Fig. 3(b), if the front face 10 is aligned with the center of the facial contour 141 and the reference article 20 is aligned with the center of the reference article contour 142, a photographic image obtained at this time Is the front image.

In order to center the user's face 10, the user terminal 100 allows the controller 110 to execute a program that recognizes both pupils or both ears, or is provided in the image processing unit 200 of the server 200. By using the face recognition unit 231, it is possible to find out the center of the front face 10 (a vertical line that substantially divides the left and right), and make sure that the center of the front face 10 is aligned with the center of the outline by making the front face 10 symmetrical. . Accordingly, a moment when the front face 10 is aligned with the center of the face outline 141 can be photographed. Such centering and left-right symmetry will be possible when the optical axis of the camera 130 passes the center of the user's face 10 in the left-right width direction.

In order to align the center of the reference article 10, an image processing function for extracting the outline of the reference article 10 may be provided.

On the other hand, when the reference article 20 matches the reference article outline 142, it is preferable to obtain a front image. That is, since the face 10 has different dimensions and shapes for each user, it is satisfactory if it is aligned with the center of the face contour 141, but the reference article 20 of known dimensions is aligned with the reference article contour 142, so that it can be arranged in a preset arrangement. Accordingly, the distance between the camera and the reference article 20 also has a constant value, and as a result, it is possible to more easily obtain the dimensions of the face 10 as described below with reference to FIG. 6.

It is preferable that everyone owns the reference item 20 and always carries it, and has the same physical dimensions. In the embodiment of the present invention, a credit card known to have the same physical dimensions is used for any card company. Of course, the reference article outline 142 matches the ratio of the width and the top and bottom length to the ratio of the credit card, and the dimensions of the credit card are set in advance.

Here, the dimensions of the face 10 can be viewed as values including dimensions for each part of the face 10, such as face width, vertical length, glabellar width, and PD value (the gap between two pupils), and the dimensions of the reference article 20 Can be viewed as values such as width and length, and it is better to set it based on any one of the dimensions of each part. In order to easily understand the description of the embodiment of the present invention, the dimensions on the front image may be the PD value of the face 10 and the width of the reference article 20. In this case, the above-described preset arrangement may be determined so that the upper end of the reference article 20 is aligned on the extension line of the two pupils. That is, making the face part for which the dimensions are to be known be placed on the same line as the part of the reference article 20 for which the dimensions are to be compared simplifies the dimension calculation formula.

However, since it is difficult to lie on the same line, in an embodiment of the present invention, the depth is artificially different on the front image, and the complementary image acquisition step (S130) is added to obtain the depth.

In the step of obtaining the supplementary image (S130), as shown in FIG. 4, the reference item is placed in front of the face so that it is not covered by the face when photographing with the side face from the side, and the photographic image obtained at this time is taken. Make it a complementary image. Of course, after guiding to take a posture to be photographed, the front image acquisition step (S120) is performed, and then, the supplementary image acquisition step (S130) by guiding to take a side image while maintaining the same posture. To do. When obtaining a complementary image, the outlines 141 and 142 are not required.

Referring to FIG. 4, if the PD value is based on the dimensions of each part of the face, the difference in depth between the pupil (or eye) and the reference article 20, that is, the difference in depth of both images on the front image (d) can be obtained. .

However, the complementary image may not be photographed from the side, and may be obtained by simultaneously photographing a face and a reference article in a photographing direction of a different angle. That is, even when photographing in a direction different from the photographing direction at the time of obtaining the front image, a difference in depth can be obtained through image processing. However, in order to get the depth difference more easily and accurately, it is better to shoot from the side.

Additionally, referring to FIGS. 3 to 6, by displaying various types of glasses list 143 on the display unit 140, the glasses of the glasses list 143 can be selected one by one.

On the other hand, the order of acquisition of the active image and the front image may be changed. Of course, guides should be provided in order, and complementary images should be obtained following the front image.

The activity image, the front image, and the complementary image obtained as described above are transmitted to the image processing unit 220 of the server 200, so that the face model acquisition step (S200), the reference product dimension correction step (S300), and the face dimension acquisition step (S400) Used for

In the face model acquisition step (S200), the face recognition unit 231 processes the received active image to recognize the face, thereby obtaining feature points of each region of the face, and the face model acquisition unit 232 This is the step of obtaining a 3D user's face model according to the recognition result. The relative position of the features is also determined by the 3D user's face model obtained as described above, and in particular, the position of the face to be worn with glasses can be known. Since the facial recognition technology and the technology for obtaining a 3D user face model are known technologies, detailed descriptions are omitted.

In the reference product dimension correction step (S300), the front image is processed by the face recognition unit 231 to recognize the face, the reference product also recognizes the outline, and the face recognition unit 231 recognizes the dimension calculation unit 233 This is a step of obtaining the dimensions of the reference article on the front image according to the result and then correcting according to the above-described preset arrangement. Of course, the dimensions on the front image are obtained by the number of occupied pixels.

Referring to FIG. 6(a), the above-described preset arrangement is applied when obtaining a front image by simultaneously photographing the front face 10 and the reference article 20, so that the front face is The center of (10) is placed on the optical axis 131 of the camera 130, and thus, the reference article 20 is placed in a direction 132 away from the optical axis. Accordingly, the shooting range 1 of the reference article 20 on the still image is the shooting range when the virtual reference article 20 ′ having the reference article 20 placed at the position of the face 10 is photographed (2) And, as a result, the dimensions of the still image also differ according to the difference in placement.

As illustrated in FIG. 6(b), even if the image is taken after placing the reference article 20 in line with the face 10, it is the same. The example of FIG. 6(b) is attached for understanding of the description with reference to FIG. 9 below.

Here, it is more accurate to obtain the dimensions of the face 10 by using the dimensions that can be obtained on the still image when photographing the virtual reference article 20' in which the reference article 20 is placed at the position of the face 10. It is advantageous to get the dimensions.

For this reason, by performing the reference article dimension correction step (S300), a correction reference article dimension obtained by correcting the dimension of the reference article in the front image by the value of the face position is obtained.

For example, the direction 132 of the reference article 20 can be obtained according to the position of the reference article 20 shown on the still image, and the correction reference article dimension is the direction 132 of the reference article 20 and the optical axis of the camera. (131) It can be obtained by establishing an equation that is calculated according to the angle between. This is because the dimensions on the photographed image vary according to the angle difference. For example, the ratio between the correction reference product dimension and the reference product dimension on a still image may be obtained for each angle between the direction 132 of the reference product 20 and the optical axis 131 of the camera, and defined as a value determined according to the angle. . On the other hand, when the reference article 20 is photographed in accordance with the reference article outline 142 as described above, since the distance between the camera 130 and the reference article 20 is determined by the above-described preset arrangement, the correction standard The equation for obtaining the article dimensions can be further simplified.

The facial dimension acquisition step (S400) is a dimension ratio acquisition step (S410) of acquiring a ratio between the dimensions of the front face 10 obtained on the still image and the correction reference product dimensions obtained in the reference product dimension correction step (S300), FIG. As described with reference to 4, the depth difference (d) between the face 10 and the reference article 20 is obtained using a complementary image and reflected in the dimensional ratio between the front face 10 dimension and the corrected reference article dimension. A ratio correction step (S420) of correcting the ratio, and dimension alignment in which the actual face dimension is determined based on the corrected dimension ratio and the preset actual dimension of the reference article 20, and the determined face dimension is applied to the 3D user's face model It includes step S430.

When describing a specific embodiment, the dimensional ratio to be obtained in the dimensional ratio obtaining step (S410) is obtained by the following equation.

In addition, referring to FIG. 6, the ratio correction step (S420) will be described, the shooting range 2 of the virtual reference article 20 ′ assumed to be placed on the camera optical axis 131 can be known, and the virtual Since the dimension of the reference article 20' is obtained as the correction reference article dimension, the distance from the virtual reference article 20' to the camera 130 can be obtained by a trigonometric function. Accordingly, using the principle that the image of the article in the captured image decreases as the article of the same size moves away from the camera, the virtual reference article ( 20') can be calculated on the front image of the city when it is retracted away from the camera.

That is, when the dimensions on the front image that are to be obtained when retracted are applied to the correction reference product dimensions of Equation 1, the final dimensional ratio is obtained. The dimensional ratio corrected in this way may be obtained by using the distance from the camera to the virtual reference article 20' and the difference in depth as variables.

However, if the above-described preset arrangement is applied so that the reference article 20 is placed on the extension line in the width direction of the face 10 and photographed, a complementary image is not obtained, and thus the ratio adjustment step S420 is not performed.

In the dimension fitting step (S430), the dimension ratio reflecting the difference in depth is applied. That is, the actual user's face dimension can be obtained by Equation 2 below, which is multiplied by a dimension ratio by a preset actual dimension of the reference article 20 (that is, the actual dimension of the reference article).

For example, when the face size is obtained as a PD value, the size of each part of the model is also determined by applying the PD value at this time to the 3D user's face model. That is, the 3D user face model is configured to fit the actual scale of the user's face.

Next, when glasses are selected in the user terminal 100 (S10), the 3D glasses model 30 of the selected glasses is loaded from the glasses model storage unit 220, and the glasses model calibration step (S500) and the simulation step (S600) ).

The glasses model correction step (S500) is performed by transforming the 3D glasses model 30 having real dimensions to fit the 3D user face model 40 adapted to the dimensions of the user's face. Give it a look.

That is, since the 3D user face model 40 obtained above is obtained as a model whose relative position and face width between eyes, ears and nose are different for each user, it is transformed according to the relative position and face width between eyes, ears and nose. .

Referring to FIG. 7(a), the 3D glasses model 30 includes lenses 31, rims 32, 33: bridges 33, and nose pads according to the structure of the actual glasses. 34), joint (35), 36: has a hinge (36), a leg (temple, 37) and a tip (38).

And, prior to actually wearing the real glasses on the user's face, the bridge connecting the rims on both sides according to the dimensions of the user's face is bent or provided at the end of the temple so that it can be hung on the ear. The tip is bent, the joint is fixed to the rim so that a hinge can be installed at the end, or the leg is bent, and when worn, a hinge The angle of the temple is adjusted by. That is, when the glasses are actually worn, the deformable part of each part is transformed to fit the user's face.

Accordingly, as illustrated in FIG. 7(b), the 3D glasses model 30 for the glasses in the actual worn state is obtained by performing the glasses model calibration step (S500). 7(b) shows a state in which the joint 35 is bent and the angle of the leg 37 is adjusted according to the dimensions of the 3D user's face model 40, but other parts may also be modified.

Here, the 3D glasses model 30 obtained by calibrating is applied to the simulation step S600.

In the simulation step (S600), when a face is photographed in the user terminal 100, the photographed image is processed by the face recognition unit 231 to recognize the face to obtain the face dimension in the image, and the face dimension in the image and the 3D user face model dimension The size of the 3D glasses model is adjusted and synthesized according to the ratio therebetween, and the synthesized image is output to the display unit 140 of the user terminal 100. Of course, it is synthesized according to the direction of the face in the captured image.

Thus, as shown in FIG. 8, each frame of the image output to the display unit 140 of the user terminal 100 is a glasses image 30' obtained from the 3D glasses model 30 to the user's face image 10. It is expressed as a synthetic one.

As described above, a technology for displaying a virtual glasses experience image using the predetermined 3D glasses model 30 and the 3D user face model 40 is a technology known by the prior art, and a detailed description thereof will be omitted.

The facial recognition of the captured image may be performed in the user terminal 100, and in this case, the relative position and spacing of the features obtained by facial recognition are treated as information on the dimensions of the face in the image, and the image of the server 200 It may be received from the processing unit 220 and used.

While performing the simulation step (S600), if the user terminal 100 changes the glasses to be virtually experienced, the 3D glasses model of the changed glasses is retrieved from the glasses model storage unit 220, and the glasses model calibration step (S500) ) And performs the simulation step (S600) in which the corrected 3D glasses model is applied.

9 is the depth between the face 10 and the reference article 20 by arranging the reference article 20 on the extension line in the width direction of the front face 10 as described above and with reference to FIG. 6(b) When obtaining a front image that does not have a difference d, the arrangement of the face contour 141 and the reference article contour 142 minimizing the depth difference d is shown.

Referring to FIG. 9, the reference article outline 142 is overlapped on either side of the left and right of the face outline 141, but overlapped so as to be skewed outward in the width direction at the position of one eye.

In the case of using such an arrangement of the facial contour 141 and the reference article contour 142, a guide is given to put the reference article 20 close to the side of one eye while placing the reference article 20 on the extension lines of both eyes. Accordingly, it is possible to obtain a front image of the reference article 20, which is placed on the extension line of both eyes, taken together with the face.

In addition, in the facial dimension obtaining step (S400), as described with reference to FIG. 6(b), a virtual reference item that can be obtained when the image of the reference item 20 recognized in the front image is taken in alignment with the center of both eyes. The dimension of (20'), that is, the correction reference product dimension, is calculated, and a PD value is obtained among the face dimensions on the front image. Then, a relative dimensional ratio between the PD value and the correction reference article dimension is obtained. There is no need to perform the dimensional ratio correction step (S420) according to the depth difference (d). Then, based on the dimensional ratio and a preset actual size of the reference article, an actual PD value is obtained, and a 3D user face model having the same PD value as the actual PD value is obtained.

However, the arrangement of the facial contour 141 and the reference article contour 142 may be different from the example of FIG. 9, and for example, the reference article 20 is attached to the end of one ear in the width direction or placed close to each other for a photograph. You can also set the layout to do so. In this case, the size of the face to be obtained using the reference product is preferably the distance between the ends of both ears.

However, since spectacles are worn by fitting the lenses to the eyes, it is preferable to obtain a PD value among the face dimensions using a reference article.

In the above, an embodiment in which the present invention is applied to online sales of glasses has been described, but the present invention is not limited thereto, and it is obvious that the present invention can be applied to various fields requiring virtual glasses wearing experience.

In addition, in the embodiment of the present invention described above, the image processing unit 230 is provided in the server 200, but may be provided in the user terminal 100. In this case, the server 200 provides an online purchase service and glasses. Just provide the model.

Although shown and described in specific embodiments to illustrate the technical idea of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiment as described above, and various modifications are within the scope of the present invention. Can be carried out in Therefore, such modifications should be regarded as belonging to the scope of the present invention, and the scope of the present invention should be determined by the claims to be described later.

10: face
20: reference article
30: 3D glasses model
31: lens 32: rim 33: bridge
34: nose pad 35: joint 36: hinge
37: leg (temple) 38: tip (tip)
40: 3D User Face Model
100: user terminal
110: control unit
120: communication module
130: camera
140: display unit 141: face contour 142: reference article contour
143: glasses list
200: server
210: online sales department
220: glasses model storage unit
230: image processing unit

Claims (8)

Using the image obtained by photographing the user's face with the camera 130 and the 3D glasses model of glasses with the actual dimensions determined, a virtual face wearing glasses is created by the image processing unit 230 and displayed on the display unit 140. In the simulation method,
The reference item is a credit card of known actual size,
In the image processing unit 230, the facial contour 141 and the reference article contour 142 are respectively output to a designated position on the screen of the display unit 140 according to a preset arrangement, and the reference article is placed in front of the face to be placed on the face from the side. After guiding to the display unit 140 so as not to be covered, the front face and side faces are taken by aligning the center of the front face with the center of the face outline 141 and aligning the center of the reference item with the center of the reference item outline 142 A complementary image photographed and an activity image photographed while turning a face are acquired through the camera 130, but the front image recognizes both pupils or both ears, so that the center of the front face is symmetrically positioned at the center of the face outline 141. A photographing step (S100) of photographing and obtaining the reference article when the reference article is centered on the reference article outline 142;
A face model acquisition step (S200) of obtaining a 3D user face model by recognizing a face from an active image in the image processing unit 230;
In the image processing unit 230, a correction reference article dimension obtained by correcting the dimension of the reference article in the front image with the value of the face position is obtained, and the ratio between the correction reference article dimension and the reference article dimension in the front image is determined as the direction of the reference article ( 132) and the camera optical axis 131 by setting an equation expressed as the angle between the camera optical axis 131, and obtaining the direction 132 of the reference article according to the position of the reference article in the front image. After obtaining the angle between (131), a reference article dimension correction step (S300) of obtaining a correction reference article dimension by applying a ratio corresponding to the angle;
In the image processing unit 230, by obtaining a relative ratio between the face dimension and the correction reference item dimension in the front image, and obtaining a depth difference between the face and the reference item in the complement image, the face dimension on the front image and the correction reference item dimension A facial dimension obtaining step (S400) of correcting the relative ratio between the two according to the difference in depth, and determining the dimensions of the 3D user's face model based on the corrected relative ratio and a preset actual dimension of the reference article;
The image processing unit 230 synthesizes a 3D glasses model whose size is adjusted according to the ratio between the size of the face in the image and the size of the 3D user's face model when photographing a face, and displays it on the display unit 140 Step S600;
Including
How to simulate wearing glasses. delete delete delete delete The method of claim 1,
A glasses model correction step (S500) of generating a corrected 3D glasses model by varying the deformable portion of the 3D glasses model according to the 3D user's face model having the dimensions determined in the face dimension obtaining step (S400);
Including more,
Applying the corrected 3D glasses model to the simulation step (S600)
How to simulate wearing glasses. The method of claim 6,
In the glasses model calibration step (S500), the calibration of the 3D glasses model is
In order to wear the glasses according to the 3D user's face model, a bridge connecting the two rims is bent, a joint that is fixed to the rim so that a hinge can be installed at the end. Bending, adjusting the angle of a temple by a hinge, bending a temple, bending a tip provided at the end of a temple so that it can be hung on the ear Containing at least any one of
How to simulate wearing glasses. delete

Images