CN114821754B - Semi-closed eye image generation method and device, readable storage medium and electronic equipment - Google Patents
Semi-closed eye image generation method and device, readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114821754B CN114821754B CN202210452365.2A CN202210452365A CN114821754B CN 114821754 B CN114821754 B CN 114821754B CN 202210452365 A CN202210452365 A CN 202210452365A CN 114821754 B CN114821754 B CN 114821754B
- Authority
- CN
- China
- Prior art keywords
- eyelid
- template image
- curve equation
- new
- contour curve
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/197—Matching; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
A semi-closed eye image generation method, a device, a readable storage medium and an electronic device, wherein the method comprises the steps of scaling an eyelid template image, and obtaining coordinates of a plurality of eyelid key points in the eyelid template image and standard vectors of the eyelid key points; randomly moving each eyelid key point, and obtaining new coordinates of each eyelid key point after random movement; determining a corresponding curve equation according to the new coordinates of each eyelid key point to obtain a target eyelid contour curve equation; adjusting the eyelid template image according to the target eyelid contour curve equation to obtain an eyelid template image matched with the target eyelid contour curve equation; and (3) performing translation processing on the adjusted eyelid template image, and covering pixels of a corresponding area in the standard open-eye image to obtain a semi-closed-eye image. The pupil labeling method is simple to operate, high in efficiency and easy to obtain high-precision pupil labeling data.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and apparatus for generating a semi-closed eye image, a readable storage medium, and an electronic device.
Background
The pupil positioning method based on deep learning is widely applied because of the advantages of good robustness and strong adaptability. In an actual application scene, the pupil positioning method based on deep learning needs a large number of labeled samples to train a model so that an algorithm can be fully learned, and the diversity, quality and the like of the samples determine the effect of the algorithm in use. Among them, an image sample of a semi-closed eye (pupil is partially covered by eyelid or eyelash) is also an important sample in the through-hole positioning method.
In the case of semi-closed eyes, where the model is intended to output accurate pupil positions, model training requires an eye diagram dataset that is accurately labeled with pupil coordinate data. The prior art is to manually label or automatically label the pupil position of the semi-closed eye image by using an algorithm so as to obtain a semi-closed eye image data set labeled with pupil coordinate data. The manual annotation is manually marked by a person through an external input device on the computer to display the position of the target object in the image. The algorithm automatic labeling is carried out by processing the image by the designed algorithm, so that the position of the target object in the image is automatically marked. Because of the occlusion, pupil position labeling is performed on the semi-closed eye diagram, and high-precision labeling data is difficult to obtain, whether manual or algorithmic. Thus, an accurate semi-closed eye image dataset is also not available.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a semi-closed-eye image generation method, device, readable storage medium, and electronic apparatus, which address the problem that a semi-closed-eye image with pupil data accurately labeled cannot be obtained in the prior art.
A half-closed eye image generation method includes,
Scaling an eyelid template image to match the eyelid template image with the standard open eye image size, wherein the eyelid template image is an image of an eyelid area in a semi-closed eye state;
Obtaining coordinates of a plurality of eyelid key points in the scaled eyelid template image and standard vectors of the eyelid key points, wherein the standard vectors are normal vectors of the eyelid key points and vectors in opposite directions of the normal vectors on an eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by the eyelid key points;
Randomly moving each eyelid key point, and acquiring new coordinates of each eyelid key point after random movement, wherein the movement of each eyelid key point meets a first constraint condition, the first constraint condition is that an included angle between the movement direction of each eyelid key point and the standard vector in which the eyelid key point is positioned is smaller than a preset angle, and the movement distance of each eyelid key point is smaller than a preset distance;
Determining a corresponding curve equation according to the new coordinates of the eyelid key points to obtain a target eyelid contour curve equation;
adjusting the scaled eyelid template image according to the target eyelid contour curve equation to obtain a new eyelid template image matched with the target eyelid contour curve equation;
and performing translation processing on the new eyelid template image, and covering pixels of a corresponding area in the standard open-eye image to obtain a semi-closed-eye image.
Further, in the method for generating a semi-closed eye image, the step of obtaining the standard vector of each eyelid key point includes:
Determining a corresponding curve equation according to the coordinates of the eyelid key points to obtain an eyelid contour curve equation;
and calculating normal vectors of the eyelid contour curve equation at the eyelid key points, and calculating vectors with opposite normal vector directions to obtain standard vectors of the eyelid key points.
Further, in the above method for generating a semi-closed eye image, the step of adjusting the scaled eyelid template image according to the target eyelid contour curve equation includes:
And erasing pixels below the target eyelid contour curve equation in the scaled eyelid template image, and performing pixel complementation on blank areas above the target eyelid contour curve equation in the scaled eyelid template image.
Further, in the above method for generating a semi-closed eye image, the step of scaling the eyelid template image includes:
Calculating the ratio of the size of the eyelid in the eyelid template image to the size of the eyelid in the standard open eye image;
And scaling the eyelid template image according to the ratio.
Further, in the method for generating a semi-closed eye image, the step of performing translation processing on the new eyelid template image includes:
And translating the new eyelid template image on the x-axis so that the midpoint of the eyelid contour curve equation in the new eyelid template image and the pupil center of the standard open eye image are the same in the x-axis coordinate.
Further, in the method for generating a semi-closed eye image, the step of performing translation processing on the new eyelid template image includes:
Translating the new eyelid template image on a y-axis, wherein the translation amount of the new eyelid template image on the y-axis meets a second constraint condition, and the second constraint condition is as follows:
Wherein t y is the translation of the new eyelid template image on the y axis, y p is the coordinate of the pupil center on the y axis in the standard open eye image, w is the diameter of the pupil in the standard open eye image, f 2 (x) is the target upper eyelid contour curve equation, x p is the coordinate of the pupil center on the x axis in the standard open eye image, gamma is a constant, and the value range is (0, 1).
The invention also discloses a semi-closed eye image generation device, which comprises:
The scaling module is used for scaling the eyelid template image to be matched with the standard open eye image in size, and the eyelid template image is an image of an eyelid area in a semi-closed eye state;
the system comprises an acquisition module, a scaling module and a scaling module, wherein the acquisition module is used for acquiring coordinates of a plurality of eyelid key points in the scaled eyelid template image and standard vectors of the eyelid key points, wherein the standard vectors are normal vectors of the eyelid key points and vectors in opposite directions of the normal vectors on an eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by the eyelid key points;
The moving and acquiring module is used for randomly moving each eyelid key point and acquiring new coordinates of each eyelid key point after random movement, wherein the movement of the eyelid key point meets a first constraint condition, the first constraint condition is that an included angle between the moving direction of the eyelid key point and the standard vector where the eyelid key point is positioned is smaller than a preset angle, and the moving distance of the eyelid key point is smaller than a preset distance;
the determining module is used for determining a corresponding curve equation according to the new coordinates of each eyelid key point to obtain a target eyelid contour curve equation;
The adjusting module is used for adjusting the scaled eyelid template image according to the target eyelid contour curve equation to obtain a new eyelid template image matched with the target eyelid contour curve equation;
And the coverage module is used for covering pixels of a corresponding area in the standard open-eye image after the new eyelid template image is subjected to translation processing so as to obtain a semi-closed-eye image.
The invention also discloses a readable storage medium having stored thereon a computer program which when executed by a processor implements the method of any of the above.
The invention also discloses an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the method of any one of the above when executing the computer program.
According to the invention, the eyelid template image after processing is covered on the open eye image marked with the accurate pupil position, so that a high-precision semi-closed eye pattern marking data set is obtained. The semi-closed eye image generation method is simple to operate, high in efficiency and easy to obtain high-precision pupil labeling data. In addition, the semi-closed eye images in different eye closing states can be obtained through the position adjustment of the eyelid key points, and the semi-closed eye image data set is enriched.
Drawings
FIG. 1 is a flowchart of a half-closed eye image generation method according to an embodiment of the present invention;
FIG. 2a is an eye diagram;
FIG. 2b is an eyelid template image;
Fig. 3 is a standard open eye image;
Fig. 4 is a block diagram of a semi-closed eye image generating apparatus according to an embodiment of the present invention;
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
These and other aspects of embodiments of the invention will be apparent from and elucidated with reference to the description and drawings described hereinafter. In the description and drawings, particular implementations of embodiments of the invention are disclosed in detail as being indicative of some of the ways in which the principles of embodiments of the invention may be employed, but it is understood that the scope of the embodiments of the invention is not limited correspondingly. On the contrary, the embodiments of the invention include all alternatives, modifications and equivalents as may be included within the spirit and scope of the appended claims.
Referring to fig. 1, a half-closed eye image generating method according to an embodiment of the invention includes steps S11 to S16.
Step S11, scaling the eyelid template image to match the eyelid template image with the standard open eye image size, wherein the eyelid template image is an image of an eyelid area in a semi-closed eye state.
In specific implementation, firstly, an eyelid template image with clear eyelid outline is selected, wherein the eyelid template image is an image of an upper eyelid area in a semi-closed state. The eyelid template image may be an image with or without eyelashes. Taking an eyelid template diagram without eyelashes as an example, the specific procedure is as follows: an eye pattern with a clear outline of the eyelid and eyelashes is first selected (as shown in fig. 2 a) and the eyelid keypoints are labeled, and then the pixel values of the area under the upper eyelid are manually or automatically erased (pixel values set to 0 or 255) by using an image editing tool or algorithm to obtain an eyelid template image (as shown in fig. 2 b), or an eyelid image template is synthesized in other manners. The eyelid template map with lashes is made substantially the same as the eyelid template map without lashes, except that the pixel values belonging to lashes remain during the process of erasing the pixel values or synthesizing the eyelid image.
The standard open eye image is eye pattern data with the pupil uncovered by the eyelid or eyelashes, and the pupil position and size, and eyelid length information of the eye pattern data set are known, as shown in fig. 3, the eyelid length is L o, the pupil center position is P p＝(xp,yp), and the pupil width is W.
Since the eyelid template image may have an eyelid size that does not fit the eyelid size in the standard open eye image, the adjusted eyelid template image needs to be scaled. Specifically, in one embodiment of the present invention, the step of scaling the adjusted eyelid template image includes:
calculating the ratio of the eyelid size in the eyelid template image after adjustment to the eyelid size in the standard open eye image;
and scaling the adjusted eyelid template image according to the ratio.
The eyelid size in the eyelid template image is the distance L x between the eyelid key points at the two ends, the eyelid size in the standard open eye image is L 0, and the scaling ratio beta is calculated by the formula of beta=L 0/Lx. And performing scaling treatment on the eyelid template image according to the scaling ratio to obtain a new eyelid template image.
And S12, acquiring coordinates of a plurality of eyelid key points in the scaled eyelid template image and standard vectors of the eyelid key points. The standard vector is a normal vector and a vector in the opposite direction of the normal vector of the eyelid key points on the eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by the eyelid key points.
The eyelid template image is pre-labeled with a plurality of eyelid key points, and the eyelid key points mainly select points on the lower outline of the eyelid, for example, two points at two ends of the upper corner of the lower outline of the eyelid and several points in the middle area can be selected. Eyelid keypoint labeling can be automatically labeled by image processing technology or manually labeled by an image labeling tool.
The standard vector of each eyelid key point can be pre-calculated and stored in the system, or the system can automatically determine the standard vector of each eyelid key point according to the obtained coordinates of the eyelid key point. In one embodiment of the present invention, the step of obtaining the standard vector of each eyelid key point includes:
Determining a corresponding curve equation according to the coordinates of the eyelid key points to obtain an eyelid contour curve equation;
and calculating normal vectors of the eyelid contour curve equation at the eyelid key points, and calculating vectors with opposite normal vector directions to obtain standard vectors of the eyelid key points.
Specifically, the upper eyelid key point of the eyelid template image is P 0,P1,P2,…Pm, and interpolation or fitting means is adopted to obtain a curve equation f 1 (x) for representing the upper eyelid contour information, wherein the interpolation can be, but is not limited to, cubic B-spline interpolation, and the fitting can be, but is not limited to, least squares fitting. The normal vector n 0,n1,n2,…nm of the eyelid contour curve equation at the key point P 0,P1,P2,…Pm is then calculated separately. The purpose of computing the vector is to move the key points along the normal vector direction or a certain angle range of the opposite direction so as to simulate the outline shape of the upper eyelid of different people.
And S13, randomly moving each eyelid key point, and acquiring new coordinates of each eyelid key point after random movement. The movement of the eyelid key point meets a first constraint condition, wherein the first constraint condition is that an included angle between the movement direction of the eyelid key point and the standard vector where the eyelid key point is located is smaller than a preset angle, and the movement distance of the eyelid key point is smaller than a preset distance.
The eyelid key point moves randomly at one position in the moving direction, the moving direction of the eyelid key point is (+/-) (n+/-alpha i)), i epsilon (1, … m), alpha i is the offset of the normal vector n i, the moving distance of the eyelid key point in the moving direction is D i, the included angle between the moving direction (+/-) (n+/-alpha i) and the normal vector n i is smaller than theta, and D i is smaller than D. Wherein, θ and D are set according to actual conditions, θ generally takes a value of 5-15 °, and D is 5-10 pixels.
And S14, determining a corresponding curve equation according to the new coordinates of the eyelid key points to obtain a target eyelid contour curve equation.
And step S15, adjusting the scaled eyelid template image according to the target eyelid contour curve equation to obtain a new eyelid template image matched with the target eyelid contour curve equation.
After obtaining the new eyelid key points, a curve equation for representing the new upper eyelid contour information, namely a target eyelid contour curve equation f 2 (x), is obtained by adopting interpolation or fitting means, so as to be used for calculating the translation distance of the subsequent templates.
Since the eyelid key points are moved, the outline of the eyelid in the eyelid template image is slightly changed, for example, when the eyelid key points are moved upwards, the pixel points below the outline are redundant, and when the eyelid key points are moved downwards, a blank area is generated above the eyelid key points. Therefore, pixels below the target eyelid contour equation f 2 (x) in the eyelid template image will need to be erased, and the blank area above the curve equation f 2 (x) is complemented using image interpolation techniques so that the adjusted eyelid template image is consistent with the new upper eyelid contour equation curve.
It should be noted that, when each eyelid key point moves, the values of α i and di are random within a given range, and when each movement is completed, a new eyelid template image is correspondingly formed. In this way, eyelid template images of various forms can be obtained to enrich semi-closed eye image sample data.
And S16, performing translation processing on the new eyelid template image, and covering pixels of a corresponding area in the standard open-eye image to obtain a semi-closed-eye image.
Specifically, the new eyelid template image is subjected to proper translation processing according to the eyelid position and the semi-closed eye degree in the new eyelid template image. Translation of the new eyelid template image is predominantly in the x-axis (horizontal) movement to avoid center position offset when the new eyelid template image is overlaid with the standard open eye image. In one embodiment of the present invention, the step of performing a translation process on the new eyelid template image includes:
And translating the new eyelid template image on the x-axis so that the midpoint of the eyelid contour curve equation in the new eyelid template image and the pupil center of the standard open eye image are the same in the x-axis coordinate.
The coordinate of the middle point of the x-axis direction of the key points at the left and right ends of the new upper eyelid is AndThe coordinates of the x-axis of the two eyelid endpoints in the new eyelid template image are respectively. The pupil center of the standard open eye image has coordinates (x p,yp). Thus, the new eyelid template image translates in the x-axis by a distance t x,tx＝xp-Cx.
After the size and the position of the new eyelid template image are adjusted, the new eyelid template image is covered in the standard open-eye image, and at the moment, pixels of an eyelid area in the standard open-eye image are replaced by pixels of a corresponding area in the new eyelid template image, so that a semi-closed-eye image is obtained.
Because the coordinates of the pupil center in the standard open eye image can be accurately and easily marked, the pupil center in the half image is not easily and accurately marked due to shielding. Therefore, the embodiment covers the eyelid template image of the semi-closed eye subjected to the scaling and translation processing on the open eye image marked with the accurate pupil position, thereby obtaining a high-precision semi-closed eye pattern marking data set. The semi-closed eye image generation method is simple to operate, high in efficiency and easy to obtain high-precision pupil labeling data. In addition, the semi-closed eye images in different eye closing states can be obtained through the position adjustment of the eyelid key points, and the semi-closed eye image data set is enriched.
Further, in another embodiment of the present invention, in order to obtain a semi-closed eye pattern with different coverage degrees, a new eyelid template image may be further translated in a y-axis direction, and specifically, the step of performing a translation process on the new eyelid template image includes:
Translating the new eyelid template image on a y-axis, wherein the translation amount of the new eyelid template image on the y-axis meets a second constraint condition, and the second constraint condition is as follows:
Wherein t y is the translation of the new eyelid template image on the y axis, y p is the coordinate of the pupil center on the y axis in the standard open eye image, w is the diameter of the pupil in the standard open eye image, f 2 (x) is the target upper eyelid contour curve equation, x p is the coordinate of the pupil center on the x axis in the standard open eye image, gamma is a constant, and the value range is (0, 1).
To avoid image distortion, the translation of the new eyelid template image in the y-axis is also moved to a certain extent, i.e. inAndThe range moves between.
In the embodiment, a semi-closed eye diagram is obtained by adopting a mode of covering eyelids, and a high-precision semi-closed eye diagram marking data set is obtained by covering an eyelid template diagram on an open eye diagram marked with pupil positions. And, by automatically calculating the scaling factor of the eyelid template and the amount of translation along the x-axis and the y-axis, a semi-closed eye diagram of suitable and different coverage levels can be obtained.
Referring to fig. 4, a half-closed eye image generating apparatus according to an embodiment of the invention includes:
A scaling module 21, configured to scale an eyelid template image to match a standard open eye image size, where the eyelid template image is an image of an eyelid area in a semi-closed eye state;
the obtaining module 22 is configured to obtain coordinates of a plurality of eyelid key points in the scaled eyelid template image, and a standard vector where each eyelid key point is located, where the standard vector is a normal vector where the eyelid key point is located and a vector opposite to the normal vector on an eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by each eyelid key point;
A moving and acquiring module 23, configured to randomly move each eyelid key point, and acquire new coordinates of each eyelid key point after random movement, where the movement of the eyelid key point meets a first constraint condition, where the first constraint condition is that an included angle between a moving direction of the eyelid key point and the standard vector where the eyelid key point is located is smaller than a preset angle, and a moving distance of the eyelid key point is smaller than a preset distance;
A determining module 24, configured to determine a corresponding curve equation according to the new coordinates of each eyelid key point, so as to obtain a target eyelid contour curve equation;
an adjustment module 25, configured to adjust the scaled eyelid template image according to the target eyelid contour curve equation, so as to obtain a new eyelid template image matched with the target eyelid contour curve equation;
And the covering module 26 is configured to cover pixels of a corresponding area in the standard open-eye image after performing the translation processing on the new eyelid template image, so as to obtain a semi-closed-eye image.
The half-closed eye image generating device provided by the embodiment of the invention has the same implementation principle and technical effects as those of the embodiment of the method, and for the sake of brevity, reference may be made to the corresponding content in the embodiment of the method.
In another aspect, referring to fig. 5, an electronic device according to a fourth embodiment of the present invention includes a processor 10, a memory 20, and a computer program 30 stored in the memory and capable of running on the processor, where the processor 10 implements the semi-closed eye image generating method as described above when executing the computer program 30.
The electronic device may be a VR device, a computer, a mobile phone, or the like. The processor 10 may in some embodiments be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chip for executing program code or processing data stored in the memory 20, etc.
The memory 20 includes at least one type of readable storage medium including flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 20 may in some embodiments be an internal storage unit of the electronic device, such as a hard disk of the electronic device. The memory 20 may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk provided on the electronic device, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. Further, the memory 20 may also include both internal storage units and external storage devices of the electronic device. The memory 20 may be used not only for storing application software installed in an electronic device and various types of data, but also for temporarily storing data that has been output or is to be output.
Optionally, the electronic device may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), a network interface, a communication bus, etc., and an optional user interface may further comprise a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the electronic device and for displaying a visual user interface. The network interface may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), and is typically used to establish a communication connection between the device and other electronic devices. The communication bus is used to enable connected communication between these components.
It should be noted that the structure shown in fig. 5 does not constitute a limitation of the electronic device, and in other embodiments the electronic device may comprise fewer or more components than shown, or may combine certain components, or may have a different arrangement of components.
The present invention also proposes a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a semi-closed eye image generation method as described above.
Those of skill in the art will appreciate that the logic and/or steps represented in the flow diagrams or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus (e.g., a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus). For the purposes of this description, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (7)
1. A method for generating a semi-closed eye image, comprising,
Scaling an eyelid template image to match the eyelid template image with the standard open eye image size, wherein the eyelid template image is an image of an eyelid area in a semi-closed eye state;
Obtaining coordinates of a plurality of eyelid key points in the scaled eyelid template image and standard vectors of the eyelid key points, wherein the standard vectors are normal vectors of the eyelid key points and vectors in opposite directions of the normal vectors on an eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by the eyelid key points;
Randomly moving each eyelid key point, and acquiring new coordinates of each eyelid key point after random movement, wherein the movement of each eyelid key point meets a first constraint condition, the first constraint condition is that an included angle between the movement direction of each eyelid key point and the standard vector in which the eyelid key point is positioned is smaller than a preset angle, and the movement distance of each eyelid key point is smaller than a preset distance;
Determining a corresponding curve equation according to the new coordinates of the eyelid key points to obtain a target eyelid contour curve equation;
adjusting the scaled eyelid template image according to the target eyelid contour curve equation to obtain a new eyelid template image matched with the target eyelid contour curve equation;
After carrying out translation processing on the new eyelid template image, covering pixels of a corresponding area in the standard open-eye image to obtain a semi-closed-eye image;
The step of adjusting the scaled eyelid template image according to the target eyelid contour curve equation comprises:
Erasing pixels below the target eyelid contour curve equation in the scaled eyelid template image, and performing pixel complementation on blank areas above the target eyelid contour curve equation in the scaled eyelid template image;
The step of performing translation processing on the new eyelid template image comprises the following steps:
Translating the new eyelid template image on an x-axis so that the midpoint of an eyelid contour curve equation in the new eyelid template image and the pupil center of the standard open eye image have the same coordinates on the x-axis;
The step of performing translation processing on the new eyelid template image comprises the following steps:
Translating the new eyelid template image on a y-axis, wherein the translation amount of the new eyelid template image on the y-axis meets a second constraint condition, and the second constraint condition is as follows:
Wherein t y is the translation of the new eyelid template image on the y axis, y p is the coordinate of the pupil center on the y axis in the standard open eye image, w is the diameter of the pupil in the standard open eye image, f 2 (x) is the target upper eyelid contour curve equation, x p is the coordinate of the pupil center on the x axis in the standard open eye image, gamma is a constant, and the value range is (0.1).
2. The method of generating a semi-closed eye image according to claim 1, wherein the step of obtaining a standard vector for each of the eyelid keypoints comprises:
Determining a corresponding curve equation according to the coordinates of the eyelid key points to obtain an eyelid contour curve equation;
and calculating normal vectors of the eyelid contour curve equation at the eyelid key points, and calculating vectors with opposite normal vector directions to obtain standard vectors of the eyelid key points.
3. The method of generating a semi-closed eye image according to claim 1, wherein the step of scaling the eyelid template image comprises:
Calculating the ratio of the size of the eyelid in the eyelid template image to the size of the eyelid in the standard open eye image;
And scaling the eyelid template image according to the ratio.
4. A half-closed eye image generating apparatus, comprising:
The scaling module is used for scaling the eyelid template image to be matched with the standard open eye image in size, and the eyelid template image is an image of an eyelid area in a semi-closed eye state;
the system comprises an acquisition module, a scaling module and a scaling module, wherein the acquisition module is used for acquiring coordinates of a plurality of eyelid key points in the scaled eyelid template image and standard vectors of the eyelid key points, wherein the standard vectors are normal vectors of the eyelid key points and vectors in opposite directions of the normal vectors on an eyelid contour curve equation, and the eyelid contour curve equation is a curve equation formed by the eyelid key points;
The moving and acquiring module is used for randomly moving each eyelid key point and acquiring new coordinates of each eyelid key point after random movement, wherein the movement of the eyelid key point meets a first constraint condition, the first constraint condition is that an included angle between the moving direction of the eyelid key point and the standard vector where the eyelid key point is positioned is smaller than a preset angle, and the moving distance of the eyelid key point is smaller than a preset distance;
the determining module is used for determining a corresponding curve equation according to the new coordinates of each eyelid key point to obtain a target eyelid contour curve equation;
The adjusting module is used for adjusting the scaled eyelid template image according to the target eyelid contour curve equation to obtain a new eyelid template image matched with the target eyelid contour curve equation;
the coverage module is used for covering pixels of a corresponding area in the standard open-eye image after the new eyelid template image is subjected to translation processing so as to obtain a semi-closed-eye image;
The step of adjusting the scaled eyelid template image according to the target eyelid contour curve equation comprises:
Erasing pixels below the target eyelid contour curve equation in the scaled eyelid template image, and performing pixel complementation on blank areas above the target eyelid contour curve equation in the scaled eyelid template image;
The step of performing translation processing on the new eyelid template image comprises the following steps:
Translating the new eyelid template image on an x-axis so that the midpoint of an eyelid contour curve equation in the new eyelid template image and the pupil center of the standard open eye image have the same coordinates on the x-axis;
The step of performing translation processing on the new eyelid template image comprises the following steps:
Translating the new eyelid template image on a y-axis, wherein the translation amount of the new eyelid template image on the y-axis meets a second constraint condition, and the second constraint condition is as follows:
Wherein t y is the translation of the new eyelid template image on the y axis, y p is the coordinate of the pupil center on the y axis in the standard open eye image, w is the diameter of the pupil in the standard open eye image, f 2 (x) is the target upper eyelid contour curve equation, x p is the coordinate of the pupil center on the x axis in the standard open eye image, gamma is a constant, and the value range is (0.1).
5. The semi-closed eye image generation apparatus of claim 4, wherein the acquisition module is configured to:
Determining a corresponding curve equation according to the coordinates of the eyelid key points to obtain an eyelid contour curve equation;
and calculating normal vectors of the eyelid contour curve equation at the eyelid key points, and calculating vectors with opposite normal vector directions to obtain standard vectors of the eyelid key points.
6. A readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 3.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 3 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452365.2A CN114821754B (en) | 2022-04-27 | 2022-04-27 | Semi-closed eye image generation method and device, readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210452365.2A CN114821754B (en) | 2022-04-27 | 2022-04-27 | Semi-closed eye image generation method and device, readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114821754A CN114821754A (en) | 2022-07-29 |
| CN114821754B true CN114821754B (en) | 2024-08-09 |
Family
ID=82510346
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210452365.2A Active CN114821754B (en) | 2022-04-27 | 2022-04-27 | Semi-closed eye image generation method and device, readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114821754B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116452621B (en) * | 2023-03-10 | 2023-12-15 | 广州市易鸿智能装备有限公司 | Ideal contour generating algorithm, device and storage medium based on reinforcement learning |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112995534A (en) * | 2021-02-05 | 2021-06-18 | 北京字跳网络技术有限公司 | Video generation method, device, equipment and readable storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6892231B2 (en) * | 2016-07-29 | 2021-06-23 | アルパイン株式会社 | Eyelid opening / closing detection device and eyelid opening / closing detection method |
| CN112384127B (en) * | 2018-07-27 | 2023-11-10 | 高雄医学大学 | Eyelid ptosis detection method and system |
| CN110956079A (en) * | 2019-10-12 | 2020-04-03 | 深圳壹账通智能科技有限公司 | Face recognition model construction method and device, computer equipment and storage medium |
| CN113705460B (en) * | 2021-08-30 | 2024-03-15 | 平安科技（深圳）有限公司 | Method, device, equipment and storage medium for detecting open and closed eyes of face in image |
-
2022
- 2022-04-27 CN CN202210452365.2A patent/CN114821754B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112995534A (en) * | 2021-02-05 | 2021-06-18 | 北京字跳网络技术有限公司 | Video generation method, device, equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114821754A (en) | 2022-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12073542B2 (en) | Image processing method, image processing apparatus, and non-transitory storage medium | |
| CN110175558B (en) | Face key point detection method and device, computing equipment and storage medium | |
| KR102045695B1 (en) | Facial image processing method and apparatus, and storage medium | |
| US11113836B2 (en) | Object detection method, device, apparatus and computer-readable storage medium | |
| CN108229386B (en) | Method, apparatus, and medium for detecting lane line | |
| US8824808B2 (en) | Methods and apparatus for automated facial feature localization | |
| CN105474263B (en) | System and method for generating three-dimensional face model | |
| US9589333B2 (en) | Image correction apparatus for correcting distortion of an image | |
| CN105184249A (en) | Method and device for processing face image | |
| CN108550176A (en) | Image processing method, equipment and storage medium | |
| CN109325907B (en) | Image beautifying processing method, device and system | |
| CN109657583A (en) | Face's critical point detection method, apparatus, computer equipment and storage medium | |
| CN109087261B (en) | Face correction method based on unlimited acquisition scene | |
| US20190057532A1 (en) | Realistic augmentation of images and videos with graphics | |
| WO2019223147A1 (en) | Liver canceration locating method and apparatus, and storage medium | |
| Beyeler | OpenCV with Python blueprints | |
| US12124534B2 (en) | Method for generating a plurality of sets of training image data for training machine learning model | |
| US12020403B2 (en) | Semantically-aware image extrapolation | |
| CN114821754B (en) | Semi-closed eye image generation method and device, readable storage medium and electronic equipment | |
| CN111667556A (en) | Form correction method and device | |
| Smiatacz | Normalization of face illumination using basic knowledge and information extracted from a single image | |
| US11367296B2 (en) | Layout analysis | |
| CN116485759A (en) | Method, system and electronic equipment for target boundary recognition in ultrasonic images | |
| CN117152805A (en) | Image processing method and device, computing device, storage medium and program product | |
| CN112785683B (en) | Face image adjusting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |