CN106371565A - Control method based on eye movement and device applied by control method - Google Patents

Abstract

The present invention provides a control method based on eye movements and a device for its application. The control method is suitable for a device and includes the following steps. The image sequence is acquired through the imaging unit. Analyze the image sequence to obtain eye image information of the user's eye area in each image of the image sequence. Based on the eye image information, the movement trajectory of the user's gaze when looking at the view pattern is detected. Determine whether the movement trajectory of the gaze complies with the preset trajectory relative to the view pattern to generate a judgment result, and perform corresponding operations on the device based on the judgment result. The invention can use the movement trajectory of the gaze as a signal trigger, and can be widely used in many fields such as security systems and eye-controlled computers.

Description

Eye movement-based control method and device for its application

technical field

The present invention relates to a control method and its application device, and in particular to an eye movement-based control method and its application device.

Background technique

In terms of current technology, eye tracking technology can be mainly divided into two types: invasive and non-invasive. Intrusive eye-tracking techniques mainly place search coils in the eyes or use electrooculograms to sense where the eyes are looking. The non-invasive eye tracking technology is mainly based on vision based, and can be divided into various implementation methods such as free-head or head-mount.

With the development of technology, eye tracking technology has been widely used in various fields such as neuroscience and computer science, and is often found in security systems (such as eye locks) or eye-controlled computers. The human eye tracking technology can track the movement of the eyeball to obtain the location of the gaze of the eyeball, and can control the security system or computer device to realize the eye control function, or trigger the security system or computer device by gazing. Control of devices such as eye-controlled computers.

Contents of the invention

The present invention provides a control method based on eye movements and an application device thereof, which allow the user to gaze and move the point of gaze on a preset view pattern, and detect the moving track of the user's line of sight. Determine whether to allow the device to perform the corresponding operation. In this way, the device can be operated through the moving track of the user's eyesight.

The present invention proposes a control method based on eye movements. This control method includes the following steps. The image sequence is acquired by the imaging unit. The image sequence is analyzed to obtain eye image information of the user's eye area in each image of the image sequence. Based on the eye image information, the line of sight movement track of the user looking at the view pattern is detected. Judging whether the line of sight moving track conforms to the preset track relative to the view pattern, so as to generate a judging result. And, according to the judgment result, perform a corresponding operation on a device.

In an embodiment of the present invention, the view pattern includes a plurality of identification patterns. The preset trajectory is a geometric connection line formed sequentially corresponding to at least one of the identification patterns.

In an embodiment of the present invention, the above-mentioned step of detecting the movement trajectory of the user's gaze while gazing at the view pattern based on the eye image information includes: analyzing the eye image information to determine whether the user is gazing at the preset view pattern position, or gaze at a preset position for more than a preset time, and when the user gazes at a preset position in the view pattern, or gazes at a preset position for more than a preset time, detect the line of sight movement track of the user staring at the view pattern.

In an embodiment of the present invention, the above-mentioned step of detecting the eye movement track of the user looking at the view pattern based on the eye image information includes: detecting an eye target in the eye area based on the eye image information, Executing an eye tracking action based on the eye target to sequentially obtain a plurality of first positions of the eye target in the image sequence, and transform the first position into a plurality of second positions according to the coordinate system corresponding to the view pattern, In order to obtain the eye movement track of the user looking at the view pattern.

In an embodiment of the present invention, the above-mentioned eye object includes a pupil and a first light reflection point, and an eye tracking operation is performed according to the eye object, so as to sequentially obtain the first position of the eye object corresponding to the image sequence The steps include: locating the pupil and the first reflective point of the image sequence to perform an eye tracking action, and sequentially obtaining the first position of the eye target corresponding to the image sequence. Moreover, the step of mapping and transforming the first position into the plurality of second positions according to the coordinate system corresponding to the view pattern, so as to obtain the moving track of the user's line of sight while gazing at the view pattern includes: calculating The distance parameter, calculating the angle parameter according to the vector of the first reflective point relative to the pupil, and analyzing the distribution relationship between the distance parameter and the angle parameter, so as to convert the first position into the second position in the coordinate system corresponding to the view pattern, and The second position is used as a line of sight movement track of the user looking at the view pattern.

In an embodiment of the present invention, the above-mentioned eye target further includes a second reflective point, and according to the coordinate system corresponding to the view pattern, the first position is converted into the plurality of second positions by mapping, so as to obtain the The step of moving the line of sight of the view pattern includes: calculating the area parameter according to the pupil, the first reflective point and the second reflective point, and calculating the area parameter according to the first connecting line between the first reflective point and the pupil and the second connecting line between the second reflective point and the pupil. At least one of the connecting lines, calculating the angle parameter, analyzing the distribution relationship between the area parameter and the angle parameter, to convert the first position into the plurality of second positions in the coordinate system corresponding to the view pattern, and converting the second position As the user's gaze movement trajectory while gazing at the view pattern.

In an embodiment of the present invention, the above-mentioned step of executing the eye tracking action based on the eye target to sequentially obtain the first position of the eye target corresponding to the image sequence further includes: after obtaining the first position After one of them, adjust the eye area to detect the eye target again, and perform the eye tracking action accordingly.

In an embodiment of the present invention, the above-mentioned step of performing corresponding operations on the device according to the judgment result includes: when the line of sight movement track conforms to the preset track, the device releases the locked state of the lock, and when the line of sight movement track does not meet the preset During the trajectory, the device maintains the locked state of the lock.

The present invention proposes a device for controlling based on eye movement, and the device includes an image capturing unit, a storage unit and a processing unit. The imaging unit is used for acquiring image sequences. The storage unit is used to record a plurality of program modules, and the storage unit also includes a database. The processing unit is coupled to the imaging unit and the storage unit to access and execute the modules recorded in the storage unit. The program modules include an image analysis module, a line of sight detection module, a judgment module and a control module. The image analysis module analyzes the image sequence, so as to obtain eye image information of the user's eye area in each image of the image sequence. The sight detection module is based on the eye image information to detect the movement track of the user's gaze when gazing at the view pattern. The judging module judges whether the line of sight moving track conforms to a preset track relative to the view pattern, so as to generate a judging result. The control module executes corresponding operations through the device according to the judgment result.

In an embodiment of the present invention, the above-mentioned view pattern includes a plurality of identification patterns, and the preset trajectory is a geometric connection corresponding to at least one of the identification patterns formed sequentially.

In an embodiment of the present invention, the sight line detection module analyzes the eye image information to determine whether the user gazes at a preset position in the view pattern or gazes at a preset position for more than a preset time. And, when the user gazes at a preset position in the view pattern, or gazes at the preset position for more than a preset time, the sight line detection module detects the user's gaze movement track while gazing at the view pattern.

In an embodiment of the present invention, the sight line detection module includes an eye detection module, an eye tracking module, and a mapping conversion module. Wherein, the eye detection module detects the eye target in the eye area based on the eye image information. The eye tracking module executes an eye tracking action according to the eye target to sequentially obtain a plurality of first positions of the eye target in the image sequence. The mapping transformation module maps and transforms the first position into a plurality of second positions according to the coordinate system corresponding to the view pattern, so as to obtain the moving track of the user's sight line when gazing at the view pattern.

In an embodiment of the present invention, the eye target includes the pupil and the first reflection point, and the eye tracking module locates the pupil and the first reflection point in the image sequence to perform the eye tracking action, and sequentially obtains the eye A sub-object corresponds to said first position of the sequence of images. Moreover, the above-mentioned mapping conversion module calculates the distance parameter according to the pupil and the first reflective point, and calculates the angle parameter according to the vector of the first reflective point relative to the pupil, and analyzes the distribution relationship between the distance parameter and the angle parameter to convert the first position are the plurality of second positions in the coordinate system corresponding to the view pattern, and use the second positions as the line of sight movement track of the user looking at the view pattern.

In an embodiment of the present invention, the eye target further includes a second reflective point, and the mapping conversion module calculates the area parameter according to the pupil, the first reflective point, and the second reflective point, and calculates the area parameter according to the relationship between the first reflective point and the pupil At least one of the first connection line and the second connection line between the second reflective point and the pupil, calculate the angle parameter, and analyze the distribution relationship between the area parameter and the angle parameter, so as to convert the first position into a coordinate system corresponding to the view pattern. The plurality of second positions, and using the second positions as a line of sight movement track of the user looking at the view pattern.

In an embodiment of the present invention, the above-mentioned eye tracking module further adjusts the eye area after obtaining one of the first positions, so that the eye object can be detected again by the eye detection module, and the eye tracking module executes accordingly The eye-tracking action.

In an embodiment of the present invention, when the line of sight movement track conforms to the preset track, the above-mentioned control module control device releases the locked state of the lock, and when the line of sight movement track does not conform to the preset track, the control module control device maintains the lock of the lock state.

Based on the above, the eye movement-based control method and its applied device proposed by the embodiments of the present invention can detect the movement trajectory of the user's gaze while gazing at the view pattern, and determine that the movement trajectory of the sight line conforms to the preset relative to the view pattern When track, the corresponding operation will be performed by the device. In this way, the embodiment of the present invention can use the line of sight movement as a signal trigger, and can be widely used in multiple fields such as security systems and eye-controlled computers.

In order to make the above-mentioned features and advantages of the present application more comprehensible, the following specific embodiments are described in detail with accompanying drawings.

Description of drawings

FIG. 1 is a schematic functional block diagram of an eye movement-based control device according to an embodiment of the present invention;

2A and 2B are schematic diagrams of an eye movement-based control method according to an embodiment of the present invention;

FIG. 3 is a flow chart of the steps of a control method based on eye movements according to an embodiment of the present invention;

FIG. 4A is a flow chart of steps of an eye movement-based control method according to another embodiment of the present invention;

FIG. 4B is a schematic functional block diagram of a line of sight detection module according to an embodiment of the present invention;

FIG. 4C is a schematic diagram of an eye region according to an embodiment of the present invention;

FIG. 5A is a flow chart of the steps of eye tracking according to an embodiment of the present invention;

FIG. 5B is a schematic diagram of eye tracking according to an embodiment of the present invention;

6A to 6G are schematic diagrams of coordinate system mapping conversion according to an embodiment of the present invention;

Fig. 7 is a schematic diagram of an access control system controlled based on eye movements according to an embodiment of the present invention;

8A and 8B are schematic diagrams of a hand-held eye-control eye-connection device controlled based on eye movements according to an embodiment of the present invention;

Fig. 9 is a schematic diagram of a control method based on eye movements according to another embodiment of the present invention.

Explanation of reference signs:

100: device;

110, 710, 810, 910: imaging unit;

120, 720, 920: processing unit;

130: storage unit;

131: image analysis module;

132: Line of sight detection module;

1321: eye detection module;

1322: eye tracking module;

1323: mapping conversion module;

133: judging module;

134: control module;

135: database;

200: view pattern;

210: identify patterns;

220: geometric connection;

230: preset trajectory;

410, 510, 610: eye targets;

512: pupil center point;

514, 614, 616: reflective points;

602: vector;

612: pupil;

630: coordinate point;

640: sector;

650: rectangle;

660: coordinate system;

700: access control system;

730: lock;

740: Prompt unit;

750: door body;

760: cover body;

800: hand-held eye control device;

820: display unit;

830: housing;

840: mirror;

850: light source;

860: safe;

900: eye control device;

ER: eye region;

H: horizontal line;

IS: image sequence;

L: distance;

θ: angle;

1~18: area;

S310～S350, S410～S480, S510～S540: steps.

detailed description

At present, eye control technology mostly detects the position of the user's line of sight, and can only control the security system or the eye control computer according to the position of the user's gaze on the screen. The eye movement-based control method and the device applied thereto in the embodiments of the present invention can use the line of sight movement track as a signal trigger, so as to realize the eye control technology in the form of a track. Thereby, the embodiment of the present invention can provide a more secure password input method, and can also realize more convenient and more intuitive eye movement control. The following is a detailed description of the eye movement-based control method proposed by the embodiments of the present invention and the device applying it.

FIG. 1 is a functional block diagram of an eye movement-based control device according to an embodiment of the present invention. The device 100 for controlling based on eye movements includes an imaging unit 110 , a processing unit 120 and a storage unit 130 . Here, the device 100 can be, for example, a safe, an access control system, or other types of security systems/device that need to verify the user's qualification (qualification) to determine whether to provide the user with a specific authority, or can be an eye control function. For electronic devices such as computers, the present invention does not limit the specific type of the device 100 .

In this embodiment, the imaging unit 110 can be used to obtain an image sequence IS along a specific direction (depending on the configuration position/angle of the imaging unit 110), that is, the imaging unit 110 continuously acquires multiple images and provides them to processing unit 120 .

The processing unit 120 is coupled to the image capturing unit 110 and is, for example, a central processing unit (central processing unit, referred to as CPU), a graphics processing unit (graphics processing unit, referred to as GPU), or other programmable microprocessors (microprocessor), etc. device.

The storage unit 130 is, for example, any type of fixed or removable random access memory (random access memory, referred to as RAM), read-only memory (read-only memory, referred to as ROM), flash memory (flash memory) or similar components or the above-mentioned A combination of components. The storage unit 130 includes a database 135 and records a plurality of program modules. The processing unit 120 is coupled to the storage unit 130 to access and execute the above program modules. These program modules include an image analysis module 131 , a line of sight detection module 132 , a judging module 133 , and a control module 134 . Wherein, the image analysis module 131 performs image processing and analysis on the image sequence IS acquired by the image capturing unit 110, so as to obtain eye image information of the user's eye area in each image of the image sequence IS. In this way, the sight line detection module 132 can detect the user's eye movement according to the obtained eye image information, and then the judgment module 133 can judge whether the movement track of the user's line of sight when gazing at the view pattern conforms to a preset track. Here, the "view pattern", the "preset track" and the corresponding relationship between the above two are described by taking the example of FIG. 2A and FIG. 2B. As shown in FIG. 2A , the view pattern 200 includes a plurality of identification patterns 210 . The identification pattern 210 can be arranged in the form of an M×N matrix. In this embodiment, both M and N are 4, but it is not limited thereto. It should be noted that, the aforementioned preset trajectory may be a geometric connection corresponding to at least one of the identification patterns 210 formed in sequence. For example, the geometric connection 220 shown in FIG. 2A may be composed of a plurality of identification patterns 210 , while FIG. 2B shows a preset trajectory 230 stored in the database 135 . It can be seen that the above-mentioned geometric connection line 220 may correspond to the preset trajectory 230 .

In this way, the control module 134 determines whether to execute the corresponding operation on the device 100 according to the judgment result of whether the above-mentioned line of sight movement track conforms to the preset track. For example, when the device 100 is a security device, and the preset trajectory is set to decode the lock in the security device, the control module 134 can switch the lock to a locked state or unlocked according to whether the line of sight movement trajectory conforms to the preset trajectory state.

The control method based on eye movement proposed by the embodiment of the present invention will be described below. FIG. 3 is a flow chart of steps of an eye movement-based control method according to an embodiment of the present invention. The method of this embodiment is applicable to the above-mentioned electronic device 100. Here, the various components of the device 100 in FIG. 1 are combined, and the concept of control based on eye movements in this embodiment of the present invention is illustrated with the flow chart of the steps in FIG. 3 .

In step S310 , the device 100 acquires an image sequence IS through the imaging unit 110 . In step S320 , the image analysis module 131 analyzes the image sequence IS, so as to obtain eye image information of the user's eye area in each image of the image sequence IS. And in step S330 , the line of sight detection module 132 detects the moving track of the line of sight of the user looking at the view pattern based on the eye image information.

It should be noted that, in some embodiments, the line of sight detection module 132 can first analyze the eye image information to determine whether the user is gazing at the preset position in the view pattern, or determine whether the user is gazing at the above-mentioned preset position. The location exceeds the preset time. When the user gazes at a preset position in the view pattern, or when the user gazes at a specific preset position for more than a preset time, the gaze detection module 132 will correspondingly start to detect the movement track of the user's gaze while gazing at the view pattern. The above-mentioned way for the line-of-sight detection module 132 to determine when to start detecting the movement track of the user’s line-of-sight is only an exemplary description. The person who applies this embodiment can also provide the user with an external trigger method, such as pressing a physical button, or accessing the operation page Click to select and enter the detection line of sight moving track mode, etc., to trigger the line of sight detection module 132 to detect the line of sight moving track, and the present invention is not limited thereto.

Please continue the step flow in FIG. 3. In step S340, the judging module 133 judges whether the line of sight moving trajectory conforms to the preset trajectory relative to the view pattern to generate a judging result, and in step S350, the control module 134 can use the judgment result to The corresponding operations are performed on the device 100 . In detail, please refer to the examples of FIG. 2A and FIG. 2B again. When the line of sight detection module 132 detects that the user is gazing at the line of sight movement track of the view pattern, and the line of sight movement track is shown as the geometric connection 220 in FIG. The trajectories are compared to determine whether the above-mentioned gaze movement trajectories conform to the preset trajectories recorded in the database 135 . In this way, the control module 134 can perform different operations on the device 100 according to the judging result whether the line of sight movement track conforms to the preset track 230 in FIG. 2B .

In addition, in the device 100 of this embodiment, a reminder unit may be optionally provided. The prompting unit can be used to prompt whether the current device 100 follows the preset trajectory according to the user's line of sight movement, so as to perform corresponding operations. Here, the prompting unit can use text display, light signal display, voice prompt or any other feasible prompting methods to enable the user to recognize the current state of the device 100 , which is not limited in the present invention.

Next, the detailed flow of the control based on eye movement is further described by using the embodiments of FIG. 4A to FIG. 4C . Among them, FIG. 4A is a flow chart of steps of a control method based on eye movements according to another embodiment of the present invention, FIG. 4B is a detailed block diagram of a line of sight detection module controlled based on eye movements according to an embodiment of the present invention, and FIG. 4C It is a schematic diagram of the eye region according to an embodiment of the present invention. Please refer to the step flow in FIG. 4A , and describe with each element in the device 100 in FIG. 1 and the line-of-sight detection module 132 in FIG. 4B . In this embodiment, the device 100 is a security device, and the eye movement-based control method can be used to control a lock in the device 100 . It should be noted that the control method based on eye movements can also be applied to any type of device, and the present invention is not limited thereto.

In step S410, the device 100 acquires the image sequence IS through the imaging unit 110, and in step S420, the image analysis module 131 analyzes the image sequence IS, so as to obtain the user's eye Eye image information for the region. The detailed description here is similar to the embodiment in FIG. 3 , so please refer to the foregoing.

As shown in FIG. 4B , the gaze detection module 132 of this embodiment includes an eye detection module 1321 , an eye tracking module 1322 and a mapping conversion module 1323 . Thereby, in step S430 , the eye detection module 1321 detects the eye target in the eye area based on the eye image information. In step S440 , the eye tracking module 1322 executes an eye tracking operation according to the eye target, so as to sequentially obtain a plurality of first positions of the eye target in the image sequence. And in step S450 , the mapping conversion module 1323 maps and converts the plurality of first positions into a plurality of second positions according to the coordinate system corresponding to the view pattern, so as to obtain the moving track of the user's gaze when gazing at the view pattern. In other words, this embodiment can use the above-mentioned eye detection module 1321, eye tracking module 1322, and mapping conversion module 1323 to analyze the continuous motion corresponding to the user's eye target in the image sequence, and use the image sequence and view The conversion between the coordinate systems corresponding to the patterns respectively is used to obtain the line of sight movement track of the user looking at the view pattern.

After obtaining the user's line of sight movement track, the judging module 133 can compare the line of sight movement track with the preset track of the view pattern, and the control module 134 can perform corresponding operations through the device 100 according to the judgment result, wherein the above-mentioned preset track It can be set as the default unlock password of the lock of the device 100 . Therefore, in step S460, the judging module 133 judges whether the moving track of the sight line conforms to the preset track relative to the view pattern. When the lock of the device 100 is in the locked state, if step S460 judges yes, the control module 134 will release the lock state of the lock (that is, the lock is switched to the unlocked state); otherwise, if the judgment is no, the control module 134 The locked state of the lock will be maintained, and a warning sound or text may be issued through the prompt unit to remind the user. In addition, from another point of view, if the lock is already in the unlocked state, no matter whether the determination result of step S460 is yes, the lock will remain in the unlocked state.

It is worth mentioning that, in an embodiment, the user can also make a set of continuous eye movements when the lock is unlocked, so as to use the line of sight movement trajectory corresponding to the continuous eye movements, So as to define the preset unlocking password by yourself. Certainly, the preset unlocking password may also be predefined by a designer and stored in the database 135, which is not limited in the present invention.

It is also worth mentioning that, in one embodiment, the device 100 may further include an identification module, which is used to first identify the biometric information of the user's eye target after the eye target is detected (step S430), so as to Confirm the identity of the user. The aforementioned biometric information is, for example, iris or retina information. If the identity of the user is correct, the identification module allows the device 100 to further execute subsequent steps S440-S480. If not, the identification module stops or refuses the device 100 to perform subsequent eye movement identification steps.

The above embodiment will be specifically described below with the schematic diagram of FIG. 4C . In this embodiment, after the eye detection module 1321 searches for the eye target 410 in the eye region ER, the eye detection module 1321 can also perform image processing on the eye image information to obtain Image information of the boundary of the target 410 . In this way, the eye tracking module 1322 can track the eye target 410 according to the image information, so as to obtain the moving track of the user's gaze. For example, by adjusting the gain value (gain) and the offset value (offset), the eye detection module 1321 can adjust the contrast of the eye image information and obtain an enhanced image. Next, the eye detection module 1321 may sequentially perform denoising, edge sharpening, binarization, and edge sharpening again on the enhanced image to obtain the image of the eye object 410 as shown in FIG. 4C .

After obtaining the above-mentioned eye object 410 , the eye tracking module 1322 performs an eye tracking operation according to the eye object 410 , so as to obtain the precise position of the user's eye object 410 in each image sequence. In one embodiment, the eye tracking module 1322 can obtain the precise positioning of the eye object 410 through the relative positional relationship between the pupil and the reflective point in the eye object 410 . The above-mentioned reflective point is, for example, a purkinje image, which can be formed by fixing a light source in the form of a light beam and reflecting it after illuminating human eyes. According to the position of the fixed light source, the reflective point will correspondingly exist at the specific position of the human eye in the image, so it can be used as a reference feature for locating the eye object 410 . Moreover, using the reflective point as a positioning reference feature can also avoid unnecessary noise interference due to insufficient light source at the reference point when the eye tracking module 1322 tracks the eye target. It should be noted that, in the following embodiments, the position of the pupil is represented by the position of the central point of the pupil, and is different according to the number of reflective points used as positioning reference features (for example, in the example of FIG. 5B , the eye target 510 includes one reflective point 514, while in the example of FIG. 6E the eye target 610 includes two reflective points 614, 616), eye tracking will be implemented in different ways.

The steps of eye tracking are shown in FIG. 5 and FIG. 5A. In step S510, the eye tracking module 1322 calculates the black-white contrast change in the eye area to obtain the pupil position in the image sequence. In step S520, the eye tracking module 1322 obtains the reference position of the reflective point. Moreover, in step S530 , the eye tracking module 1322 adjusts and obtains the first position of the eye target in the image sequence according to the relative relationship between the pupil position and the reference position of the reflective point.

In detail, FIG. 5B shows an example in which the eye object 510 includes a pupil (corresponding to the pupil center point 512 ) and a light reflection point 514 , and the eye object 510 corresponds to one of the images in the image sequence. The eye detection module 1321 can use the statistical information of the pupil to obtain the eye target 510. The above statistical information is, for example, the type of statistical data related to the area, length, width, aspect ratio, shape, black and white contrast of the eyeball and/or pupil at least one or a combination thereof. After the eye detection module 1321 obtains the eye object 510, the eye tracking module 1322 can calculate the black-white contrast change in the eye region. Using the edge of the pupil and eyeball in the human eye (referring to the iris in the structure of the human eye, which is the non-white area of the eye such as black, blue or brown) corresponds to the extreme value of the contrast between black and white (generally speaking, relative to the adjacent eyeball, the pupil is darker (or darker) image), the eye tracking module 1322 can obtain the approximate position of the pupil center point 512 . As for the reflective point 514, the eye tracking module 1322 can also obtain the reference position of the reflective point 514 in a manner similar to the above, and also consider the condition of the white contrast area located closest to the center of the pupil as a criterion for judging whether it is a reflective point 514 basis. Afterwards, the eye tracking module 1322 performs edge detection on the pupil, and uses the reference position of the reflective point 514 as a reference point for edge detection to obtain an accurate pupil edge to estimate the position of the pupil center point 512, thereby correcting the position of the pupil center point 512 To achieve precise pupil positioning. The above-mentioned edge detection method can detect edges by utilizing sharp changes of pixels from white (corresponding to the eyeball) to black (corresponding to the pupil) or from black to white. In addition, the eye tracking module 1322 can finely adjust the reference position of the reflective point 512 to provide a more accurate reference position as a reference point for pupil positioning. Wherein, the eye tracking module 1322 can use the center of the reflective point as a reference point for its edge detection, and adjust the reference position of the reflective point according to a process similar to that of pupil edge detection. In this way, by repeatedly and continuously executing the above process, the eye tracking module 1322 can sequentially obtain the positions of the eye objects in the image sequence. These positions correspond to the moving track of the user's gaze when gazing at the view pattern.

It is worth mentioning that the flowchart in FIG. 5A also includes step S540. In step S540, after the eye tracking module 1322 acquires the first position of the eye target 510, the range of the eye area will be further adjusted so that the eye tracking module 1322 can accurately and efficiently execute the next Eye tracking action. For example, in one embodiment, the eye tracking module 1322 can adjust the range of the eye area according to the maximum range that the human eye can move. In addition, in some embodiments, the eye tracking module 1322 can also adaptively filter out the noise in the eye target before performing the eye tracking action, so as to improve the accuracy of locating and tracking the eye target 510 . The above methods for adjusting the eye area and filtering noise can be properly designed according to requirements, and the present invention is not limited thereto.

The specific embodiment of obtaining the line of sight movement track of the user gazing at the view pattern in step S450 will be described below with reference to the schematic diagrams of FIG. 5B , and FIG. 6A to FIG. 6D . In this embodiment, the eye tracking module 1322 can obtain the positions of the pupil center point 512 and the reflective point 514 by using the implementation of steps S510 and S520 in the foregoing embodiments. Then, the mapping conversion module 1323 can use the distance L between the pupil center point 512 and the reflective point 514, and a reference line (ie, the horizontal axis in FIG. The angle θ defined by the vector 602 of the center point 512, through the relationship between the above-mentioned distance L and the angle θ, the user's pupil displacement position in the image sequence (that is, corresponding to the above-mentioned first position) is transferred to The coordinate system of the view pattern is used to obtain the corresponding line of sight movement track. In the embodiment of using the above parameters of the distance L and the angle θ to perform coordinate transformation, the grouping correspondence method and the interpolation correspondence method are provided below as illustrations.

Firstly, the grouping correspondence method is described with the embodiments of FIG. 6B and FIG. 6C . In this embodiment, the grouping correspondence method may include a training phase and an application phase, wherein, the mapping conversion module 1323 obtains the coefficients required for coordinate conversion through the training phase, and can convert the current eye object accordingly in the application phase into the coordinate system of the view drawing. In detail, during the training phase, the mapping transformation module 1323 can display a group of areas from areas 1 to 16 in FIG. 6B each time for the user to watch (in FIG. The user’s viewing situation), so that the mapping conversion module 1323 can locate and obtain the distance and angle of the user’s pupil center and reflective point corresponding to each group of regions, so that the distance-angle distribution of each group of regions in FIG. 6B is shown in The distance-angle diagram of Figure 6C. In this way, during the application stage of the grouping correspondence method, the mapping transformation module 1323 can analyze the distance L and the angle corresponding to the pupil and the reflection point in the eye object through the eye object currently obtained by the imaging unit 110 θ, and obtain coordinate points in the distance-angle diagram, such as coordinate point 630 in FIG. 6C . Next, the mapping transformation module 1323 can use the minimum objective function to find the area closest to the coordinate point 630, so as to locate the current eye object to the coordinate system of the view pattern, thereby completing the coordinate transformation. The formula (1) of the above minimum objective function is as follows:

Min Obj.＝W ₁ |Dis_t-Dis_i|+W ₂ |Angle_t-Angle_i|, (i＝1～16)...(1)

Wherein, Dis_i and Angle_i are the distance value and angle value of the target, and Dis_t and Angle_t are the distance value and angle value corresponding to the current eye target. The above-mentioned distance value and angle value can be obtained by calculating the x-axis and y-axis coordinates of the coordinate points, and W ₁ and W ₂ are distribution weights. It can be seen that the number of regions in FIG. 6B will determine the degree of accuracy when the mapping conversion module 1323 converts coordinates using the group correspondence method. Therefore, when the number of regions included in the grouping correspondence method of this embodiment is larger, the mapping transformation module 1323 can correspondingly obtain more accurate transformation coordinates.

On the other hand, the embodiment of FIG. 6D is used to illustrate the interpolation correspondence method. In this embodiment, the mapping conversion module 1323 can convert the sector 640 drawn by the coordinate points in the distance-angle diagram into a more regular rectangle 650 through an un-warping operation, and convert the shape of the rectangle 650 into a regular rectangle 650. The location corresponds to the coordinate system 660 in which the viewport resides. This embodiment can also include a training phase and an application phase, and the implementation of the training phase can also be similar to the foregoing embodiment. The difference is that the mapping conversion module 1323 of this embodiment also uses the above-mentioned twisting operation to obtain a normalized distribution map, and can perform moving calibration on the normalized distribution map by affine transform. The formulas (2) and (3) for obtaining the normalized distance-angle distribution diagram through the twisting operation are as follows:

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right.<span\; class="notranslate">\; ...</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccccc}{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& {\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\\ {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}& {\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; 5</span>}}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; the\; y</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\end{array}\right]<span\; class="notranslate">\; ...</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

The twisting operation converts the sampling point (X,Y)={(x ₁ ,y ₁ )...(x _n ,y _n )} to the target point (X _T ,Y _T )={( x _T1 ,y _T1 )...(x _Tn ,y _Tn )}, where X and Y represent the above distance L and angle θ, n is the number of samples, and a ₀ ~ a ₅ , b ₀ ~ b ₅ are Twist positive conversion factor. In this way, the mapping conversion module 1323 can obtain the optimal solution of the coefficients a ₀ -a ₅ and b ₀ -b ₅ through the inverse matrix operation. In this way, the mapping conversion module 1323 can use the above coefficients to perform twisting conversion on the currently unknown coordinate points.

On the other hand, the formula (4) that the mapping conversion module 1323 performs motion correction using affine technology is as follows:

$\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\\ {\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{<span\; class="notranslate">\; \&prime;</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; c</span>}& \mathrm{<span\; class="notranslate">\; d</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right]<span\; class="notranslate">\; +</span>\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; e</span>}\\ \mathrm{<span\; class="notranslate">\; f</span>}\end{array}\right]<span\; class="notranslate">\; ...</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Among them, x', y' are new coordinates after movement correction, and a~f are affine transformation coefficients. In this way, the mapping transformation module 1323 can also be calculated by inverse matrix, or by inputting three pairs of coordinate points of any three corners of the display unit before correction and three pairs of coordinate points of any three corners of the corrected corner to obtain The above-mentioned affine transformation coefficients a to f. In this way, the mapping transformation module 1323 can correct the movement of the currently unknown coordinate points, so as to eliminate the influence caused by factors such as image scaling, translation, and rotation during the coordinate transformation. Compared with the embodiment of the grouping correspondence method proposed in FIGS. 6B and 6C , the mapping transformation module 1323 can complete the coordinate transformation with a smaller number of samples by using the interpolation correspondence method.

It should be noted that, in one embodiment, after the eye tracking module 1322 sequentially obtains a plurality of first positions of the eye target in the image sequence, the mapping conversion module 1323 converts these first positions in a continuous manner. The form is transformed into the trace of the user's gaze when gazing at the view pattern. In another embodiment, the mapping conversion module 1323 can be synchronized with the eye tracking module 1322 to obtain the first positions of the eye objects, so as to sequentially perform coordinate conversion on the above-mentioned first positions, thereby obtaining The eye movement trajectory of the reader. Those who apply this embodiment can determine the way to obtain the line of sight movement track according to their design requirements, which is not limited by the present invention.

On the other hand, the example in FIG. 6E illustrates the situation when the eye object 610 includes a pupil 612 and two light reflection points 614 , 616 . Similar to the previous embodiments, the eye tracking module 1322 can use the black and white contrast changes of the image (for example, binarize the image of the eye object 610 ) to find the pupil 612 and the reflection point 614 in the eye object 610 , 616 , and obtain their individual center points to represent the respective positions of the pupil 612 and the reflection points 614 and 616 . The mapping conversion module 1323 can calculate the area of the triangle formed by the pupil 612 and the reflective points 614 and 616 when they are not collinear, and can adaptively correct errors that may be caused by the movement of the user's head. In detail, for the area algorithm considering the above error, after calculating the area of the triangle formed by the pupil 612 and the reflective points 614, 616, the above calculated area can be further normalized by a normalization factor. For example, the above-mentioned normalization factor may be the square of the distance between the reflective points 614 and 616 divided by 2, and the mapping transformation module 1323 will divide the calculated triangle area by this normalization factor to perform error correction to get the area parameter.

On the other hand, the mapping conversion module 1323 also uses the pupil 612 and the reflection points 614 and 616 to calculate the angle parameter. In one embodiment, the mapping conversion module 1323 can obtain the first angle α1 between the above-mentioned two lines through the horizontal line H passing through the center point of the pupil 612 and the line connecting the pupil 612 and the reflection point 614 . In addition, the mapping conversion module 1323 may also use the horizontal line H and the line connecting the pupil 612 and the light reflection point 616 to obtain the second included angle α2 between the above two lines. In other words, the angle parameter can be the first included angle α1, the second included angle α2, or the difference between the second included angle α2 and the first included angle α1 (that is, α2-α1, taking the pupil 612 as center, the line connecting the pupil 612 and the reflective point 614, and the angle between the line connecting the pupil 612 and the reflective point 616).

Therefore, when the mapping transformation module 1323 is used for the first time, it can perform correction through the correction areas 1-18 or correction points as shown in FIG. 6F to obtain the correction coordinates on the angle-area plan view. Similar to the training stage of the previous embodiment, the distribution of the area and angle formed by the pupil 612 and the reflective points 614, 616 corresponding to each group of regions of the user is shown in FIG. 6G, and the above-mentioned The original coordinates are mapped to coordinates corresponding to the correction regions 1-18 to obtain coefficients required for coordinate transformation (eg, affine transformation method). In this way, after entering the application phase, the mapping conversion module 1323 can complete the coordinate conversion of the pupil 612 and the reflection points 614 and 616 through the coefficients used for coordinate conversion.

The embodiments shown in FIG. 7 to FIG. 9 are practical application examples of the above eye movement-based control method. First, take FIG. 7 as an example for description, wherein FIG. 7 is a schematic diagram of an access control system controlled based on eye movements according to an embodiment of the present invention.

The access control system 700 includes an imaging unit 710 , a processing unit 720 , a lock 730 , a prompt unit 740 and a door body 750 . The lock 730 is disposed on the door body 750 and is used to control the opening or closing of the door body 750 . In this embodiment, the prompt unit 740 is, for example, a light signal display device, and uses the light signal to prompt that the current configuration state of the lock 730 is a locked state or an unlocked state, but the embodiment of the present invention does not limit the form of the prompt unit.

In this embodiment, the imaging unit 710 is arranged on the door body 750, for example, and the imaging unit 710 is covered by a cover body 760, and only an image acquisition area is exposed in the imaging direction of the imaging unit 710, The user can align the eye area with the image acquisition area, so that the imaging unit 710 can acquire an image sequence for the user's eye area to avoid peeping by others, thereby improving the security of the access control system 700 . Here, the setting of the cover body 760 is also a designer can choose whether to add it according to his design requirements, and the present invention is not limited thereto.

Based on the architecture of the access control system 700, the processing unit 720 can read the program modules recorded in the storage unit as described in the previous embodiment according to the image sequence acquired by the imaging unit 710, and implement according to FIG. 2 to FIG. 6D The procedure of the example is used to detect the eye movement made by the user, and judge whether the moving track of the user's sight line conforms to the preset track. In this way, the processing unit 720 can determine whether to send a corresponding control signal to control the unlocking state of the lock 730 according to the judgment result, so that the user can open the door body 750 to enter the door body 750 when the moving track of his line of sight conforms to the preset track. area behind.

FIG. 8A and FIG. 8B are another example of the present invention, and show a schematic diagram of a hand-held eye-control eye-connecting device 800 controlled based on eye movements according to an embodiment of the present invention. The handheld eye control device 800 includes a processing unit, an image capturing unit 810, a display unit 820, a housing 830, a mirror 840, and a light source 850, and is connected to a security device (such as a safe 860) to authenticate the user. In this embodiment, the hand-held eye control device 800 can be connected via a wireless transmission interface (for example: Short Distance Wireless Communication (Short Distance Wireless Communication), Radio Frequency Identification (Radio Frequency Identification, referred to as RFID), Bluetooth (Bluetooth) or Wi-Fi -Fi, etc.) connected to the safe 860. Both the image capturing unit 810 and the light source 850 are disposed in the casing 830 and adjacent to the window of the hand-held eye control device 800 . The light source 850 can be turned on when the image capturing unit 810 captures the user's eye image, or turned on when the user approaches, so as to provide sufficient brightness, and the present invention is not limited thereto. The display unit 820 is disposed in the casing 830 and can be used to display a picture related to password input information. The reflector 840 can reflect the picture displayed on the display unit 820 to the window of the hand-held eye control device 800 so that the user can watch the content of the picture displayed on the display unit 820 through the window. The password input information is, for example, words such as "Please enter the password" to prompt the user to start eye movement input, or to indicate whether the user's password is output correctly, and the present invention is not limited thereto.

Therefore, when the user intends to use the safe 860 , the user can hold the hand-held eye-control eye contact device 800 against the eyes and make eye movements. Similarly, based on the architecture of the hand-held eye control device 800, the imaging unit 810 photographs the user's eyes and acquires an image sequence, and the processing unit can read the program modules recorded in the storage unit as described in the previous embodiments , and according to the steps in the embodiment of FIG. 2 to FIG. 6D to detect the eye movements made by the user, so as to obtain the line of sight movement track of the user's gaze at the view pattern displayed on the display unit 820, and move the line of sight The track is used as the input password of the safe 860, and is compared with a preset security password in the form of a track. When the above-mentioned security password matches the input password, the processing unit may generate verification success information and send it to the safe 860 , thereby controlling to open its lock. Other details of the operation of this embodiment can be similar to the above embodiment, so please refer to the foregoing.

FIG. 9 is a schematic diagram of an eye control device 900 for controlling based on eye movements according to yet another embodiment of the present invention. The eye control device 900 includes an image capturing unit 910 and a processing unit. In this embodiment, the eye tracking module provides another embodiment of tracking eye objects. In detail, the image capturing unit 910 may have a lens that can be rotatably adjusted in direction and angle, so that the lens can be adjusted to look up at the user's face. For example, the lens of the imaging unit 910 can be directed toward the user's face at an elevation angle of 45 degrees to enhance the recognition of the user's nostril image, and help to obtain the nostril image, thereby serving as a reference for defining the position of the eye target feature.

Specifically, the processing unit may use the characteristic that the nostril region is significantly darker than other regions, and use the intersection of the longest horizontal axis and the longest vertical axis of the nostril region as the center point of the nostril. Next, the eye tracking module calculates the distance between the center points of the two nostrils to determine the coordinates of the calculation point (s ₁ , t ₁ ), and then calculates the coordinates of the reference point (s ₂ , t ₂ ) based on the above distance and the coordinates of the starting point , wherein, s ₂ =s ₁ +k ₁ ×D, t ₂ =t ₁ +k ₂ ×D, k ₁ =1.6-1.8, k ₂ =1.6-1.8, and preferably k ₁ =k ₂ . The values of k ₁ and k ₂ above can be obtained according to statistical results, and the reference point coordinates (s ₂ , t ₂ ) obtained through the above relational formula can just fall close to an eye target in the face image. center point. In this way, the eye tracking module can accurately locate the user's eye target through the characteristics of the nostrils and statistical characteristics of the eyes.

Thus, based on the hardware architecture of the eye control device 900 described above, the processing unit can read the program modules recorded in the storage unit as described in the previous embodiments in a similar manner, and according to the steps of the embodiments shown in FIG. 2 to FIG. 6D process to detect the eye movement made by the user, and correspondingly obtain the moving track of the user’s line of sight while gazing at the view pattern, so as to judge whether it conforms to the preset track, so that the eye control device 900 can use the moving track of the user’s line of sight And perform corresponding operations, such as detecting the movement track of the user's gaze to unlock the computer screen, or operating the cursor of the computer window and other functions.

To sum up, the eye movement-based control method and the device applying it proposed by the embodiments of the present invention can detect the movement trajectory of the user's gaze while gazing at the view pattern, and determine that the movement trace of the sight line conforms to the direction relative to the view pattern. When the track is preset, the device will perform the corresponding operation. In this way, the embodiment of the present invention can use the line of sight movement as a signal trigger, and can be widely used in multiple fields such as security systems and eye-controlled computers.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (16)

1. A control method based on eye movements, comprising: Obtain an image sequence through an imaging unit; analyzing the sequence of images, whereby in each image of the sequence of images, ocular image information of the eye region of the user is obtained; Based on the eye image information, to detect the movement track of the user's gaze while gazing at the view pattern; judging whether the line of sight moving track conforms to a preset track relative to the view pattern to generate a judging result; and According to the judgment result, corresponding operations are performed on the device. 2 . The control method according to claim 1 , wherein the view pattern includes a plurality of identification patterns, and the preset trajectory is a geometric connection corresponding to at least one of the identification patterns formed in sequence. 3 . 3. The control method according to claim 1, wherein, based on the eye image information, the step of detecting the line of sight movement track of the user looking at the view pattern comprises: analyzing the eye image information to determine whether the user gazes at a predetermined position in the view pattern, or gazes at the predetermined position for more than a predetermined time; and When the user gazes at the preset position in the view pattern, or gazes at the preset position for more than the preset time, detecting the sight movement of the user looking at the view pattern trajectory. 4. The control method according to claim 1, characterized in that, based on the eye image information, the step of detecting the moving track of the user's gaze while gazing at the view pattern comprises: detecting eye objects within the eye region based on the eye image information; performing an eye tracking action according to the eye target to sequentially obtain a plurality of first positions of the eye target in the image sequence; and According to the coordinate system corresponding to the view pattern, the first positions are mapped and transformed into a plurality of second positions, so as to obtain the line-of-sight movement track of the user looking at the view pattern. 5. The control method according to claim 4, wherein the eye target includes a pupil and a first reflective point, and the eye tracking action is performed according to the eye target to sequentially obtain the The step of the eye target corresponding to the first positions of the image sequence comprises: locating the pupil and the first reflex point of the sequence of images to perform the eye-tracking action; and sequentially obtaining the first positions of the eye target corresponding to the image sequence, Wherein, according to the coordinate system corresponding to the view pattern, the step of mapping and transforming the first positions into the second positions, so as to obtain the line of sight movement track of the user looking at the view pattern includes: Calculating a distance parameter according to the pupil and the first reflective point; calculating an angle parameter according to the vector of the first reflective point relative to the pupil; analyzing the distribution relationship of the distance parameter and the angle parameter to convert the first positions into the second positions in the coordinate system corresponding to the view pattern; and These second positions are used as the line of sight movement track of the user looking at the view pattern. 6. The control method according to claim 5, wherein the eye target further includes a second reflective point, and according to the coordinate system corresponding to the view pattern, the first positions are mapped and transformed into the the second positions, the step of obtaining the line of sight movement track of the user looking at the view pattern includes: calculating an area parameter according to the pupil, the first reflective point, and the second reflective point; calculating an angle parameter according to at least one of a first connection line between the first reflection point and the pupil and a second connection line between the second reflection point and the pupil; analyzing the distribution relationship of the area parameter and the angle parameter to convert the first positions into the second positions in the coordinate system corresponding to the view pattern; and These second positions are used as the line of sight movement track of the user looking at the view pattern. 7. The control method according to claim 4, wherein the eye tracking action is performed according to the eye object, so as to sequentially obtain the first images of the eye object corresponding to the image sequence. The location steps also include: After one of the first positions is obtained, the eye area is adjusted to detect the eye object again, and the eye tracking action is performed accordingly. 8. The control method according to claim 1, wherein the step of performing the corresponding operation on the device according to the judgment result comprises: When the moving track of the line of sight conforms to the preset track, the device releases the locked state of the lock; and When the moving track of the line of sight does not conform to the preset track, the device maintains the locked state of the lock. 9. A device for controlling based on eye movements, comprising: An image capturing unit to obtain an image sequence; a storage unit that records a plurality of program modules and includes a database; and A processing unit, coupled to the imaging unit and the storage unit, to access and execute the program module recorded in the storage unit, the program module includes: An image analysis module, analyzing the image sequence, so as to obtain the eye image information of the user's eye area in each image of the image sequence; A line of sight detection module, based on the eye image information, to detect the moving track of the user's line of sight while gazing at the view pattern; A judging module, judging whether the movement track of the line of sight conforms to a preset track relative to the view pattern, so as to generate a judgment result; and The control module executes corresponding operations through the device according to the judgment result. 10 . The device according to claim 9 , wherein the view pattern includes a plurality of identification patterns, and the preset trajectory corresponds to at least one geometric connection formed by the identification patterns in sequence. 11 . 11. The device according to claim 9, wherein the line of sight detection module analyzes the eye image information to determine whether the user is gazing at a preset position in the view pattern, or gazing at the The preset position exceeds the preset time, and when the user gazes at the preset position in the view pattern, or gazes at the preset position for more than the preset time, the line of sight detection module detects the The moving track of the line of sight of the user looking at the view pattern. 12. The device according to claim 9, wherein the line-of-sight detection module comprises: an eye detection module, configured to detect an eye target in the eye region based on the eye image information; an eye tracking module, performing an eye tracking action according to the eye target, to sequentially obtain a plurality of first positions of the eye target in the image sequence; and The mapping transformation module maps and transforms the first positions into a plurality of second positions according to the coordinate system corresponding to the view pattern, so as to obtain the line-of-sight movement track of the user looking at the view pattern. 13. The device according to claim 12, wherein the eye target includes a pupil and a first reflection point, and the eye tracking module performs an operation on the pupil and the first reflection point of the image sequence point for performing the eye tracking action, and sequentially obtaining the first positions of the eye target corresponding to the sequence of images, Wherein, the mapping transformation module calculates a distance parameter according to the pupil and the first reflective point, and calculates an angle parameter according to a vector of the first reflective point relative to the pupil, and analyzes the distance parameter and The distribution relationship of the angle parameters is used to convert the first positions into the second positions in the coordinate system corresponding to the view pattern, and to use the second positions as the user looking at the The gaze movement trajectory of the view pattern. 14. The device according to claim 13, wherein the eye object further includes a second reflection point, and the mapping conversion module is based on the pupil, the first reflection point and the second reflection point point, calculate the area parameter, and calculate the angle parameter according to at least one of the first connection line between the first reflection point and the pupil and the second connection line between the second reflection point and the pupil, and analyze The distribution relationship of the area parameter and the angle parameter is used to convert the first positions into the second positions in the coordinate system corresponding to the view pattern, and use the second positions as the used The viewer is looking at the line of sight movement track of the view pattern. 15. The device according to claim 12, wherein the eye tracking module further adjusts the eye area after obtaining one of the first positions, so that the eye detection module again The eye target is detected, and the eye tracking module performs the eye tracking action accordingly. 16. The device according to claim 9, characterized in that, when the moving track of the line of sight conforms to the preset track, the control module controls the device to release the locked state of the lock, and when the moving track of the line of sight conforms to the preset track When the preset trajectory is not met, the control module controls the device to maintain the locked state of the lock.