CN110633014B - Head-wearing eye movement tracking device - Google Patents
Head-wearing eye movement tracking device Download PDFInfo
- Publication number
- CN110633014B CN110633014B CN201911011899.6A CN201911011899A CN110633014B CN 110633014 B CN110633014 B CN 110633014B CN 201911011899 A CN201911011899 A CN 201911011899A CN 110633014 B CN110633014 B CN 110633014B
- Authority
- CN
- China
- Prior art keywords
- eye
- module
- video image
- circuit unit
- head
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004424 eye movement Effects 0.000 title claims abstract description 13
- 210000001508 eye Anatomy 0.000 claims abstract description 103
- 238000012545 processing Methods 0.000 claims abstract description 45
- 210000003128 head Anatomy 0.000 claims abstract description 25
- 230000005540 biological transmission Effects 0.000 claims abstract description 20
- 230000004438 eyesight Effects 0.000 claims abstract description 9
- 230000033001 locomotion Effects 0.000 claims abstract description 8
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 7
- 235000002911 Salvia sclarea Nutrition 0.000 claims abstract description 5
- 244000182022 Salvia sclarea Species 0.000 claims abstract description 5
- 230000009471 action Effects 0.000 claims abstract description 5
- 238000003384 imaging method Methods 0.000 claims description 9
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 claims description 8
- 229910052744 lithium Inorganic materials 0.000 claims description 8
- 229920000642 polymer Polymers 0.000 claims description 8
- 230000003139 buffering effect Effects 0.000 claims description 3
- 230000008033 biological extinction Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 claims description 2
- 208000020016 psychiatric disease Diseases 0.000 abstract description 3
- 238000003745 diagnosis Methods 0.000 abstract description 2
- 230000001575 pathological effect Effects 0.000 abstract description 2
- 241000699670 Mus sp. Species 0.000 abstract 2
- 230000004397 blinking Effects 0.000 abstract 2
- 241000699666 Mus <mouse, genus> Species 0.000 abstract 1
- 210000001747 pupil Anatomy 0.000 description 12
- 238000013461 design Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000003340 mental effect Effects 0.000 description 2
- 238000000034 method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/14—Arrangements specially adapted for eye photography
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Veterinary Medicine (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a head-mounted eye movement tracking device. The invention is worn on the head of a user, and comprises a head wearing piece, a miniature infrared camera module, an eye video image processing module, a wireless data transmission module and a power supply module, wherein the miniature infrared camera module is arranged in the head wearing piece; when the eye-protection device is worn on the head, the infrared camera bracket on the wearing piece is adjusted to be aligned with eyes, so that clear eye images can be displayed; the eyeball movement track can be recorded, so that a reference is provided for diagnosis of mental diseases; the system can track eye sight information, detect blinking actions and convert the blinking actions into mouse signals to control digital terminals such as computers, mobile phones and the like. The invention can provide assistance for doctors to diagnose pathological examination and mental diseases of eyes of patients, can also realize that cursors are controlled by eye movement information to roughly replace mice, and brings convenience for people inconvenient to use the mice by hands, and has low equipment cost and convenient wearing.
Description
Technical Field
The invention relates to the field of eye tracking devices, in particular to a head-mounted eye tracking device.
Background
Eyes are important organs for human to acquire information from the surrounding world, and by accurately measuring the gaze point of a person, the eyes can be used for providing important basis for researching human mental activities and extracting mental consciousness, and the gaze point can be used for replacing a mouse or touch operation, so that human-computer interaction by using the gaze point is realized; the eye tracking device is used as a vision drop point measuring device, and has wide application prospects in various industries, including fields of rehabilitation, advertisement design, game design, man-machine interface design and the like. Although the eye tracking device has wide application prospect, most of the eye tracking devices in China are head-limited, and a measured person needs to fix the head on a plane and can not move the head during measurement, so that the eye tracking device is very inflexible. The camera is far away from the eyes, the occupation of the eyes pixels in each frame of image acquired by the camera is small, interference caused by complex background images cannot be avoided, the complexity of a tracking algorithm on non-wearable equipment is high, the calculated amount is large, the requirement on hardware operation speed is high, the hardware resources of controlled equipment are occupied very, the instantaneity is poor, and the user experience is poor.
If an eye tracking device capable of being worn on the head can be provided, the wearing tightness and the angle of a camera are adjustable, different users can be applied, the wearing is comfortable, the self-contained computing core is not used, a large amount of hardware resources of controlled equipment are occupied, the requirement on the hardware performance of the controlled equipment is reduced, the use experience of the eye tracking technology can be greatly improved, and the market prospect is wide. The present invention has been made accordingly.
Disclosure of Invention
The invention aims to solve some defects of the existing eye tracking device, and discloses a wearable eye tracking device with low cost.
A head-mounted eye movement tracking device comprises a head wearing piece (1), a miniature infrared camera module (2) arranged in the head wearing piece, an eye video image processing module (3), a wireless data transmission module (4) and a power supply module (5); the eye tracking system has three working modes of eye imaging, eyeball track tracking and human eye sight tracking;
the head wearing piece (1) is provided with an USB Type-C input socket (1-1), a rear fixed clamping belt (1-2), a front fixed clamping belt (1-3), an LED indicator lamp (1-4), a multifunctional light touch key (1-5), a left camera bracket (1-6), a right camera bracket (1-7), a clamping belt tightness adjustment left knob (1-8), a left camera bracket rotation adjustment knob (1-9), a clamping belt tightness adjustment right knob (1-10) and a right camera bracket rotation adjustment knob (1-11);
wherein the multifunctional touch key (1-5) is positioned at the middle position of the front side of the front fixed clamping belt (1-3); the LED indicator lamps (1-4) are positioned at two sides of the multifunctional touch key (1-5); the left camera bracket (1-6) is connected with the left end of the front fixed clamping band (1-3); the right camera bracket (1-7) is connected with the right end of the front fixed clamping band (1-3); the USB Type-C input socket (1-1) is positioned at the middle position outside the cavity of the rear fixing clamp belt (1-2); the left clamping band tightness adjusting knob (1-8) is positioned at the left connecting part of the rear fixed clamping band (1-2) and the front fixed clamping band (1-3); the clamping band tightness adjusting right knob (1-10) is positioned at the right joint of the rear fixed clamping band (1-2) and the front fixed clamping band (1-3); the left camera bracket rotating adjusting knob (1-9) is positioned at the joint of the left camera bracket (1-6) and the front fixed clamping belt (1-3), and the right camera bracket rotating adjusting knob (1-11) is positioned at the right joint of the right camera bracket (1-7) and the front fixed clamping belt (1-3); the USB Type-C input socket (1-1) is used as a charging input and limited data transmission socket; the multifunctional touch keys (1-5) are used for opening or closing the head-mounted eye tracking device, indicating the state of a switch and switching the working mode; the LED indicator lamps (1-4) are used for indicating different working modes, charging states and residual electric quantity; the rear fixing clamp (1-2) and the front fixing clamp (1-3) are used for wearing the device on the head; the left clamping band tightness adjusting knob (1-8) and the right clamping band tightness adjusting knob (1-10) are used for adjusting the tightness of the device in wearing so as to be suitable for different users; the left camera bracket (1-6) and the right camera bracket (1-7) are respectively used for fixing the miniature infrared camera and can respectively rotate up and down by taking the left camera bracket rotary adjusting knob (1-9) and the right camera bracket rotary adjusting knob (1-11) as the center; the left camera bracket rotary adjusting knob (1-9) and the right camera bracket rotary adjusting knob (1-11) are respectively used for adjusting the rotary positions of the left camera bracket (1-6) and the right camera bracket (1-7) so that the micro infrared cameras are aligned to eyes;
an eye video image processing module (3) which is internally arranged in the head wearing piece (1) is respectively connected with the miniature infrared camera module (2), the wireless data transmission module (4) and the power module (5);
the eye video image processing module (3) comprises an eye video image data buffer circuit unit (3-1), an FPGA image processing and control circuit unit (3-2), an LED indicator light driving circuit unit (3-3) and a key circuit unit (3-4); the miniature infrared camera module (2) is used for collecting eye images of a user; the eye video image processing module (3) is used for processing video image signals output by the miniature infrared camera module (2) to finish pupil center positioning and eye action recognition; the wireless data transmission module (4) is used for transmitting the processing result obtained by the eye video image processing module (3) to a mobile phone, a computer or other digital terminals in a wireless mode; the power module (5) comprises a charging circuit unit (5-1) and a 3.7V polymer lithium battery (5-2) which is used for supplying power to the system.
The number of cameras of the miniature infrared camera module (2) is 2, and the left camera bracket (1-6) and the right camera bracket (1-7) which are respectively arranged in the wearing piece (1) are integrated with infrared light emitting diodes; the middle position of the rear side of the rear fixing clamp belt (1-2) is provided with a larger cavity for fixing and protecting the eye video image processing module (3), the wireless data transmission module (4) and the power supply module (5); the eye video image processing module (3) comprises an FPGA image processing and control circuit unit (3-2), an eye video image data buffer circuit unit (3-1), an LED indicator light driving circuit unit (3-3) and a key circuit unit (3-4) which are respectively connected with the FPGA image processing and control circuit unit; the eye video image data buffer circuit unit (3-1) is used for buffering video image data output by the miniature infrared camera module (2) by taking frames as units; the FPGA image processing and control circuit unit (3-2) is used for reading eye video image data from the eye video image data buffer circuit unit (3-1) by taking a frame as a unit, processing the eye video image data to realize pupil center positioning and eye motion recognition, transmitting the obtained processing result to the wireless data transmission module (4), judging the state of the multifunctional key (1-5) according to the output result of the key circuit unit (3-4) to realize the switching of working modes and controlling the display of the LED indicator lamp circuit unit (3-3); the charging circuit (5-1) in the power module (5) is used for charging the 3.7V polymer lithium battery (5-2) through the USB Type-C input socket (1-1); the 3.7V polymer lithium battery (5-2) provides working voltage for each module.
The beneficial effects of the invention are as follows:
the head-mounted eye movement tracking device provided by the invention has the advantages that the camera view finding angle is adjustable, the wearing tightness is adjustable, the head-mounted eye movement tracking device is applicable to different wearing individuals and is comfortable to wear, the camera bracket is adjusted to enable the camera to be positioned at a position where a clear eye image can be shot and the sight of a human eye is not blocked, the eye gaze point can be tracked, the eyeball movement track can be tracked, and the clear eye image can be displayed; the wireless data transmission can be utilized to be associated with digital terminals such as computers, mobile phones and the like; corresponding driving software is installed on the computer end or the mobile phone end, and when the computer end or the mobile phone end is in a human eye sight line tracking working mode, the computer or the mobile phone can be used by utilizing a human eye gaze point to replace a mouse or touch operation; when the device is in an eyeball track tracking or eye imaging mode, clear real-time eye images can be displayed on a computer or mobile phone end. The device can assist doctors in pathological examination of eyes of patients and diagnosis of mental diseases, can also realize rough replacement of a mouse by controlling a cursor through eye movement information, and brings convenience to people inconvenient to use the mouse by hands. The embedded hardware is taken as a computing core to process the digital image, so that the occupancy rate of hardware resources of the controlled equipment is low, the requirement on the hardware performance of the controlled equipment is lower, the use limit is less, the equipment cost is low, and the wearing is convenient.
Drawings
FIG. 1 is a left side view of a head wear of a head mounted eye tracking device in accordance with the present invention;
FIG. 2 is a right side view of a head wear of a head mounted eye tracking device in accordance with the present invention;
FIG. 3 is a rear view of a head-mounted eye-tracking device head wear in accordance with the present invention;
FIG. 4 is a block diagram of a head mounted eye tracker system module in accordance with the present invention;
Detailed Description
Exemplary embodiments of the present invention will be described hereinafter with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in this specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made in order to achieve the developer's specific goals, such as compliance with system-and business-related constraints, and that these constraints will vary from one implementation to another.
It should be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only the modules and/or circuits closely related to the solution according to the present invention are shown in the drawings, while other details not greatly related to the present invention are omitted.
The first embodiment is as follows:
as shown in fig. 1, 2, and 3, the head-mounted eye tracking device according to the present embodiment includes:
a head wearing member 1 for wearing the device on the head of different individuals and securing and protecting the electronic components required for the device; the wearing piece 1 is provided with a USB Type-C input socket 1-1, a rear fixed clamping band 1-2, a front fixed clamping band 1-3, an LED indicator lamp 1-4, a multifunctional touch key 1-5, a left camera bracket 1-6, a right camera bracket 1-7, a clamping band tightness adjustment left knob 1-8, a left camera bracket rotation adjustment knob 1-9, a clamping band tightness adjustment right knob 1-10 and a right camera bracket rotation adjustment knob 1-11;
the multifunctional touch key 1-5 is positioned in the middle of the front side of the front fixed clamping band 1-3 and is used for opening or closing the eye movement centering device, indicating the state of a switch and switching the working mode; the multifunctional key 1-5 is pressed lightly, the multifunctional key 1-5 emits red light, the eye movement tracking device is opened and is in an eye imaging working mode by default, and the left and right camera brackets can be adjusted according to the imaging effect, so that the camera is in a position capable of capturing clear eye images and not shielding the vision of human eyes; then quickly and lightly pressing the two multifunctional keys 1-5, and switching to an eyeball motion track tracking mode after the red light emitted by the two multifunctional keys flickers; then quickly and lightly pressing the two multifunctional keys 1-5, and switching to a human eye sight tracking mode after the red light emitted by the two multifunctional keys flickers; then the two multifunctional keys 1-5 are pressed lightly, and the red light emitted by the two multifunctional keys flashes once and then is switched back to a default eye imaging mode; long-time pressing of the multifunctional key 1-5, extinction of red light of the functional key 1-5 and closing of the eye movement tracking device;
the LED indicator lamps 1-4 are positioned at two sides of the multifunctional touch key 1-5; when the eye tracking device is in different working states, the LED indicator lights on the left side of the multifunctional keys 1-5 show different colors so as to indicate different working states; the LED indicator lights on the right sides of the multifunctional keys 1-5 are used for indicating different electric quantity grades, and when the residual electric quantity is 100% -70%, the LED indicator lights emit green light; when the residual electric quantity is 69% -40%, the yellow light is emitted; when the electric quantity is 39% -10%, the LED emits red light; when the electric quantity is lower than 10%, the LED lamp blinks to emit red light; the device blinks to emit green light when charged, and is constant green light when the electric quantity is full;
the left camera bracket 1-6 is connected with the left end of the front fixed clamping band 1-3; the device can be used for fixing and adjusting the miniature infrared camera module; the right camera bracket 1-7 is connected to the right end of the front fixing clamping band 1-3 and is used for fixing and adjusting the miniature infrared camera module;
the USB Type-C input socket 1-1 is positioned at the middle position outside the cavity of the rear fixing clamp 1-2; the device is used for realizing the charging and wired data transmission of the device;
the left clamping band tightness adjusting knob 1-8 is positioned at the left joint of the rear fixed clamping band 1-2 and the front fixed clamping band 1-3; the clamping band tightness adjusting right knob 1-10 is positioned at the right joint of the rear fixed clamping band 1-2 and the front fixed clamping band 1-3; the diameter of a circle formed by the rear fixing clamp 1-2 and the front fixing clamp 1-3 can be adjusted by rotating the left knob 1-8 and the right knob 1-10, so that individuals with different head diameters can wear the device on the head firmly;
the left camera bracket rotary adjusting knob 1-9 is positioned at the joint of the left camera bracket 1-6 and the front fixed clamping band 1-3, the right camera bracket rotary adjusting knob 1-11 is positioned at the right joint of the right camera bracket 1-7 and the front fixed clamping band 1-3, and the rotary positions of the left camera bracket 1-6 and the right camera bracket 1-7 are respectively used for adjusting the micro infrared camera to be aligned with eyes;
the second embodiment is as follows: unlike the first embodiment, the head-mounted eye tracking apparatus according to the present embodiment, as shown in fig. 1 and 4, includes:
a miniature infrared camera module 2, an eye video image processing module 3, a wireless data transmission module 4 and a power module 5 which are arranged in the head wearing piece 1; the eye video image processing module 3 is respectively connected with the miniature infrared camera module 2, the wireless data transmission module 4 and the power module 5; the number of the micro infrared camera module cameras is 2, and the micro infrared light emitting diodes are integrated to be responsible for acquiring human eye video image data; the eye video image processing module 3 is responsible for the processing of the whole eye tracking device algorithm, the wireless data transmission module 4 is used for transmitting the result obtained after the eye video image processing module 3 processes the eye video image data to the related digital equipment such as a computer or a mobile phone in a wireless mode, and the power supply module 5 is used for supplying power and charging the system;
and a third specific embodiment: unlike the second embodiment, the head-mounted eye tracking apparatus of the present embodiment is shown in fig. 4:
the eye video image processing module (3) comprises an eye video image data buffer circuit unit 3-1, an FPGA image processing and control circuit unit 3-2, an LED indicator light driving circuit unit 3-3 and a key circuit unit 3-4; an FPGA image processing and control circuit unit 3-2 in the eye video image processing module 3 is respectively connected with an eye video image data buffer circuit unit 3-1, an LED indicator light driving circuit unit 3-3 and a key circuit unit 3-4; the eye video image data buffer circuit unit 3-1 is used for buffering the video image data output by the micro infrared camera module 2 in units of frames; the FPGA image processing and control circuit unit 3-2 is used for reading eye video image data from the eye video image data buffer circuit unit 3-1 by taking a frame as a unit, processing the eye video image data to realize pupil center positioning and eye action recognition, transmitting the obtained processing result to the wireless data transmission module 4, judging the state of the multifunctional key 1-5 according to the output result of the key circuit unit 3-4 to realize the switching of working modes and controlling the display of the LED indicator lamp driving circuit unit 3-3;
the power supply module 5 comprises a charging circuit unit 5-1 and a 3.7V polymer lithium battery 5-2; wherein the charging circuit (5-1) is used for realizing the charging of the 3.7V polymer lithium battery 5-2 through the USB Type-C input socket 1-1.
The specific embodiment IV is as follows: unlike the third embodiment, the specific steps of the algorithm for eye tracking of the head-mounted eye tracking device of the present embodiment are as follows:
step one, bright and dark pupil image acquisition:
the miniature infrared light emitting diode integrated by the miniature infrared camera module 2 flashes at a certain frequency, the FPGA image processing and control circuit unit 3-2 in the eye video image processing module 3 controls the frame rate of the output image of the miniature infrared camera module 2 and the flashing frequency of the miniature infrared light emitting diode to make the two frequencies identical, the obtained eye images are continuous 2-frame images, only one frame image is an image with the light supplementing effect of the miniature infrared diode, the continuous two frames of images containing the pupil of the human eye are respectively images with alternating bright pupils and dark pupils, and the two frames of images are cached in the eye video image data cache circuit unit 3-1.
Step two, pupil center coordinate extraction:
the first step is utilized to obtain bright and dark pupil images, the FPGA image processing and control circuit unit 3-2 reads two continuous frames of images from the eye video image data buffer circuit unit 3-1, and differential operation is carried out by using pixels in the current frame of images and pixels corresponding to the next frame of images according to a formula (1-1); wherein T is 1 For a set threshold value, (x) i ) Representing the current frame x in two consecutive frames of images i The value of each pixel, B (x i ) Values representing pixels of a subsequent frame in the two successive frames; pixels satisfying the formula (1-1) are determined as pupil image pixels in the eye image, otherwise, are determined as non-pupil image pixels; simultaneously setting the pupil image pixel as 1 and the non-pupil image pixel as 0 to form a binary image, thus obtaining a binary image only containing pupils, respectively calculating the minimum circumcircle of the pupil image pixels, and taking the center of the circle as the center coordinate of the pupil;
F(x i )-B(x i )>T 1 (1-1)
step three, mouse time sequence simulation:
the wireless data transmission module 4 encodes the obtained pupil center coordinate data into a signal data format of a mouse, simulates the signal time sequence of mouse equipment and transmits the data, so that an associated computer or mobile phone and the like can treat the eye tracking device as the mouse equipment, and the movement of a mouse cursor is controlled by utilizing the movement of the center coordinate, so that the equipment which supports the mouse control can be controlled by using the eye tracking device;
while the invention has been described in terms of the above embodiments, those skilled in the art, having benefit of the foregoing description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention, which is defined by the appended claims.
Claims (2)
1. A head-mounted eye tracking device, characterized in that: the device comprises a head wearing piece (1), a miniature infrared camera module (2) arranged in the head wearing piece, an eye video image processing module (3), a wireless data transmission module (4) and a power supply module (5); the eye tracking system has three working modes of eye imaging, eyeball track tracking and human eye sight tracking;
the head wearing piece (1) is provided with an USB Type-C input socket (1-1), a rear fixed clamping belt (1-2), a front fixed clamping belt (1-3), an LED indicator lamp (1-4), a multifunctional light touch key (1-5), a left camera bracket (1-6), a right camera bracket (1-7), a clamping belt tightness adjustment left knob (1-8), a left camera bracket rotation adjustment knob (1-9), a clamping belt tightness adjustment right knob (1-10) and a right camera bracket rotation adjustment knob (1-11); wherein the multifunctional touch key (1-5) is positioned at the middle position of the front side of the front fixed clamping belt (1-3); the LED indicator lamps (1-4) are positioned at two sides of the multifunctional touch key (1-5); the left camera bracket (1-6) is connected with the left end of the front fixed clamping band (1-3); the right camera bracket (1-7) is connected with the right end of the front fixed clamping band (1-3); the USB Type-C input socket (1-1) is positioned at the middle position outside the cavity of the rear fixing clamp belt (1-2); the left clamping band tightness adjusting knob (1-8) is positioned at the left connecting part of the rear fixed clamping band (1-2) and the front fixed clamping band (1-3); the clamping band tightness adjusting right knob (1-10) is positioned at the right joint of the rear fixed clamping band (1-2) and the front fixed clamping band (1-3); the left camera bracket rotating adjusting knob (1-9) is positioned at the joint of the left camera bracket (1-6) and the front fixed clamping belt (1-3), and the right camera bracket rotating adjusting knob (1-11) is positioned at the right joint of the right camera bracket (1-7) and the front fixed clamping belt (1-3);
the USB Type-C input socket (1-1) is used as a charging input and wired data transmission socket; the multifunctional touch keys (1-5) are used for opening or closing the head-mounted eye tracking device, indicating the state of a switch and switching the working mode; the multifunctional key (1-5) is pressed lightly, the multifunctional key (1-5) emits red light, the eye movement tracking device is opened and defaults to be in an eye imaging working mode, and the left and right camera brackets can be adjusted according to the imaging effect, so that the camera is in a position capable of shooting clear eye images and not shielding the vision of human eyes; then quickly and lightly pressing the two multifunctional keys (1-5), and switching to an eyeball motion track tracking mode after the red light emitted by the two multifunctional keys flickers; then quickly and lightly pressing the two multifunctional keys (1-5), and switching to a human eye sight tracking mode after the red light emitted by the two multifunctional keys flickers; then the two multifunctional keys (1-5) are pressed lightly, and the red light emitted by the multifunctional keys flashes once and then is switched back to a default eye imaging mode; long-time pressing of the multifunctional key (1-5), extinction of red light of the functional key (1-5) and closing of the eye movement tracking device; the LED indicator lamps (1-4) are used for indicating different working modes, charging states and residual electric quantity; the device comprises a left camera bracket (1-6) and a right camera bracket (1-7), a left camera bracket rotary adjusting knob (1-9) and a right camera bracket rotary adjusting knob (1-11) respectively, wherein the rear fixed clamping band (1-2) and the front fixed clamping band (1-3) are used for wearing the device on the head;
an eye video image processing module (3) arranged in the wearing piece (1) is respectively connected with the miniature infrared camera module (2), the wireless data transmission module (4) and the power module (5); the eye video image processing module (3) comprises an eye video image data buffer circuit unit (3-1), an FPGA image processing and control circuit unit (3-2), an LED indicator light driving circuit unit (3-3) and a key circuit unit (3-4);
the miniature infrared camera module (2) is used for collecting eye images of a user; the eye video image processing module (3) is used for processing video image signals output by the miniature infrared camera module (2) to finish pupil center positioning and eye action recognition; the wireless data transmission module (4) is used for transmitting the processing result obtained by the eye video image processing module (3) to a mobile phone, a computer or other digital terminals in a wireless mode; the power module (5) comprises a charging circuit unit (5-1) and a 3.7. 3.7V lithium polymer battery (5-2) which is used for supplying power to the system.
2. A head-mounted eye-tracking device according to claim 1, wherein: the number of cameras of the miniature infrared camera module (2) is 2, and the left camera bracket (1-6) and the right camera bracket (1-7) which are respectively arranged in the head wearing piece (1) are integrated with miniature infrared light emitting diodes; the middle position of the rear side of the rear fixing clamp belt (1-2) is provided with a larger cavity for fixing and protecting the eye video image processing module (3), the wireless data transmission module (4) and the power supply module (5); the FPGA image processing and control circuit unit (3-2) in the eye video image processing module (3) is respectively connected with the eye video image data buffer circuit unit (3-1), the LED indicator light circuit unit (3-3) and the key circuit unit (3-4); the eye video image data buffer circuit unit (3-1) is used for buffering video image data output by the miniature infrared camera module (2) by taking frames as units; the FPGA image processing and control circuit unit (3-2) is used for reading eye video image data from the eye video image data buffer circuit unit (3-1) by taking a frame as a unit, processing the eye video image data to realize pupil center positioning and eye motion recognition, transmitting the obtained processing result to the wireless data transmission module (4), judging the state of the multifunctional key (1-5) according to the output result of the key circuit unit (3-4) to realize the switching of working modes and controlling the display of the LED indicator lamp circuit unit (3-3); the charging circuit (5-1) in the power module (5) is used for charging the 3.7V lithium polymer battery (5-2) through the USB Type-C input socket (1-1); the 3.7V lithium polymer battery (5-2) provides working voltage for each module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911011899.6A CN110633014B (en) | 2019-10-23 | 2019-10-23 | Head-wearing eye movement tracking device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911011899.6A CN110633014B (en) | 2019-10-23 | 2019-10-23 | Head-wearing eye movement tracking device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110633014A CN110633014A (en) | 2019-12-31 |
| CN110633014B true CN110633014B (en) | 2024-04-05 |
Family
ID=68977431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911011899.6A Active CN110633014B (en) | 2019-10-23 | 2019-10-23 | Head-wearing eye movement tracking device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633014B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111714080B (en) * | 2020-06-30 | 2021-03-23 | 重庆大学 | Disease classification system based on eye movement information |
| CN114035673A (en) * | 2020-11-18 | 2022-02-11 | 武汉联影医疗科技有限公司 | Ultrasonic eye movement control system and method |
| CN112674716B (en) * | 2020-12-11 | 2024-02-27 | 首都医科大学附属北京儿童医院 | Head position measuring device capable of collecting eye vibration data |
| CN113238660A (en) * | 2021-07-01 | 2021-08-10 | 顺德职业技术学院 | Induction tracking method based on virtual reality technology |
| US12114180B2 (en) * | 2022-01-24 | 2024-10-08 | Qualcomm Incorporated | Wireless communication techniques employing beamforming based on spatial relationship to human tissue |
| CN117530201A (en) * | 2023-10-20 | 2024-02-09 | 中国科学院深圳先进技术研究院 | Wearable head eye tracking device, method, processing device and system |
| CN117814794B (en) * | 2023-12-13 | 2024-08-27 | 山东省体育科学研究中心 | Eye movement feedback instrument for monitoring and evaluating motor psychological quality |
| CN118490159B (en) * | 2024-06-11 | 2025-02-11 | 中国人民解放军空军军医大学 | An eye movement recorder with a wearable anti-falling structure |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002000567A (en) * | 2000-06-23 | 2002-01-08 | Kansai Tlo Kk | Method of measuring pupil center position and method of detecting view point position |
| CN104055478A (en) * | 2014-07-08 | 2014-09-24 | 金纯� | Medical endoscope control system based on sight tracking control |
| CN104866105A (en) * | 2015-06-03 | 2015-08-26 | 深圳市智帽科技开发有限公司 | Eye movement and head movement interactive method for head display equipment |
| CN107247511A (en) * | 2017-05-05 | 2017-10-13 | 浙江大学 | A kind of across object exchange method and device based on the dynamic seizure of eye in virtual reality |
| CN107388201A (en) * | 2017-09-05 | 2017-11-24 | 哈尔滨理工大学 | The dynamic control operation illuminating lamp of medical wear-type eye |
| WO2018076202A1 (en) * | 2016-10-26 | 2018-05-03 | 中国科学院深圳先进技术研究院 | Head-mounted display device that can perform eye tracking, and eye tracking method |
| WO2019196133A1 (en) * | 2018-04-09 | 2019-10-17 | 杭州瑞杰珑科技有限公司 | Head-mounted visual aid device |
-
2019
- 2019-10-23 CN CN201911011899.6A patent/CN110633014B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002000567A (en) * | 2000-06-23 | 2002-01-08 | Kansai Tlo Kk | Method of measuring pupil center position and method of detecting view point position |
| CN104055478A (en) * | 2014-07-08 | 2014-09-24 | 金纯� | Medical endoscope control system based on sight tracking control |
| CN104866105A (en) * | 2015-06-03 | 2015-08-26 | 深圳市智帽科技开发有限公司 | Eye movement and head movement interactive method for head display equipment |
| CN108170279A (en) * | 2015-06-03 | 2018-06-15 | 塔普翊海（上海）智能科技有限公司 | The eye of aobvious equipment is moved moves exchange method with head |
| WO2018076202A1 (en) * | 2016-10-26 | 2018-05-03 | 中国科学院深圳先进技术研究院 | Head-mounted display device that can perform eye tracking, and eye tracking method |
| CN107247511A (en) * | 2017-05-05 | 2017-10-13 | 浙江大学 | A kind of across object exchange method and device based on the dynamic seizure of eye in virtual reality |
| CN107388201A (en) * | 2017-09-05 | 2017-11-24 | 哈尔滨理工大学 | The dynamic control operation illuminating lamp of medical wear-type eye |
| WO2019196133A1 (en) * | 2018-04-09 | 2019-10-17 | 杭州瑞杰珑科技有限公司 | Head-mounted visual aid device |
Non-Patent Citations (2)
| Title |
|---|
| 基于视线追踪的眼控鼠标设计;朱麒文;闫隆鑫;张若兰;张帅;杨兰兰;;电子器件;20160420(02);全文 * |
| 穿戴式视线追踪系统研究;曹晔;;计算机与数字工程;20160620(06);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110633014A (en) | 2019-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633014B (en) | Head-wearing eye movement tracking device | |
| CN103838378B (en) | A kind of wear-type eyes control system based on pupil identification positioning | |
| CN104055478B (en) | Based on the medical endoscope control system that Eye-controlling focus controls | |
| CN205903239U (en) | Visual acuity test and trainer based on virtual reality | |
| CN104766590B (en) | Head-mounted display device and backlight adjusting method thereof | |
| KR100949743B1 (en) | Apparatus and method for wearable eye tracking having goggle typed | |
| US8988373B2 (en) | Skin input via tactile tags | |
| CN201477518U (en) | A gaze tracking device based on pupil cornea reflection method | |
| Rantanen et al. | A Wearable, Wireless Gaze Tracker with Integrated Selection Command Source for Human‐Computer Interaction | |
| CN205318022U (en) | Wearable intelligent vision enhancement equipment | |
| CN104090371B (en) | A kind of 3D glasses and 3D display systems | |
| WO2018214520A1 (en) | Method and apparatus for synchronizing video data and exercise effect animation | |
| CN104820500A (en) | Virtual reality helmet with electroencephalogram control function | |
| WO2017203815A1 (en) | Information processing device, information processing method, and recording medium | |
| CN205359411U (en) | Endoscope image control system | |
| CN107122039A (en) | A kind of intelligent vision auxiliary strengthening system and its application method | |
| CN107088145A (en) | Visual function training method and system | |
| CN205318021U (en) | Wearable intelligent vision enhancement equipment of disconnect -type | |
| CN113138664A (en) | Eyeball tracking system and method based on light field perception | |
| CN106842565A (en) | A kind of wearable intelligent vision enhancing equipment of separate type | |
| Kim et al. | Development of an untethered, mobile, low-cost head-mounted eye tracker | |
| CN105380590B (en) | A kind of equipment and its implementation with eye position detection function | |
| WO2023159526A1 (en) | Eyeball tracking system and use method therefor | |
| CN104483754A (en) | Head-wearing type multimedia terminal assisted watching system aiming at patient with dysopia | |
| CN114052649A (en) | Alternate covering strabismus diagnosis method based on virtual reality and eye movement tracking technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221202 Address after: 213031 NO.666, Liaohe Road, Xinbei District, Changzhou City, Jiangsu Province Applicant after: CHANGZHOU INSTITUTE OF TECHNOLOGY Address before: 150080 No. 52, Xuefu Road, Nangang District, Heilongjiang, Harbin Applicant before: HARBIN University OF SCIENCE AND TECHNOLOGY |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |