CN107380064B - Vehicle-mounted sight tracking device based on augmented reality technology - Google Patents
Vehicle-mounted sight tracking device based on augmented reality technology Download PDFInfo
- Publication number
- CN107380064B CN107380064B CN201710539990.XA CN201710539990A CN107380064B CN 107380064 B CN107380064 B CN 107380064B CN 201710539990 A CN201710539990 A CN 201710539990A CN 107380064 B CN107380064 B CN 107380064B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- sight
- automobile
- data
- augmented reality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 25
- 238000005516 engineering process Methods 0.000 title claims abstract description 25
- 230000000007 visual effect Effects 0.000 claims abstract description 20
- 230000003993 interaction Effects 0.000 claims abstract description 15
- 239000000284 extract Substances 0.000 claims abstract description 5
- 239000011521 glass Substances 0.000 claims description 8
- 230000002452 interceptive effect Effects 0.000 claims description 5
- 230000004397 blinking Effects 0.000 claims 1
- 238000005457 optimization Methods 0.000 description 8
- 210000004087 cornea Anatomy 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000001747 pupil Anatomy 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000000034 method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/804—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for lane monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Mechanical Engineering (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a vehicle-mounted sight tracking device based on augmented reality technology, which comprises an image acquisition device, a human-vehicle interaction display device, a sight tracking module and a control module, wherein the control module comprises a controller, a data processor and a data memory, the sight tracking module comprises a light source component and an image sensor, the sight tracking module tracks a sight gazing target of a user of an automobile and captures sight characteristics, the controller generates a corresponding control instruction according to the sight characteristics, the image acquisition device acquires an external image of the automobile according to the control instruction and extracts the image characteristics, the image characteristics are processed by the data processor to form data information, the data information is matched with stored data in the data memory, and the controller displays a visual program corresponding to the successfully-matched stored data on the human-vehicle interaction display device. The vehicle-mounted sight tracking device based on the augmented reality technology can help a vehicle user to know information data of an interest target in a visual field range in real time.
Description
Technical Field
The invention belongs to the field of automobile accessories, and particularly relates to a vehicle-mounted sight tracking device for an augmented reality technology.
Background
With the popularization and development of automobiles, people put forward higher requirements on automobiles, intellectualization becomes the direction of the current automobile development, and the popularization of the augmented reality technology also provides a new path for the intellectualized development of automobiles.
The augmented reality technology can obtain information of surrounding targets through video and internet technologies, the information is displayed on a specific display interface, and a user interacts with the surrounding real targets through the display interface.
The augmented reality technology is applied to the field of automobiles, so that automobile users can better interact with the surrounding environment of the automobiles, and brand new driving experience is brought to the automobile users.
Disclosure of Invention
The technical scheme of the invention can help the automobile user to know the information data of the interest target in the visual field range in real time.
The specific technical scheme of the invention is as follows: the utility model provides a vehicle-mounted sight tracking device based on augmented reality technique, is including the image acquisition device that is used for shooing the outside image of car, set up in the interactive display device of people car in car user the place ahead, set up sight tracking module and the control module in the car, and control module includes controller, data processor and data memory, and sight tracking module includes light source subassembly and image sensor. The sight tracking module tracks the sight of the automobile user and determines the gazing target of the automobile user according to the corresponding relation between the sight of the automobile user and the space position of the target outside the automobile; the sight tracking module captures sight features, the controller generates corresponding control instructions according to the sight features, the image acquisition device acquires external images of the automobile and extracts the image features according to the control instructions, the image features are processed by the data processor to form data information, the data information is matched with stored data in the data storage device, and the controller displays a visual program corresponding to the successfully matched stored data on the human-vehicle interaction display device.
As a further optimization of the invention, the sight characteristic of the vehicle user is that the user's sight gaze position resides in the same spatial range for a set period of time.
As a further optimization of the invention, the sight characteristic of the vehicle user is that the user's sight gaze position resides in the same spatial range within 2 seconds. When the set time period is less than 2 seconds, the noise identified by the device is excessive, and the misjudgment rate of the device is increased; when the set time period is more than 2 seconds, because the automobile is in a driving state, the condition that objects are separated from the visual field is increased, and further recognition failure is caused. The time period set to 2 seconds can be summarized by a large amount of experimental data to obtain a good experience.
As a further optimization of the present invention, the stored data in the data storage is street view data, and the vehicle user can know various information of the street where the vehicle user is located through the vehicle-mounted sight tracking device, wherein the information includes street traffic condition information, information of the businesses in the building, historical information of the building, and the like. The technical scheme can enable the automobile user to know various information at the side in real time without taking off the automobile.
As a further optimization of the invention, the vehicle-mounted sight tracking device based on the augmented reality technology is installed on an automatic driving automobile. Because the automatic driving automobile does not need the automobile user to watch the road condition and the road surface information, the automobile user can concentrate on attention to appreciate the interest target, and better experience can be obtained.
As a further optimization of the invention, the sight tracking module is arranged on glasses connected with the automobile control platform, and the connection mode of the glasses and the automobile control platform is wired or wireless connection. The scheme can realize the customized customization of the sight tracking module and improve the identification accuracy of the device.
As further optimization of the invention, the image acquisition device is arranged at the front part of the automobile, so that better image acquisition of a target object in the driving direction of the automobile can be realized.
As a further optimization of the invention, the sight tracking module tracks the sight gaze position of the automobile user, the sight characteristic is that the automobile user blinks continuously for more than two times within 1 second, the control instruction is that a control cursor appears on the human-vehicle interaction display device, and the automobile user controls the control cursor through the sight, so that the control of a visual program on the human-vehicle interaction display device is realized.
As a further optimization of the invention, the sight tracking module tracks the sight gaze position of the automobile user, the sight characteristic is that the automobile user blinks continuously for more than two times within 1 second, the control instruction is that a control cursor appears on the human-vehicle interaction display device, the automobile user controls the control cursor through the sight, and further controls a visual program on the human-vehicle interaction display device, and when the stored data is map data, the automobile user can select the driving direction or the driving destination through the vehicle-mounted sight tracking device.
The vehicle-mounted sight tracking device based on the augmented reality technology disclosed by the invention can help a vehicle user to know the information data of an interest target in a visual field range in real time.
Drawings
FIG. 1 is a block diagram of a vehicle-mounted visual tracking device based on augmented reality technology according to the present invention;
FIG. 2 is a schematic layout view of a vehicle-mounted sight tracking device based on augmented reality technology in an automobile according to the present invention;
fig. 3 is a block diagram of an operation flow of the vehicle-mounted gaze tracking apparatus based on the augmented reality technology according to the present invention.
Detailed Description
To explain the technical solution of the present invention in detail, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
A preferred embodiment of a vehicle-mounted sight line tracking device based on augmented reality technology, as shown in fig. 1 and fig. 2, comprises an image acquisition device 300 for taking an image of the outside of a vehicle, wherein the image acquisition device 300 is arranged at the front part of the vehicle; the interactive display device 400 is disposed in front of a user of a vehicle, the eye tracking module 100 and the control module 200 are disposed in the vehicle, the control module 200 includes a controller, a data processor and a data memory, and the eye tracking module 100 includes a light source assembly and an image sensor.
As shown in fig. 3, when an automobile user gazes at an object outside the automobile, the sight tracking module 100 tracks the sight of the automobile user, and determines the target gazed by the automobile user according to the corresponding relationship between the sight of the automobile user and the spatial position of the target outside the automobile, specifically, light emitted by a light source assembly of the sight tracking module 100 is reflected on the outer cornea of the eyes of the automobile user to form a purkinje spot, an image sensor captures the purkinje spot and determines the direction of the sight gazed by the automobile user according to the positions of the purkinje spot and the center of the pupil, so as to determine the target object gazed by the automobile user, and when the sight of the automobile user stays at the target object for more than 2 seconds, a controller sends a control instruction for acquiring an image of the target object.
As shown in fig. 3, the image acquisition device 300 acquires an external image of the automobile according to the control instruction and extracts image features, the image features are processed by the data processor to form data information, the data information is matched with the stored data in the data memory, and the controller displays a visualization program corresponding to the successfully matched stored data on the human-vehicle interaction display device 400. For example, the stored data in this embodiment is street view data, and the automobile user can know various information of the street through the man-car interactive display device 400, where the information includes street traffic condition information, information of the business in the building, and historical information of the building.
In another preferred embodiment of the present invention, as shown in fig. 1 and 2, an augmented reality technology-based on-vehicle gaze tracking device includes an image capturing device 300 for capturing an image of the outside of a vehicle, the image capturing device 300 being disposed at the front of the vehicle; the interactive display device 400 is disposed in front of a user of a vehicle, the eye tracking module 100 and the control module 200 are disposed in the vehicle, the control module 200 includes a controller, a data processor and a data memory, and the eye tracking module 100 includes a light source assembly and an image sensor.
As shown in fig. 3, when an automobile user gazes at an object outside the automobile, the sight tracking module 100 tracks the sight gaze position of the automobile user, specifically, light emitted by a light source assembly of the sight tracking module 100 is reflected on an outer cornea of an eye of the automobile user to form a purkinje spot, an image sensor captures the purkinje spot and determines the sight gaze direction of the automobile user according to the purkinje spot and the center of the pupil, so as to determine a target object gazed by the automobile user, and when the sight gaze of the automobile user stays at the target object for more than 2 seconds, a controller sends a control instruction for acquiring an image of the target object. The image acquisition device 300 acquires an external image of the automobile according to the control instruction and extracts image characteristics, the image characteristics are processed by the data processor to form data information, the data information is matched with the stored data in the data memory, and the controller displays the visual program corresponding to the successfully matched stored data on the human-vehicle interaction display device 400. The sight tracking module 100 tracks the sight gaze position of the automobile user, and when the automobile user blinks continuously for more than two times in 1 second, a control cursor appears on the human-vehicle interaction display device 400, the automobile user controls the control cursor through the sight, and then the control on the visual program on the human-vehicle interaction display device 400 is realized. For example, in the embodiment, the stored data is map data, the visualization program includes visualization options of a driving direction and a driving destination, and a vehicle user can select the driving direction or the driving destination by controlling a control cursor on the human-vehicle interaction display device 400.
In another preferred embodiment of the invention, the vehicle-mounted sight tracking device based on augmented reality technology is mounted on an autonomous vehicle.
In another preferred embodiment of the present invention, the eye tracking module is disposed on the glasses connected to the vehicle console, and the connection between the glasses and the vehicle console is a wired connection.
In another preferred embodiment of the present invention, the sight line tracking module is disposed on glasses connected to the vehicle console, and the glasses are connected to the vehicle console in a wireless manner.
While the present invention has been described in detail with reference to the preferred embodiments, it should be understood that the above description should not be taken as limiting the invention. Various modifications and alterations to this invention will become apparent to those skilled in the art upon reading the foregoing description. Accordingly, the scope of the invention should be determined from the following claims.
Claims (7)
1. An on-vehicle sight tracking device based on augmented reality technology includes:
the image acquisition device is used for shooting an image outside the automobile;
a man-vehicle interactive display device arranged in front of a vehicle user;
the system comprises a sight tracking module and a control module, wherein the sight tracking module and the control module are arranged in an automobile, the control module comprises a controller, a data processor and a data memory, and the sight tracking module comprises a light source assembly and an image sensor;
the sight tracking module tracks the sight of an automobile user, and determines that the automobile user watches a target according to the corresponding relation between the sight of the automobile user and the space position of the target outside the automobile;
the sight tracking module tracks the sight gaze position of an automobile user, captures sight characteristics which are more than two times of continuous blinking of the automobile user within 1 second, the controller generates a corresponding control instruction according to the sight characteristics, the control instruction is that a control cursor appears on the human-vehicle interaction display device, the image acquisition device acquires an image of a gaze target of the automobile user according to the control instruction and extracts image characteristics, the image characteristics are processed by the data processor to form data information, the data information is matched with stored data in the data memory, the controller displays a visual program corresponding to the successfully-matched stored data on the human-vehicle interaction display device, the automobile user controls the control cursor through sight, and further controls the visual program on the human-vehicle interaction display device, wherein the stored data are street view data, the automobile user knows various information of the block where the automobile user is located through the vehicle-mounted sight tracking device, wherein the information comprises street traffic condition information, information of merchants in the building and historical information of the building.
2. The vehicle-mounted visual line tracking device based on augmented reality technology according to claim 1, wherein the visual line characteristic is that the user visual line fixation position resides in the same spatial range within a set time period.
3. The augmented reality technology-based vehicle-mounted gaze tracking device of claim 2, wherein the set time period is 2 seconds.
4. The augmented reality technology-based vehicle-mounted visual line tracking device according to claim 1, wherein the augmented reality technology-based vehicle-mounted visual line tracking device is mounted on an autonomous automobile.
5. The vehicle-mounted sight tracking device based on the augmented reality technology according to claim 1, wherein the sight tracking module is arranged on glasses connected to the vehicle control platform, and the glasses are connected with the vehicle control platform in a wired or wireless manner.
6. The vehicle-mounted visual line tracking device based on augmented reality technology according to claim 1, wherein the image acquisition device is arranged at the front of the automobile.
7. The vehicle-mounted visual line tracking device based on augmented reality technology according to claim 1, wherein the stored data is map data, and a vehicle user can select a driving direction or a driving destination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710539990.XA CN107380064B (en) | 2017-07-04 | 2017-07-04 | Vehicle-mounted sight tracking device based on augmented reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710539990.XA CN107380064B (en) | 2017-07-04 | 2017-07-04 | Vehicle-mounted sight tracking device based on augmented reality technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107380064A CN107380064A (en) | 2017-11-24 |
| CN107380064B true CN107380064B (en) | 2022-08-19 |
Family
ID=60335071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710539990.XA Active CN107380064B (en) | 2017-07-04 | 2017-07-04 | Vehicle-mounted sight tracking device based on augmented reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107380064B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109191587B (en) * | 2018-08-23 | 2019-12-31 | 百度在线网络技术（北京）有限公司 | Color recognition method and device, electronic equipment and storage medium |
| CN109739352A (en) * | 2018-12-27 | 2019-05-10 | 斑马网络技术有限公司 | The methods of exhibiting and equipment of Land-scape picture |
| CN111951454B (en) * | 2020-10-16 | 2021-01-05 | 兰和科技（深圳）有限公司 | Fingerprint biological identification unlocking device of intelligent access control and judgment method thereof |
| CN112455327A (en) * | 2020-12-03 | 2021-03-09 | 上海汽车集团股份有限公司 | Vehicle lamp control method and device and computer readable storage medium |
| CN113492756A (en) * | 2021-07-01 | 2021-10-12 | 中汽创智科技有限公司 | Method, device, equipment and storage medium for displaying vehicle external information |
| EP4506786A1 (en) * | 2022-04-30 | 2025-02-12 | Shenzhen Yinwang Intelligent Technologies Co., Ltd. | Interaction method and electronic device |
| EP4421614A1 (en) * | 2023-02-27 | 2024-08-28 | Bayerische Motoren Werke Aktiengesellschaft | Data processing device and method for providing a video to a passenger of a vehicle |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0914174D0 (en) * | 2009-08-13 | 2009-09-16 | Light Blue Optics Ltd | Head up displays |
| CN103838378A (en) * | 2014-03-13 | 2014-06-04 | 广东石油化工学院 | Head wearing type eye control system based on pupil recognition positioning |
| CN104260669A (en) * | 2014-09-17 | 2015-01-07 | 北京理工大学 | Intelligent vehicle head-up display |
| CN205049814U (en) * | 2015-10-10 | 2016-02-24 | 青岛歌尔声学科技有限公司 | On -vehicle new line display terminal |
| CN105425967A (en) * | 2015-12-16 | 2016-03-23 | 中国科学院西安光学精密机械研究所 | Sight tracking and human eye region-of-interest positioning system |
| CN105739705A (en) * | 2016-02-04 | 2016-07-06 | 重庆邮电大学 | Human-eye control method and apparatus for vehicle-mounted system |
| CN105829938A (en) * | 2013-12-05 | 2016-08-03 | 罗伯特·博世有限公司 | Arrangement For Creating An Image Of A Scene |
| CN106155290A (en) * | 2014-11-06 | 2016-11-23 | 现代自动车株式会社 | Utilize the menu setecting equipment of Eye-controlling focus |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9164281B2 (en) * | 2013-03-15 | 2015-10-20 | Honda Motor Co., Ltd. | Volumetric heads-up display with dynamic focal plane |
-
2017
- 2017-07-04 CN CN201710539990.XA patent/CN107380064B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0914174D0 (en) * | 2009-08-13 | 2009-09-16 | Light Blue Optics Ltd | Head up displays |
| CN105829938A (en) * | 2013-12-05 | 2016-08-03 | 罗伯特·博世有限公司 | Arrangement For Creating An Image Of A Scene |
| CN103838378A (en) * | 2014-03-13 | 2014-06-04 | 广东石油化工学院 | Head wearing type eye control system based on pupil recognition positioning |
| CN104260669A (en) * | 2014-09-17 | 2015-01-07 | 北京理工大学 | Intelligent vehicle head-up display |
| CN106155290A (en) * | 2014-11-06 | 2016-11-23 | 现代自动车株式会社 | Utilize the menu setecting equipment of Eye-controlling focus |
| CN205049814U (en) * | 2015-10-10 | 2016-02-24 | 青岛歌尔声学科技有限公司 | On -vehicle new line display terminal |
| CN105425967A (en) * | 2015-12-16 | 2016-03-23 | 中国科学院西安光学精密机械研究所 | Sight tracking and human eye region-of-interest positioning system |
| CN105739705A (en) * | 2016-02-04 | 2016-07-06 | 重庆邮电大学 | Human-eye control method and apparatus for vehicle-mounted system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107380064A (en) | 2017-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107380064B (en) | Vehicle-mounted sight tracking device based on augmented reality technology | |
| US11194154B2 (en) | Onboard display control apparatus | |
| US10037699B1 (en) | Systems and methods for motivating a driver according to behaviors of nearby vehicles | |
| CN106218506B (en) | Vehicle display device and vehicle including the vehicle display device | |
| US9771083B2 (en) | Cognitive displays | |
| CA3127247A1 (en) | System and method for correlating user attention direction and outside view | |
| KR20190029488A (en) | Method, apparatus and computer readable storage medium having instructions for controlling a display of an augmented-reality-display-device for a motor vehicle | |
| CN109849788B (en) | Information providing method, device and system | |
| WO2019017215A1 (en) | Vehicle control device and vehicle control method | |
| US20180229654A1 (en) | Sensing application use while driving | |
| Langner et al. | Traffic awareness driver assistance based on stereovision, eye-tracking, and head-up display | |
| JP6891926B2 (en) | Vehicle systems, methods performed on vehicle systems, and driver assistance systems | |
| EP3654143A1 (en) | Method, apparatus and system for determining line of sight, and wearable eye movement device | |
| CN106062613A (en) | Control device enabled to look forward in hud system | |
| US10185392B2 (en) | Method for transmitting information to a driver of a motor vehicle, and adaptive driver assistance system | |
| CN105711511A (en) | Method for operating a head-up display, presentation apparatus, vehicle | |
| JP2019091255A (en) | Information processing apparatus, driver monitoring system, information processing method, and information processing program | |
| WO2020208804A1 (en) | Display control device, display control method, and display control program | |
| JP2005228139A (en) | Display device for vehicle | |
| JP2021162542A (en) | Driving support device and data collection system | |
| US9495871B2 (en) | Display control device, display control method, non-transitory recording medium, and projection device | |
| CN112954486B (en) | Vehicle-mounted video trace processing method based on sight attention | |
| CN107472140B (en) | Eye-control vehicle-mounted HUD system | |
| CN111263133B (en) | Information processing method and system | |
| US20250050054A1 (en) | Method, apparatus and computer program product for selecting content for display during a journey to alleviate motion sickness |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |