CN110363555A - Recommended method and device based on eye tracking vision algorithm - Google Patents
Recommended method and device based on eye tracking vision algorithm Download PDFInfo
- Publication number
- CN110363555A CN110363555A CN201810316237.9A CN201810316237A CN110363555A CN 110363555 A CN110363555 A CN 110363555A CN 201810316237 A CN201810316237 A CN 201810316237A CN 110363555 A CN110363555 A CN 110363555A
- Authority
- CN
- China
- Prior art keywords
- eye
- eyegaze
- tested user
- commodity
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 24
- 239000013065 commercial product Substances 0.000 claims abstract description 24
- 210000001508 eye Anatomy 0.000 claims description 193
- 238000004590 computer program Methods 0.000 claims description 23
- 239000000047 product Substances 0.000 claims description 14
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000005286 illumination Methods 0.000 claims description 7
- 230000007613 environmental effect Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 description 20
- 210000003128 head Anatomy 0.000 description 8
- 210000005252 bulbus oculi Anatomy 0.000 description 7
- 230000001815 facial effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 2
- 230000004424 eye movement Effects 0.000 description 2
- 210000000720 eyelash Anatomy 0.000 description 2
- 210000000744 eyelid Anatomy 0.000 description 2
- 230000008921 facial expression Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0241—Advertisements
- G06Q30/0251—Targeted advertisements
- G06Q30/0268—Targeted advertisements at point-of-sale [POS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Accounting & Taxation (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Multimedia (AREA)
- Eye Examination Apparatus (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
This application involves a kind of recommended method based on eye tracking vision algorithm, device, computer equipment and storage mediums.This method comprises: acquiring the eye motion data of tested user；Eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze direction of tested user；According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.Eye tracing model based on preparatory foundation, the eye motion data of current tested user are handled, obtain the Eyegaze direction of tested user, and the interested commodity of user are determined according to the gaze-direction of user, to carry out commercial product recommending, it allows users to the commodity based on recommendation and quickly chooses required commodity, and then improve shopping efficiency.
Description
Technical field
This application involves technical field of computer vision, more particularly to a kind of recommendation based on eye tracking vision algorithm
Method, apparatus, computer equipment and storage medium.
Background technique
With social progress and technology and high speed development, traditional retail purchases mode is also occurring earthshaking
Variation, there is vending machine, i.e., commodity be placed in vending machine, while being placed on vending machine and screen is set so that with
Family can obtain some merchandise newss or be inquired by screen.
However this retail purchases mode still has some defects, user still needs recommendation or master by salesman
It moves and checks all commodity in screen to carry out choosing for commodity, there are problems that inefficiency of doing shopping.
Summary of the invention
Based on this, it is necessary in view of the above technical problems, provide it is a kind of can be improved shopping efficiency based on eye tracking
Recommended method, device, computer equipment and the storage medium of vision algorithm.
A kind of recommended method based on eye tracking vision algorithm, which comprises
Acquire the eye motion data of tested user；
The eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze side of tested user
To；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
The basis is tested the Eyegaze direction of user in one of the embodiments, carries out commodity to tested user
Recommend, comprising:
According to the Eyegaze direction of tested user, the end article in the Eyegaze direction is determined；
Based on the end article, Recommendations are obtained；
Show the Recommendations.
It is described in one of the embodiments, to be based on the end article, obtain Recommendations, comprising:
Obtain the feature of the end article；
Commodity similar in the determining feature with the end article；
According to commodity similar in the end article and feature, Recommendations are obtained.
The eye motion data include: to acquire the position of the video camera of eye motion image in one of the embodiments,
It sets, gaze vector, eyes image and lighting environment parameter；
Described that the eye motion data are inputted to the eye tracing model pre-established, the eye for obtaining tested user is solidifying
Apparent direction, comprising:
According to the eyes image, the configuration parameter of eye tracing model corresponding with eye shape is determined；
According to the position of the configuration parameter and the video camera, gaze vector and lighting environment parameter, the eye is utilized
Portion's tracing model obtains the Eyegaze direction of tested user.
A kind of recommendation apparatus based on eye tracking vision algorithm, which is characterized in that described device includes:
Data acquisition module, for acquiring the eye motion data of tested user；
Direction of visual lines determining module is obtained for the eye motion data to be inputted the eye tracing model pre-established
To the Eyegaze direction of tested user；
Commercial product recommending module, for carrying out commercial product recommending to tested user according to the Eyegaze direction for being tested user.
The commercial product recommending module includes: in one of the embodiments,
End article determining module, for determining the Eyegaze direction according to the Eyegaze direction for being tested user
End article；
Recommendations determining module obtains Recommendations for being based on the end article；
Display module, for showing the Recommendations.
The Recommendations determining module includes: in one of the embodiments,
Product features obtain module, for obtaining the feature of the end article；
Close commodity determining module, for commodity similar in the determining feature with the end article；
Recommendations determine submodule, are used for the commodity according to similar in the end article and feature, obtain Recommendations.
The eye motion data include: to acquire the position of the video camera of eye motion image in one of the embodiments,
It sets, gaze vector, eyes image and lighting environment parameter；The direction of visual lines determining module includes:
Configuration parameter determining module, for determining that eye corresponding with eye shape tracks mould according to the eyes image
The configuration parameter of type；
Direction of visual lines determines submodule, for according to the position of the configuration parameter and the video camera, gaze vector and
Lighting environment parameter obtains the Eyegaze direction of tested user using the eye tracing model.
A kind of computer equipment, including memory and processor, the memory are stored with computer program, the processing
Device performs the steps of when executing the computer program
Acquire the eye motion data of tested user；
The eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze side of tested user
To；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
A kind of computer readable storage medium, is stored thereon with computer program, and the computer program is held by processor
It is performed the steps of when row
Acquire the eye motion data of tested user；
The eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze side of tested user
To；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
The above-mentioned recommended method based on eye tracking vision algorithm, device, computer equipment and storage medium, based on preparatory
Foundation eye tracing model, the eye motion data of current tested user are handled, the eye of tested user is obtained
Gaze-direction, and determine that the interested commodity of user are allowed users to carry out commercial product recommending according to the gaze-direction of user
Commodity based on recommendation quickly choose required commodity, and then improve shopping efficiency.
Detailed description of the invention
Fig. 1 is the applied environment figure of the recommended method based on eye tracking vision algorithm in one embodiment；
Fig. 2 is the flow diagram of the recommended method based on eye tracking vision algorithm in one embodiment；
Fig. 3 is the flow diagram that Recommendations step is obtained in one embodiment；
Fig. 4 is the flow diagram that Recommendations step is obtained in another embodiment；
Fig. 5 is the structural block diagram of the recommendation apparatus based on eye tracking vision algorithm in one embodiment；
Fig. 6 is the internal structure chart of computer equipment in one embodiment.
Specific embodiment
It is with reference to the accompanying drawings and embodiments, right in order to which the objects, technical solutions and advantages of the application are more clearly understood
The application is further elaborated.It should be appreciated that specific embodiment described herein is only used to explain the application, not
For limiting the application.
Recommended method provided by the present application based on eye tracking vision algorithm, can be applied to application as shown in Figure 1
In environment.Wherein, sales counter 102 includes display screen 104 and product display zone 106.Display screen 104 includes Recommendations region,
For showing the commodity sign of determining commodity to be recommended.Product display zone 106 is the region that display has all commodity, specifically
Ground, product display zone 106 can be the practical display position region of commodity, be also possible to all commodity shown by display screen
Display area, be not limited thereto.
In one embodiment, as shown in Fig. 2, providing a kind of recommended method based on eye tracking vision algorithm, with
This method is applied to be illustrated for the sales counter in Fig. 1, comprising the following steps:
S202 acquires the eye motion data of tested user.
Wherein, when eye motion data refer to commodity of the user in concern sales counter, eye motion and local environment
The relevant parameter of generation.For example, eye motion data may include: the position for acquiring the video camera of eye motion image, stare
One of vector, eyes image and lighting environment parameter are a variety of.Gaze vector, eyes image and lighting environment parameter
It is obtained according to the data of corresponding acquisition device acquisition.For example, the eye that gaze vector and eyes image can be acquired according to video camera
Portion's moving image obtains, and lighting environment parameter can be obtained according to illumination acquisition device.
Eye motion data are inputted the eye tracing model pre-established by S204, obtain the Eyegaze of tested user
Direction.
In the present embodiment, it is pre-established with eye tracing model, eye motion data are input to corresponding eye and are chased after
Track model, eye tracing model export to obtain Eyegaze direction.
Wherein, eye tracing model can be trained to obtain based on artificial eye training data.Firstly, passing through simulation eye
The training data in kinematic configuration eye motion data and Eyegaze direction establishes the eye model being made of eyeball.To tool
There is the head model of different facial expressions to be scanned, the eyeball grid scanned is extractd, and by established eye model
It is placed in the head model for having extractd eyeball grid, based in the eye motion data control head model in training data
Eye movement, to simulate eye gaze-direction.Further, since face deformation can also cause certain journey to Eyegaze direction
Therefore the influence of degree further creates facial deformation training data and illumination training data, facial deformation training data can be with
Including eyelid movement data and eyelashes exercise data, based on facial deformation training data, illumination training data, eye motion data
And the training data in Eyegaze direction is trained head model, based on the head model that training obtains, finally obtains
The eye tracing model and its configuration parameter of different eye shapes.
S206 carries out commercial product recommending to tested user according to the Eyegaze direction of tested user.
Eyegaze direction based on tested user, obtains user's commodity of interest, so according to the commodity of concern into
Row commercial product recommending.
The above-mentioned recommended method based on eye tracking vision algorithm, the eye tracing model based on preparatory foundation, to working as
The eye motion data of preceding tested user are handled, and obtain the Eyegaze direction of tested user, and staring according to user
Direction determines the interested commodity of user, to carry out commercial product recommending, allows users to the commodity based on recommendation and quickly chooses institute
The commodity needed, and then improve shopping efficiency.
In one embodiment, as shown in figure 3, according to the Eyegaze direction of tested user, commodity are carried out to tested user
Recommend, comprising:
S302 determines the end article in Eyegaze direction according to the Eyegaze direction of tested user.
End article refers to the commodity that tested user currently pays close attention to.The commodity that can be chosen there are many being displayed in sales counter,
In the present embodiment, the commodity that shopper displays in browsing sales counter determine current note according to the Eyegaze direction of shopper
Depending on commodity as end article.
Specifically, the position that crosses of tested user's sight and sales counter, knot can be obtained according to the Eyegaze direction of tracking
Close the commodity for the determining tested user's concern in position and commodity display position that crosses.
Sales counter is previously stored with the display position information of each commodity, is obtaining tested user's view based on Eyegaze direction
Behind the position that crosses of line and sales counter, the position that will will cross is compared with the display position information of commodity each in sales counter, sentences
Breaking off a friendship, it is corresponding with the wherein display position information of which commodity to converge position, according to corresponding display position information you can learn that this is old
The commodity of the corresponding display of column position information, namely the corresponding commodity in the position that crosses, and then the corresponding commodity conduct in the position that will cross
End article.
S304 is based on end article, obtains Recommendations.
Based on end article obtained, that is, it can determine the interested type of merchandise of tested user, and then according to tested use
The interested type of merchandise in family carries out the recommendation of commodity, obtains Recommendations.
S306 shows Recommendations.
Recommendations are shown by display screen, are quickly selected to be tested user according to the Recommendations of display
Purchase.
In one embodiment, as shown in figure 4, being based on end article, Recommendations are obtained, comprising:
S402 obtains the feature of end article.
End article is characterized in the attributive character for referring to that end article has, it may include the type of merchandise, Brand, quotient
Product effect etc. is one such or a variety of.
In the present embodiment, sales counter is stored with the product features of all display goods, when according to the eye for being tested user
When gaze-direction determines end article, store in sale case, the corresponding product features of the end article can be obtained.Wherein,
The feature of each commodity can be obtained and be stored in by the scanning means items scanning of sales counter and sold goods when carrying out commodity display
In the storage equipment of cabinet, it can also be written by way of being manually entered in the storage equipment of sales counter.
Commodity similar in the feature of S404, determination and end article.
Specifically, obtain the feature of other commodity in addition to end article itself in sales counter, and respectively with target quotient
The features of product carries out similarity calculation, obtains the similarity of other commodity and end article, and according to similarity and preset
Close commodity determine rule, commodity similar in the determining feature with end article.Wherein, preset close commodity determine that rule can
To be that similarity is greater than preset value or similarity ranking in preceding default ranking.
S406 obtains Recommendations according to commodity similar in end article and feature.
Using commodity similar in identified feature and end article as Recommendations, recommend to tested user.
In one embodiment, eye motion data include: the position for acquiring the video camera of eye motion image, stare arrow
Amount, eyes image and lighting environment parameter.Eye motion data are inputted to the eye tracing model pre-established, obtain tested use
The Eyegaze direction at family, comprising: according to eyes image, determine the configuration ginseng of eye tracing model corresponding with eye shape
Number；According to the position of configuration parameter and video camera, gaze vector and lighting environment parameter, using eye tracing model, obtain by
Survey the Eyegaze direction of user.
Wherein, video camera may include one or more.Based on different camera positions, can obtain different
Gaze vector and eyes image, and by being analyzed and processed to multi-group data, available more accurate Eyegaze side
To.Specifically, gaze vector may include the pitch and yaw angle of eyeball.
In the present embodiment, the eye tracing model for there are different configuration parameters is established previously according to different eye shapes,
The eye shape of tested user is obtained according to eyes image, and the eye shape of tested user is tracked with the eye pre-established
The corresponding eye shape of model configuration parameter is compared, and obtains and the most similar eye tracing model of tested user's eye shape
Configuration parameter can be obtained and carry out Eyegaze direction prediction based on the configuration parameter of identified eye tracing model
Eye tracing model, and then the position of video camera, gaze vector and lighting environment parameter are input to the eye of configured parameter
In tracing model, the Eyegaze direction of tested user is obtained.
The above-mentioned recommended method based on eye tracking vision algorithm, by by the eye shape of tested user with pre-establish
The corresponding eye shape of eye tracing model configuration parameter be compared, obtain and the tested most similar eye of user's eye shape
The configuration parameter of portion's tracing model utilizes eye according to the position of configuration parameter and video camera, gaze vector and lighting environment parameter
Portion's tracing model obtains the Eyegaze direction of tested user, and then determines that user is interested according to the gaze-direction of user
Commodity are recommended, and are shown the information of Recommendations by screen, information of the user based on Recommendations can be quick
Carry out choosing for commodity.
It should be understood that although each step in the flow chart of Fig. 2-4 is successively shown according to the instruction of arrow,
These steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly stating otherwise herein, these steps
Execution there is no stringent sequences to limit, these steps can execute in other order.Moreover, at least one in Fig. 2-4
Part steps may include that perhaps these sub-steps of multiple stages or stage are not necessarily in synchronization to multiple sub-steps
Completion is executed, but can be executed at different times, the execution sequence in these sub-steps or stage is also not necessarily successively
It carries out, but can be at least part of the sub-step or stage of other steps or other steps in turn or alternately
It executes.
In one embodiment, as shown in figure 5, providing a kind of recommendation apparatus based on eye tracking vision algorithm, packet
It includes: data acquisition module 502, direction of visual lines determining module 504 and commercial product recommending module 506, in which:
Data acquisition module 502, for acquiring the eye motion data of tested user.
Wherein, when eye motion data refer to commodity of the user in concern sales counter, eye motion and local environment
The relevant parameter of generation.
Direction of visual lines determining module 504 is obtained for eye motion data to be inputted the eye tracing model pre-established
The Eyegaze direction of tested user.
In the present embodiment, it is pre-established with eye tracing model, eye motion data are input to corresponding eye and are chased after
Track model, eye tracing model export to obtain Eyegaze direction.
Wherein, eye tracing model can be trained to obtain based on artificial eye training data.Firstly, passing through simulation eye
The training data in kinematic configuration eye motion data and Eyegaze direction establishes the eye model being made of eyeball.To tool
There is the head model of different facial expressions to be scanned, the eyeball grid scanned is extractd, and by established eye model
It is placed in the head model for having extractd eyeball grid, based in the eye motion data control head model in training data
Eye movement, to simulate eye gaze-direction.Further, since face deformation can also cause certain journey to Eyegaze direction
Therefore the influence of degree further creates facial deformation training data and illumination training data, facial deformation training data can be with
Including eyelid movement data and eyelashes exercise data, based on facial deformation training data, illumination training data, eye motion data
And the training data in Eyegaze direction is trained head model, based on the head model that training obtains, finally obtains
The eye tracing model and its configuration parameter of different eye shapes.
Commercial product recommending module 506, for carrying out commodity to tested user and pushing away according to the Eyegaze direction for being tested user
It recommends.
Eyegaze direction based on tested user, obtains user's commodity of interest, so according to the commodity of concern into
Row commercial product recommending.
The above-mentioned recommendation apparatus based on eye tracking vision algorithm, the eye tracing model based on preparatory foundation, to working as
The eye motion data of preceding tested user are handled, and obtain the Eyegaze direction of tested user, and staring according to user
Direction determines the interested commodity of user, to carry out commercial product recommending, allows users to the commodity based on recommendation and quickly chooses institute
The commodity needed, and then improve shopping efficiency.
In one embodiment, commercial product recommending module includes: end article determining module, Recommendations determining module and displaying
Module.
End article determining module, for determining the mesh in Eyegaze direction according to the Eyegaze direction for being tested user
Mark commodity.
Specifically, the position that crosses of tested user's sight and sales counter, knot can be obtained according to the Eyegaze direction of tracking
Close the commodity for the determining tested user's concern in position and commodity display position that crosses.
Sales counter is previously stored with the display position information of each commodity, is obtaining tested user's view based on Eyegaze direction
Behind the position that crosses of line and sales counter, the position that will will cross is compared with the display position information of commodity each in sales counter, sentences
Breaking off a friendship, it is corresponding with the wherein display position information of which commodity to converge position, according to corresponding display position information you can learn that this is old
The commodity of the corresponding display of column position information, namely the corresponding commodity in the position that crosses, and then the corresponding commodity conduct in the position that will cross
End article.
Recommendations determining module obtains Recommendations for being based on end article.Based on end article obtained,
It can determine the interested type of merchandise of tested user, and then pushing away for commodity carried out according to the interested type of merchandise of tested user
It recommends, obtains Recommendations.
Display module, for showing Recommendations.Recommendations are shown by display screen, to be tested user's root
It is quickly chosen according to the Recommendations of display.
In one embodiment, Recommendations determining module include: product features obtain module, close commodity determining module and
Recommendations determine submodule.
Product features obtain module, for obtaining the feature of end article.In the present embodiment, sales counter is stored with all
The product features of display goods can obtain sale case when determining end article according to the Eyegaze direction for being tested user
Middle storage, the corresponding product features of the end article.
Close commodity determining module, for commodity similar in the determining feature with end article.
Specifically, obtain the feature of other commodity in addition to end article itself in sales counter, and respectively with target quotient
The features of product carries out similarity calculation, obtains the similarity of other commodity and end article, and according to similarity and preset
Close commodity determine rule, commodity similar in the determining feature with end article.Wherein, preset close commodity determine that rule can
To be that similarity is greater than preset value or similarity ranking in preceding default ranking.
Recommendations determine submodule, are used for the commodity according to similar in end article and feature, obtain Recommendations.Specifically
To recommend to tested user using commodity similar in identified feature and end article as Recommendations.
In one embodiment, eye motion data include: to acquire the position of the video camera of eye motion image, stare arrow
Amount, eyes image and lighting environment parameter；Direction of visual lines determining module includes: that configuration parameter determining module and direction of visual lines determine
Submodule.Wherein, configuration parameter determining module, for determining that eye corresponding with eye shape tracks mould according to eyes image
The configuration parameter of type.Direction of visual lines determines submodule, for according to the position of configuration parameter and video camera, gaze vector and illumination
Environmental parameter obtains the Eyegaze direction of tested user using eye tracing model.
In the present embodiment, the eye tracing model for there are different configuration parameters is established previously according to different eye shapes,
The eye shape of tested user is obtained according to eyes image, and the eye shape of tested user is tracked with the eye pre-established
The corresponding eye shape of model configuration parameter is compared, and obtains and the most similar eye tracing model of tested user's eye shape
Configuration parameter can be obtained and carry out Eyegaze direction prediction based on the configuration parameter of identified eye tracing model
Eye tracing model, and then the position of video camera, gaze vector and lighting environment parameter are input to the eye of configured parameter
In tracing model, the Eyegaze direction of tested user is obtained.
The above-mentioned recommendation apparatus based on eye tracking vision algorithm, by by the eye shape of tested user with pre-establish
The corresponding eye shape of eye tracing model configuration parameter be compared, obtain and the tested most similar eye of user's eye shape
The configuration parameter of portion's tracing model utilizes eye according to the position of configuration parameter and video camera, gaze vector and lighting environment parameter
Portion's tracing model obtains the Eyegaze direction of tested user, and then determines that user is interested according to the gaze-direction of user
Commodity are recommended, and are shown the information of Recommendations by screen, information of the user based on Recommendations can be quick
Carry out choosing for commodity.
About the recommendation apparatus based on eye tracking vision algorithm it is specific restriction may refer to above for sight with
The restriction of the recommended method of track vision algorithm, details are not described herein.It is each in the recommendation apparatus of above-mentioned eye tracking vision algorithm
A module can be realized fully or partially through software, hardware and combinations thereof.Above-mentioned each module can be embedded in the form of hardware or
Independently of in the processor in computer equipment, can also be stored in a software form in the memory in computer equipment, with
It is called convenient for processor and executes the corresponding operation of the above modules.
In one embodiment, a kind of computer equipment is provided, which can be terminal, for example sell goods
Cabinet, internal structure chart can be as shown in Figure 6.The computer equipment include by system bus connect processor, memory,
Network interface, display screen and input unit.Wherein, the processor of the computer equipment is for providing calculating and control ability.It should
The memory of computer equipment includes non-volatile memory medium, built-in storage.The non-volatile memory medium is stored with operation
System and computer program.The built-in storage is that the operation of the operating system and computer program in non-volatile memory medium mentions
For environment.The network interface of the computer equipment is used to communicate with external terminal by network connection.The computer program quilt
A kind of recommended method of eye tracking vision algorithm is realized when processor executes.The display screen of the computer equipment can be liquid
Crystal display screen or electric ink display screen, the input unit of the computer equipment can be the touch layer covered on display screen,
Be also possible to the key being arranged on computer equipment shell, trace ball or Trackpad, can also be external keyboard, Trackpad or
Mouse etc..
It will be understood by those skilled in the art that structure shown in Fig. 6, only part relevant to application scheme is tied
The block diagram of structure does not constitute the restriction for the computer equipment being applied thereon to application scheme, specific computer equipment
It may include perhaps combining certain components or with different component layouts than more or fewer components as shown in the figure.
In one embodiment, a kind of computer equipment, including memory and processor are provided, is stored in memory
Computer program, the processor perform the steps of when executing computer program
Acquire the eye motion data of tested user；
Eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze direction of tested user；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
In one embodiment, it is also performed the steps of when processor executes computer program
According to the Eyegaze direction of tested user, the end article in Eyegaze direction is determined；
Based on end article, Recommendations are obtained；
Show Recommendations.
In one embodiment, it is also performed the steps of when processor executes computer program
Obtain the feature of end article；
Commodity similar in the determining feature with end article；
According to commodity similar in end article and feature, Recommendations are obtained.
In one embodiment, it is also performed the steps of when processor executes computer program
According to eyes image, the configuration parameter of eye tracing model corresponding with eye shape is determined；
It is obtained according to the position of configuration parameter and video camera, gaze vector and lighting environment parameter using eye tracing model
To the Eyegaze direction of tested user.
In one embodiment, a kind of computer readable storage medium is provided, computer program is stored thereon with, is calculated
Machine program performs the steps of when being executed by processor
Acquire the eye motion data of tested user；
Eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze direction of tested user；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
In one embodiment, it is also performed the steps of when computer program is executed by processor
According to the Eyegaze direction of tested user, the end article in Eyegaze direction is determined；
Based on end article, Recommendations are obtained；
Show Recommendations.
In one embodiment, it is also performed the steps of when computer program is executed by processor
Obtain the feature of end article；
Commodity similar in the determining feature with end article；
According to commodity similar in end article and feature, Recommendations are obtained.
In one embodiment, it is also performed the steps of when computer program is executed by processor
According to eyes image, the configuration parameter of eye tracing model corresponding with eye shape is determined；
It is obtained according to the position of configuration parameter and video camera, gaze vector and lighting environment parameter using eye tracing model
To the Eyegaze direction of tested user.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Instruct relevant hardware to complete by computer program, computer program to can be stored in a non-volatile computer readable
It takes in storage medium, the computer program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, this Shen
Please provided by any reference used in each embodiment to memory, storage, database or other media, may each comprise
Non-volatile and/or volatile memory.Nonvolatile memory may include read-only memory (ROM), programming ROM
(PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include
Random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms,
Such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhancing
Type SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM
(RDRAM), direct memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment
In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance
Shield all should be considered as described in this specification.
Above embodiments only express the several embodiments of the application, and the description thereof is more specific and detailed, but can not
Therefore it is construed as limiting the scope of the patent.It should be pointed out that for those of ordinary skill in the art,
Under the premise of not departing from the application design, various modifications and improvements can be made, these belong to the protection scope of the application.
Therefore, the scope of protection shall be subject to the appended claims for the application patent.
Claims (10)
1. a kind of recommended method based on eye tracking vision algorithm, which comprises
Acquire the eye motion data of tested user；
The eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze direction of tested user；
According to the Eyegaze direction of tested user, commercial product recommending is carried out to tested user.
2. the method according to claim 1, wherein the basis is tested the Eyegaze direction of user, to quilt
It surveys user and carries out commercial product recommending, comprising:
According to the Eyegaze direction of tested user, the end article in the Eyegaze direction is determined；
Based on the end article, Recommendations are obtained；
Show the Recommendations.
3. according to the method described in claim 2, it is characterized in that, it is described be based on the end article, obtain Recommendations, wrap
It includes:
Obtain the feature of the end article；
Commodity similar in the determining feature with the end article；
According to commodity similar in the end article and feature, Recommendations are obtained.
4. the method according to claim 1, wherein the eye motion data include: acquisition eye motion figure
Position, gaze vector, eyes image and the lighting environment parameter of the video camera of picture；
It is described that the eye motion data are inputted to the eye tracing model pre-established, obtain the Eyegaze side of tested user
To, comprising:
According to the eyes image, the configuration parameter of eye tracing model corresponding with eye shape is determined；
According to the position of the configuration parameter and the video camera, gaze vector and lighting environment parameter, chased after using the eye
Track model obtains the Eyegaze direction of tested user.
5. a kind of recommendation apparatus based on eye tracking vision algorithm, described device include:
Data acquisition module, for acquiring the eye motion data of tested user；
Direction of visual lines determining module, for the eye motion data to be inputted the eye tracing model that pre-establishes, obtain by
Survey the Eyegaze direction of user；
Commercial product recommending module, for carrying out commercial product recommending to tested user according to the Eyegaze direction for being tested user.
6. device according to claim 5, which is characterized in that the commercial product recommending module includes:
End article determining module, for determining the mesh in the Eyegaze direction according to the Eyegaze direction for being tested user
Mark commodity；
Recommendations determining module obtains Recommendations for being based on the end article；
Display module, for showing the Recommendations.
7. according to the method described in claim 6, it is characterized in that, the Recommendations determining module includes:
Product features obtain module, for obtaining the feature of the end article；
Close commodity determining module, for commodity similar in the determining feature with the end article；
Recommendations determine submodule, are used for the commodity according to similar in the end article and feature, obtain Recommendations.
8. according to the method described in claim 5, it is characterized in that, the eye motion data include: acquisition eye motion figure
The position of the video camera of picture, gaze vector, eyes image and lighting environment parameter；The direction of visual lines determining module includes:
Configuration parameter determining module, for determining eye tracing model corresponding with eye shape according to the eyes image
Configuration parameter；
Direction of visual lines determines submodule, for according to the position of the configuration parameter and the video camera, gaze vector and illumination
Environmental parameter obtains the Eyegaze direction of tested user using the eye tracing model.
9. a kind of computer equipment, including memory and processor, the memory are stored with computer program, feature exists
In the step of processor realizes any one of claims 1 to 4 the method when executing the computer program.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
The step of method described in any one of Claims 1-4 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810316237.9A CN110363555B (en) | 2018-04-10 | 2018-04-10 | Recommendation method and device based on vision tracking visual algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810316237.9A CN110363555B (en) | 2018-04-10 | 2018-04-10 | Recommendation method and device based on vision tracking visual algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110363555A true CN110363555A (en) | 2019-10-22 |
| CN110363555B CN110363555B (en) | 2024-04-09 |
Family
ID=68213996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810316237.9A Active CN110363555B (en) | 2018-04-10 | 2018-04-10 | Recommendation method and device based on vision tracking visual algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110363555B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111105267A (en) * | 2019-11-22 | 2020-05-05 | 苏州浪潮智能科技有限公司 | Recommendation method and device based on ALS algorithm and readable medium |
| CN111400605A (en) * | 2020-04-26 | 2020-07-10 | Oppo广东移动通信有限公司 | Recommendation method and device based on eyeball tracking |
| CN113015998A (en) * | 2019-10-23 | 2021-06-22 | 株式会社视信 | Sight line analysis device and sight line analysis system and method using same |
| CN113129112A (en) * | 2021-05-11 | 2021-07-16 | 杭州海康威视数字技术股份有限公司 | Article recommendation method and device and electronic equipment |
| CN114255106A (en) * | 2021-12-21 | 2022-03-29 | 深圳康佳电子科技有限公司 | Activity information pushing method and system, mobile terminal and readable storage medium |
| WO2023010301A1 (en) * | 2021-08-04 | 2023-02-09 | 华为技术有限公司 | Line-of-sight detection method and apparatus, and eyeball model modeling method and apparatus |
| TWI802881B (en) * | 2020-05-12 | 2023-05-21 | 普安科技股份有限公司 | System and method for visitor interest extent analysis |
| CN116932861A (en) * | 2022-03-30 | 2023-10-24 | 博泰车联网(南京)有限公司 | Information recommendation method and device and vehicle |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| CN102830793A (en) * | 2011-06-16 | 2012-12-19 | 北京三星通信技术研究有限公司 | Sight tracking method and sight tracking device |
| CN103645806A (en) * | 2013-12-24 | 2014-03-19 | 惠州Tcl移动通信有限公司 | Commodity browse method and system based on eyeball tracking |
| CN104809424A (en) * | 2014-01-23 | 2015-07-29 | 北京七鑫易维信息技术有限公司 | Method for realizing sight tracking based on iris features |
| US20160139665A1 (en) * | 2014-11-14 | 2016-05-19 | The Eye Tribe Aps | Dynamic eye tracking calibration |
| CN106774863A (en) * | 2016-12-03 | 2017-05-31 | 西安中科创星科技孵化器有限公司 | A kind of method that Eye-controlling focus are realized based on pupil feature |
| CN107133854A (en) * | 2017-05-26 | 2017-09-05 | 北京小度信息科技有限公司 | Information recommendation method and device |
| CN107154109A (en) * | 2017-06-02 | 2017-09-12 | 深圳正品创想科技有限公司 | A kind of commodity rendering method, device and self-service cabinet |
| CN107247759A (en) * | 2017-05-31 | 2017-10-13 | 深圳正品创想科技有限公司 | A kind of Method of Commodity Recommendation and device |
| CN107533634A (en) * | 2015-03-23 | 2018-01-02 | 控制辐射系统有限公司 | eye tracking system |
-
2018
- 2018-04-10 CN CN201810316237.9A patent/CN110363555B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102830793A (en) * | 2011-06-16 | 2012-12-19 | 北京三星通信技术研究有限公司 | Sight tracking method and sight tracking device |
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| CN103645806A (en) * | 2013-12-24 | 2014-03-19 | 惠州Tcl移动通信有限公司 | Commodity browse method and system based on eyeball tracking |
| CN104809424A (en) * | 2014-01-23 | 2015-07-29 | 北京七鑫易维信息技术有限公司 | Method for realizing sight tracking based on iris features |
| US20160139665A1 (en) * | 2014-11-14 | 2016-05-19 | The Eye Tribe Aps | Dynamic eye tracking calibration |
| CN107533634A (en) * | 2015-03-23 | 2018-01-02 | 控制辐射系统有限公司 | eye tracking system |
| CN106774863A (en) * | 2016-12-03 | 2017-05-31 | 西安中科创星科技孵化器有限公司 | A kind of method that Eye-controlling focus are realized based on pupil feature |
| CN107133854A (en) * | 2017-05-26 | 2017-09-05 | 北京小度信息科技有限公司 | Information recommendation method and device |
| CN107247759A (en) * | 2017-05-31 | 2017-10-13 | 深圳正品创想科技有限公司 | A kind of Method of Commodity Recommendation and device |
| CN107154109A (en) * | 2017-06-02 | 2017-09-12 | 深圳正品创想科技有限公司 | A kind of commodity rendering method, device and self-service cabinet |
Non-Patent Citations (2)
| Title |
|---|
| 苌道方等: "考虑行为和眼动跟踪的用户兴趣模型", 《河南科技大学学报(自然科学版)》 * |
| 苌道方等: "考虑行为和眼动跟踪的用户兴趣模型", 《河南科技大学学报(自然科学版)》, no. 01, 25 February 2014 (2014-02-25) * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113015998A (en) * | 2019-10-23 | 2021-06-22 | 株式会社视信 | Sight line analysis device and sight line analysis system and method using same |
| CN111105267A (en) * | 2019-11-22 | 2020-05-05 | 苏州浪潮智能科技有限公司 | Recommendation method and device based on ALS algorithm and readable medium |
| CN111400605A (en) * | 2020-04-26 | 2020-07-10 | Oppo广东移动通信有限公司 | Recommendation method and device based on eyeball tracking |
| TWI802881B (en) * | 2020-05-12 | 2023-05-21 | 普安科技股份有限公司 | System and method for visitor interest extent analysis |
| CN113129112A (en) * | 2021-05-11 | 2021-07-16 | 杭州海康威视数字技术股份有限公司 | Article recommendation method and device and electronic equipment |
| WO2023010301A1 (en) * | 2021-08-04 | 2023-02-09 | 华为技术有限公司 | Line-of-sight detection method and apparatus, and eyeball model modeling method and apparatus |
| CN114255106A (en) * | 2021-12-21 | 2022-03-29 | 深圳康佳电子科技有限公司 | Activity information pushing method and system, mobile terminal and readable storage medium |
| CN116932861A (en) * | 2022-03-30 | 2023-10-24 | 博泰车联网(南京)有限公司 | Information recommendation method and device and vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110363555B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110363555A (en) | Recommended method and device based on eye tracking vision algorithm | |
| US10777020B2 (en) | Virtual representation creation of user for fit and style of apparel and accessories | |
| CN109740466B (en) | Method for obtaining advertisement delivery strategy, and computer-readable storage medium | |
| Baytar et al. | Evaluating garments in augmented reality when shopping online | |
| Choi et al. | Viton-hd: High-resolution virtual try-on via misalignment-aware normalization | |
| US10665022B2 (en) | Augmented reality display system for overlaying apparel and fitness information | |
| CN110503074A (en) | Information labeling method, apparatus, equipment and the storage medium of video frame | |
| EP3493138A1 (en) | Recommendation system based on a user's physical features | |
| US10911695B2 (en) | Information processing apparatus, information processing method, and computer program product | |
| US20160328790A1 (en) | Virtual reality system including smart objects | |
| CN108446943A (en) | Sales counter Method of Commodity Recommendation, device, computer equipment and storage medium | |
| US20050162419A1 (en) | System and method for 3-dimension simulation of glasses | |
| US20120089488A1 (en) | Virtual reality system including smart objects | |
| KR20230025906A (en) | Systems and methods for improved facial attribute classification and its use | |
| US10685457B2 (en) | Systems and methods for visualizing eyewear on a user | |
| US20210407153A1 (en) | High-resolution controllable face aging with spatially-aware conditional gans | |
| US12056743B2 (en) | Image and data processing methods and apparatuses | |
| JPWO2021015265A5 (en) | ||
| WO2012141988A2 (en) | Integrated finding experience systems and methods | |
| CN112884556A (en) | Shop display method, system, equipment and medium based on mixed reality | |
| CN116703507A (en) | Image processing method, display method and computing device | |
| Zhang et al. | Stylized text-to-fashion image generation | |
| Samy et al. | Revolutionizing online shopping with FITMI: A realistic virtual try-on solution | |
| Suryanto et al. | Virtual reality technology-based impact of fashion design technology using optimized neural network | |
| De Almeida et al. | Interactive makeup tutorial using face tracking and augmented reality on mobile devices |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240310 Address after: Room 901, No. 2, Lane 288, Qianfan Road, Xinqiao Town, Songjiang District, Shanghai 201612 Applicant after: Shikong (Shanghai) brand planning Co.,Ltd. Country or region after: China Address before: 518000 East Wing 03A-2, 3rd Floor, Building 1, Weixin Software Park, No. 9 Gaoxin South Ninth Road, Yuehai Street, Nanshan District, Shenzhen, Guangdong Province Applicant before: SHENZHEN AXMTEC CO.,LTD. Country or region before: China |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |