CN109672868A - Multi-cam safety pre-warning system, method, vehicle and terminal device - Google Patents
Multi-cam safety pre-warning system, method, vehicle and terminal device Download PDFInfo
- Publication number
- CN109672868A CN109672868A CN201910092790.3A CN201910092790A CN109672868A CN 109672868 A CN109672868 A CN 109672868A CN 201910092790 A CN201910092790 A CN 201910092790A CN 109672868 A CN109672868 A CN 109672868A
- Authority
- CN
- China
- Prior art keywords
- camera
- target object
- head
- security standpoint
- default security
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000001514 detection method Methods 0.000 claims description 43
- 230000015654 memory Effects 0.000 claims description 33
- 230000000007 visual effect Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 2
- 230000001737 promoting effect Effects 0.000 abstract description 2
- 210000003128 head Anatomy 0.000 description 162
- 230000006870 function Effects 0.000 description 13
- 238000004590 computer program Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 238000012800 visualization Methods 0.000 description 9
- 210000001217 buttock Anatomy 0.000 description 8
- 206010039203 Road traffic accident Diseases 0.000 description 6
- 230000035939 shock Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000004424 eye movement Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 210000004709 eyebrow Anatomy 0.000 description 2
- 230000001815 facial effect Effects 0.000 description 2
- 230000008921 facial expression Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011017 operating method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 241000406668 Loxodonta cyclotis Species 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000007795 chemical reaction product Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 210000004209 hair Anatomy 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 230000005389 magnetism Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000638 stimulation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
The present invention provides a kind of multi-cam safety pre-warning system, method, vehicle and terminal devices, wherein the system includes: the first camera, for obtaining the first head image of target object；First eye tracker, whether the deflection angle for determining target object head relative to the first camera is within the scope of the first default security standpoint；Second camera, for obtaining the second head image of target object；Second eye tracker, determine target object head relative to second camera deflection angle whether within the scope of the second default security standpoint；Precaution device, for target object head relative to the first camera deflection angle not within the scope of the first default security standpoint, and target object head relative to second camera deflection angle not within the scope of the second default security standpoint in the case where, carry out safe early warning.It through the above scheme can be while effectively promoting drive safety, it is possible to reduce to the technical effect of the interference of driver.
Description
Technical field
The present invention relates to equipment safety technical field, in particular to a kind of multi-cam safety pre-warning system, method, vehicle
And terminal device.
Background technique
As people's living standard is higher and higher, the utilization rate of automobile is also higher and higher.Automobile is as a kind of vehicles
Safety and the attention of driver have much relations, if driver's rotary head etc., will lead to sight can't see front
Road conditions, be easy cause traffic accident.
In normal driving procedure, because driver's energy is not concentrated, the case where the problems such as rotary head results in an automobile accident, is very high,
This ensure the safety driven cannot.
In view of the above-mentioned problems, currently no effective solution has been proposed.
Summary of the invention
The embodiment of the invention provides a kind of multi-cam safety pre-warning system, method, vehicle and terminal devices, to reach
Promote the purpose of vehicle driving safety.
The embodiment of the invention provides a kind of multi-cam safety pre-warning system, method, vehicle and terminal devices, specifically
:
A kind of multi-cam safety pre-warning system, comprising:
First camera, for obtaining target pair in the case where target object is in the detection range of the first camera
The first head image of elephant；
First eye tracker, for determining that the target object head is taken the photograph relative to first according to first head image
As whether the deflection angle of head is within the scope of the first default security standpoint；
Second camera, for obtaining target in the case where target object is not in the detection range of the first camera
Second head image of object；
Second eye tracker, for determining that the target object head is taken the photograph relative to second according to second head image
As whether the deflection angle of head is within the scope of the second default security standpoint；
Precaution device, for not presetting peace first relative to the deflection angle of the first camera on the target object head
Within the scope of full angle, and, the target object head relative to second camera deflection angle not in the second default safety angle
In the case where spending in range, safe early warning is carried out.
In one embodiment, first camera and first eye tracker be arranged in that vehicle master drives just before
The right side of first camera and second eye tracker is arranged in side, the second camera and second eye tracker.
In one embodiment, the precaution device includes at least one of: shock sensor, stereo set, display
Device, display lamp.
In one embodiment, first eye tracker is TobiiX series eye tracker, and second eye tracker is
TobiiX series eye tracker.
A kind of multi-cam safe early warning method, comprising:
Determine target object whether in the detection range of the first camera；
In the case where target object is in the detection range of the first camera, the first head figure of target object is obtained
Picture；
According to first head image, determine that the target object head is relative to the deflection angle of the first camera
It is no within the scope of the first default security standpoint；
In the case where target object is not in the detection range of the first camera, the second head figure of target object is obtained
Picture；
According to second head image, determine that the target object head is relative to the deflection angle of second camera
It is no within the scope of the second default security standpoint；
The target object head relative to the first camera deflection angle not in the first default security standpoint range
It is interior, and, the target object head relative to second camera deflection angle not within the scope of the second default security standpoint
In the case of, carry out safe early warning.
In one embodiment, the target object head relative to the first camera deflection angle not first
Within the scope of default security standpoint, and, the target object head is default not second relative to the deflection angle of second camera
In the case where within the scope of security standpoint, safe early warning is carried out, comprising:
The target object head relative to the first camera deflection angle not in the first default security standpoint range
It is interior, and, the target object head relative to second camera deflection angle not within the scope of the second default security standpoint
In the case that duration reaches the first duration, early warning is carried out according to the first alarm mode；
The target object head relative to the first camera deflection angle not in the first default security standpoint range
It is interior, and, the target object head relative to second camera deflection angle not within the scope of the second default security standpoint
In the case that duration reaches the second duration, early warning is carried out according to the second alarm mode；
Wherein, second duration is greater than first duration, and the intensity of second alarm mode is higher than described first
The intensity of alarm mode.
In one embodiment, the target object head relative to the first camera deflection angle not first
Within the scope of default security standpoint, and, the target object head is default not second relative to the deflection angle of second camera
In the case where within the scope of security standpoint, during carrying out safe early warning, further includes:
Obtain the current speed of service of vehicle；
According to the speed of service, first duration and second duration are determined, wherein the speed of service is got over
Height, first duration and second duration are shorter.
In one embodiment, first alarm mode includes at least one of: visualization progress bar mode, sound
Sound mode, vibration mode；Second alarm mode includes at least one of: visualization progress bar mode, voice mode, shake
Flowing mode.
In one embodiment, according to second head image, determine the target object head relative to second
Whether the deflection angle of camera is within the scope of the second default security standpoint, comprising:
Determine the angle that the head of the target object is tilted to the right；
According to the angle being tilted to the right, the second default security standpoint corresponding with the angle being tilted to the right is obtained
Range；
According to second head image, determine that the target object head is relative to the deflection angle of second camera
It is no within the scope of the second default security standpoint；
Wherein, the different angle being tilted to the right, corresponding the second different default security standpoint range.
In one embodiment, the described first default security standpoint range includes: the security standpoint model on left and right directions
Enclose with the security standpoint range in up and down direction, the second default security standpoint range includes: the safety angle on left and right directions
Spend the security standpoint range in range and up and down direction.
In one embodiment, described before determining target object whether in the detection range of the first camera
Method further include:
Position according to the target object face feature point relative to first camera, just to the target object
It is calibrated the spatial position on head when immediately ahead of often driving and being visual；
The head for controlling the target object is moved, with the default security standpoint range of determination described first.
A kind of vehicle, comprising: above-mentioned multi-cam safety pre-warning system.
A kind of terminal device, including processor and for the memory of storage processor executable instruction, the processing
The step of device realizes following method when executing described instruction:
Determine target object whether in the detection range of the first camera；
In the case where target object is in the detection range of the first camera, the first head figure of target object is obtained
Picture；
According to first head image, determine that the target object head is relative to the deflection angle of the first camera
It is no within the scope of the first default security standpoint；
In the case where target object is not in the detection range of the first camera, the second head figure of target object is obtained
Picture；
According to second head image, determine that the target object head is relative to the deflection angle of second camera
It is no within the scope of the second default security standpoint；
The target object head relative to the first camera deflection angle not in the first default security standpoint range
It is interior, and, the target object head relative to second camera deflection angle not within the scope of the second default security standpoint
In the case of, carry out safe early warning.
A kind of computer readable storage medium is stored thereon with computer instruction, and it is as follows that described instruction is performed realization
The step of method:
Determine target object whether in the detection range of the first camera；
In the case where target object is in the detection range of the first camera, the first head figure of target object is obtained
Picture；
According to first head image, determine that the target object head is relative to the deflection angle of the first camera
It is no within the scope of the first default security standpoint；
In the case where target object is not in the detection range of the first camera, the second head figure of target object is obtained
Picture；
According to second head image, determine that the target object head is relative to the deflection angle of second camera
It is no within the scope of the second default security standpoint；
The target object head relative to the first camera deflection angle not in the first default security standpoint range
It is interior, and, the target object head relative to second camera deflection angle not within the scope of the second default security standpoint
In the case of, carry out safe early warning.
In embodiments of the present invention, a kind of multi-cam safety pre-warning system is provided, comprising: the first camera, first
Eye tracker, second camera and the second eye tracker because being provided with two sets of detection systems, bend over to the right in user or
If being that eyes are to face front when taking object to the right, it will not be mistaken for not seeing front, only pass through so as to reduce
The possibility of the erroneous judgement of presence caused by a set of detection device, so as to subtract while effectively promoting drive safety
The technical effect of few interference to driver.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, not
Constitute limitation of the invention.In the accompanying drawings:
Fig. 1 is the method flow diagram of multi-cam safe early warning method according to an embodiment of the present invention；
Fig. 2 is the building block schematic diagram of multi-cam safety pre-warning system according to an embodiment of the present invention；
Fig. 3 is 68 facial feature points schematic diagrames according to an embodiment of the present invention；
Fig. 4 is the configuration diagram of terminal according to an embodiment of the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, right below with reference to embodiment and attached drawing
The present invention is described in further details.Here, exemplary embodiment and its explanation of the invention is used to explain the present invention, but simultaneously
It is not as a limitation of the invention.
In the present specification, such as adjective as first and second can be only used for by an element or movement with it is another
One element or movement distinguish, without requiring or implying any actual this relationship or sequence.In the feelings that environment allows
Under condition, one in only element, component or step should not be interpreted as limited to referring to element or component or step (s), and can
To be the one or more etc. in element, component or step.
For the problem that in existing vehicle operation, because user's rotary head etc. leads to traffic accident, it is contemplated that if
The inclined head degree of driver can be detected, and in the case where the inclined head degree of determination is excessively high, driver is reminded,
Attention when driver drives so can be effectively promoted, to reduce the generation of traffic accident.For this purpose, in this example,
It does not need to remind in fact whens equal in view of driver is tilted to the right sometimes, provides a kind of hair for reducing traffic accident
It is raw, and the method that the interference to driver can be reduced.
Fig. 1 is a kind of method flow diagram of herein described multi-cam safe early warning method one embodiment.Although this
Application provides as the following examples or method operating procedure shown in the drawings or apparatus structure, but based on conventional or without wound
The labour for the property made may include more or less operating procedure or modular unit in the method or device.In logicality
In upper the step of there is no necessary causalities or structure, the execution sequence of these steps or the modular structure of device are not limited to this
Apply for embodiment description and execution shown in the drawings sequence or modular structure.The method or modular structure in practice
Device or end product according to embodiment or method shown in the drawings or modular structure connection progress sequence in application, can hold
Capable or parallel execution (such as environment or even distributed processing environment of parallel processor or multiple threads).
As shown in Figure 1, providing a kind of multi-cam safe early warning method in this example, may include steps of:
Step 101: determining target object whether in the detection range of the first camera；
Step 102: in the case where target object is in the detection range of the first camera, obtaining the first of target object
Head image；
Step 103: according to first head image, determining the target object head relative to the inclined of the first camera
Whether gyration is within the scope of the first default security standpoint；
Step 104: in the case where target object is not in the detection range of the first camera, obtaining the of target object
Two head images；
Step 105: according to second head image, determining the target object head relative to the inclined of second camera
Whether gyration is within the scope of the second default security standpoint；
Step 106: default safe not first relative to the deflection angle of the first camera on the target object head
In angular range, and, the target object head relative to second camera deflection angle not in the second default security standpoint
In the case where in range, safe early warning is carried out.
In upper example, a kind of multi-cam safe early warning method is provided, is only taken the photograph on target object head relative to first
As the deflection angle of head is not within the scope of the first default security standpoint, and, target object head is relative to the inclined of second camera
Carry out safe early warning can be just triggered in the case that gyration is not within the scope of the second default security standpoint, because by being provided with two
A camera more accurately judges so as to realize, can to avoid because driver be only be tilted to the right just carry out it is pre- safely
Alert problem can promote user experience, avoid error prompting.
In view of in actual driving operational process, driver's rotary head duration is longer, and risk is higher, because
This, can be arranged different degrees of prompting according to rotary head duration, for example, can not remind if continuing 1s, 3s can be with
It is simple to remind, 5s and high-intensitive prompting can be carried out with first-class, thus realize the prompting of the varying strength based on duration,
Also illustrate that the height of current safety coefficient.For example, the target object head relative to the first camera deflection angle not
Within the scope of the first default security standpoint, and, the target object head relative to second camera deflection angle not
In the case where within the scope of two default security standpoints, carry out safe early warning when, can the target object head relative to
The deflection angle of first camera not within the scope of the first default security standpoint, and, the target object head is relative to second
In the case that duration of the deflection angle of camera not within the scope of the second default security standpoint reaches the first duration, according to
First alarm mode carries out early warning；It is default not first relative to the deflection angle of the first camera on the target object head
Within the scope of security standpoint, and, the target object head is default safe not second relative to the deflection angle of second camera
In the case that duration in angular range reaches the second duration, early warning is carried out according to the second alarm mode；Wherein, described
Two durations are greater than first duration, and the intensity of second alarm mode is higher than the intensity of first alarm mode.
It should be noted, however, that above-mentioned cited duration etc. is only a kind of exemplary description, may include:
First duration and the second duration can also include: third duration, the 4th duration etc..Several security levels are specifically arranged can be with
It is set according to actual needs with situation, the application is not construed as limiting this.
In view of in the faster situation of Vehicle Speed, allow the time of system for distraction of driver be it is very short, therefore,
Target object head relative to the first camera deflection angle not within the scope of the first default security standpoint, and, the target
Object header relative to second camera deflection angle not within the scope of the second default security standpoint in the case where, carry out safety
During early warning, the current speed of service of available vehicle；According to the speed of service, first duration and institute are determined
State the second duration, wherein the speed of service is higher, and first duration and second duration are shorter.
That is, being directed to the different speeds of service, different durations can be set, the speed of service is higher, then being just arranged shorter
Duration threshold value, so as to improve the safety of traveling.
The first above-mentioned alarm mode can include but is not limited at least one of: visualization progress bar mode, sound
Mode, vibration mode；Above-mentioned second alarm mode can include but is not limited at least one of: visualization progress bar mode,
Voice mode, vibration mode.
It should be noted, however, that the concrete mode of above-mentioned listed the first alarm mode and the second alarm mode enumerated
It is only a kind of exemplary description, when actually realizing, early warning can also be carried out using other way, as long as can remind
The mode that driver pays attention to all can serve as alarm mode, and the application is not construed as limiting this.
In view of when driver, which is sitting in operator seat, realizes that body inclination side is visited, buttocks will not occur the whole of left-right position and move
It is dynamic, it is to be tilted by axis of buttocks, above the waist and head tilts together, also, everyone upper part of the body height is to fix not
Become, therefore, when body seat-type water inclined side is visited, is similar to a bar around fixed-axis rotation.Two shoulder lines are vertical with body direction
, when body inclination side is visited, by the angle β between two shoulder lines and vertical direction, can be obtained upper part of the body body relative to
The inclination angle alpha of vertical direction, α=90 °-β.Since the upper part of the body height of human body is fixed and invariable, driver is sitting in driving
When position realizes that body inclination side is visited, the overall movement of left-right position, which will not occur, for buttocks is tilted by axis of buttocks, upper half
Body and head tilt together, and therefore, after determining above the waist relative to the inclination angle alpha of vertical direction, the position on head is also just true
It is fixed.
Therefore, for different tilt angles, the default security standpoint range of different second can be determined, correspondingly, root
According to second head image, determine whether the target object head is pre- second relative to the deflection angle of second camera
If may include: the angle that the head of the determining target object is tilted to the right within the scope of security standpoint；According to described to Right deviation
Oblique angle obtains the second default security standpoint range corresponding with the angle being tilted to the right；According to second head
Image, determine the target object head relative to second camera deflection angle whether in the second default security standpoint range
It is interior；Wherein, the different angle being tilted to the right, corresponding the second different default security standpoint range.
In one embodiment, the first above-mentioned default security standpoint range may include: the safety on left and right directions
Security standpoint range on angular range and up and down direction, the second default security standpoint range may include: left and right directions
On security standpoint range and up and down direction on security standpoint range.I.e., it is possible to from left and right directions deflection and upper and lower
The two dimensions of upward deflection are judged.
Because different vehicle or the different heights for driving vehicle or sitting height etc. are different, in order to enable testing result is more
It, can be according to the target object face feature point relative to described before the head image for obtaining target object to be accurate
It calibrates the spatial position of the position of camera, head when to the target object normal driving and visual front；Also
The head that can control the target object is moved, with the determination default security standpoint range.That is, being directed to each vehicle
Each driver carries out position and default security standpoint range is calibrated.It certainly can also using same set of default security standpoint
With, but accuracy can effectively be promoted by calibrating personalizedly.
If can provide a memory and face identification functions using personalized ground calibrating mode, be driven for each
The person of sailing is calibrated, and by the corresponding camera position of the driver, the first default security standpoint range, the second default safety angle
Degree range etc. is stored, and user first passes through the identity that recognition of face goes out current drivers when driving, and is then transferred pair
The calibration information answered is configured.
In this example, a kind of multi-cam safety pre-warning system is additionally provided, as shown in Fig. 2, may include:
First camera 201 can be used in the case where target object is in the detection range of the first camera, obtain
First head image of target object；
First eye tracker 202, can be used for according to first head image, determine the target object head relative to
Whether the deflection angle of the first camera is within the scope of the first default security standpoint；
Second camera 203 can be used for obtaining in the case where target object is not in the detection range of the first camera
Take the second head image of target object；
Second eye tracker 204, can be used for according to second head image, determine the target object head relative to
Whether the deflection angle of second camera is within the scope of the second default security standpoint；
Precaution device 205, can be used for the target object head relative to the first camera deflection angle not
Within the scope of one default security standpoint, and, the target object head is pre- not second relative to the deflection angle of second camera
If in the case where within the scope of security standpoint, carrying out safe early warning.
The front that vehicle master drives, second camera is arranged in above-mentioned the first camera 201 and the first eye tracker 202
203 and 204 instrument of the second eye movement the right side of the first camera and the second eye tracker is set.
Wherein, above-mentioned precaution device can include but is not limited at least one of: shock sensor, stereo set, display
Device, display lamp.For example, above-mentioned first eye tracker is TobiiX series eye tracker, the second eye tracker is also possible to TobiiX series
Eye tracker.
The above method is illustrated below with reference to a specific embodiment, it should be noted, however, that the specific implementation
Example does not constitute an undue limitation on the present application merely to the application is better described.
In this example, a kind of driver based on face recognition is provided to pay attention to tracking feedback system, it can based on the system
The head position of driver is identified with 68 characteristic points by facial expression camera identification face；Judged by eye tracker
Above and below when driver is relative to front, left and right security standpoint range；According to head relative to above and below front, deflection position
Set with up and down, compared with the security standpoint of left and right and add up deflection period and judge whether to give driver safety warning to feed back.
The system and method are specifically described below as follows:
1) face feature point arrangement mode is as follows:
As shown in figure 3,68 characteristic points can be set in face, wherein this 68 characteristic points can be respectively left and right eyebrow
Each 5 points, successively arrange according to the position of eyebrow, from inside to outside respectively LB1, LB2, LB3, LB4, LB5, RB1, RB2, RB3,
RB4,RB5；Each 6 points of left and right eye, each 2 points of the upper palpebra inferior of respectively every eye, each 1 point in left and right canthus, respectively
LE1,LE2,LE3,LE4,LE5,LE6,RE1,RE2,RE3,RE4,RE5,RE6；9 points of nose, respectively 4 points of the bridge of the nose, from
On be down respectively N1, N2, N3, N4, the point of nostril lower part 5, from left to right respectively N5, N6, N7, N8, N9；Mouth 20
Point, respectively upper each 7 points of lower lip, be from left to right followed successively by UL1, UL2, UL3, UL4, UL5, UL6, UL7, LL1, LL2,
LL3, LL4, LL5, LL6, LL7, upper each 3 points of lower tooth, are from left to right followed successively by as UT1, UT2, UT3, LT1, LT2, LT3；Face
17 points of cheek, be from left to right followed successively by C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15,
C16,C17；Wherein N1, N2, N3, N4, N7, UL4, LL4, UT2, LT2, C9 point is point-blank.Pass through 68 facial characteristics
Point can effectively identify face.
2) above system platform may include three big modules, be respectively as follows: system setup module, parameter control module, in real time
State display module, in which:
System setup module, for being configured to whole system configuration, identification facial expression camera, identification eye movement
Instrument, identification shock sensor, Recognition feedback audio etc..
Parameter control module, for being configured to the feedback system in entire feedback procedure, feedback stage can be by
Four test phases: first stage, second stage, phase III and fourth stage are preset according to the time；Wherein, the first stage is peace
Full driving phase, at this stage, subject head deflection range are to allow to be tested more than security standpoint range, without carrying out safe police
Show, may be considered non-security driving phase in rear three phases, three phases after accumulative deflection period is in, then can be with
According to the difference of feedback setting, different security warnings is provided, to prompt subject.
Wherein, feedback form can include but is not limited at least one of: triggering shock sensor, vision progress bar,
Sound warning etc., during actual feedback, can not had according to subject the duration of fixating straight ahead it is different into
The prompt of row varying strength is fed back.
In order to realize above-mentioned demand, the above-mentioned driver based on face recognition notices that tracking feedback system may include:
Pay attention to Trancking Software platform, face recognition behavior camera, shock sensor, put stereo set, Tobii X series eye movement outside
Instrument, the system eye tracker or so can survey range and can be set are as follows: wide * high is about 50*36cm (24*14 °).Wherein, above-mentioned to take the photograph
In equipment as where head, shock sensor, stereo set and eye tracker etc. can be inserted into software platform by USB interface.
It should be noted, however, that the above-mentioned cited driver based on face recognition pays attention to tracking in feedback system
Included component is only a kind of exemplary description, when actually realizing, can select several settings therein, can be with
It is arranged using other components or other types of component, the application is not construed as limiting this.
Specifically, prompting can be tracked in accordance with the following steps:
S1: face recognition, head position determine:
Face recognition camera is placed on immediately ahead of head, the relative position to head relative to camera carries out school
Standard, specifically, calibration can be carried out as follows:
Using camera position as space origins 0 (0,0,0) point, camera is placed on the front on head, the sky on head
Between position calculating can be calculate face feature point in position of the N4 point relative to camera, N4 point coordinate be N4n (Xn, Yn,
Zn), by calibration, with N40 (X0, Y0, Z0) for position of the head relative to the origin of camera.Wherein, head is relative to taking the photograph
As the position N40 (X0, Y0, Z0) of the origin of head refers to: the spatial position on head when subject normal driving and visually front.
S2: head security standpoint range determines:
Eye tracker is placed on subject front, calculates the security standpoint range of subject head deflection.By to the left, to
It is right, upwards, move down head, calculate the first security standpoint range of head deflection, circular is as follows:
1) determine head in X, Y, Z value of spatial position, wherein X, Y, Z value are the positions on head relative to camera
Position, wherein the deflection angle on the left and right directions of head is calculated by X, the deflection angle of head up and down direction is calculated by Y,
The distance of head distance camera is calculated by Z, n is each sampled point.For example, can be set head relative to camera away from
From in 500mm or so；Specific distance can determines according to actual conditions, and the application is not construed as limiting this.
Deflection angle of the head relative to camera refers to a certain sampling instant, and line between camera and N4 point is taken the photograph
Angle as the line between head and N40 (X0, Y0, Z0), between two lines.
2) head is set relative to camera deflection angle as Qn (an, bn), wherein an is the deflection angle on left and right directions
Degree, bn be up and down direction on deflection angle, after the completion of calibration, head relative to camera origin position N40 (X0, Y0,
It Z0 is) Q0 (0,0) relative to the deflection angle of camera, that is, head is Q0 (0,0) relative to the deflection angle of camera；
3) the head position N4n (Xn, Yn, Zn) of each sampling is recorded in real time；
4) angle that each frame head portion is deflected relative to camera is calculated in real time, specifically, can be according to following formula meter
It calculates:
An=arccos [(X0-0)2+(Xn-0)2-(Xn-X0)2]/[2 (X0-0) (Xn-0)]=arccos [X02+Xn2-
(Xn-X0)2]/(2X0Xn),
Bn=arccos [Y02+Yn2-(Yn-Y0)2]/(2Y0Yn)；
5) boundary head position when subject watches front attentively is obtained by eye tracker, and determines boundary head position and camera shooting
Angle between head, so that it is determined that the first security standpoint range out.
Determine head to the left, to the right, upwards, when deflecting down, eye tracker cannot capture angle when eyes, wherein
Head or so full angle range content to retain sovereignty over a part of the country is A (L1, R1), upper and lower full angle range content to retain sovereignty over a part of the country is B (U1, D1).
Eye tracker is placed in front of subject, by realization of debugging, when subject watches front attentively, eye tracker can be captured always
To the eyes of subject, when subject deflects head to the left or to the right and do not see front, eye tracker is captured completely less than being tested eyes.
Subject head deflects to the left, when eye tracker is captured less than eyes completely, records subject head position at this time i.e.
N4 point position as left border head position, and calculates the angle between N4 point and camera at this time, opens as left side interference
Beginning angle L1.
Subject head deflects to the right, when eye tracker is captured less than eyes completely, records subject head position at this time i.e.
N4 point position as right side boundary head position, and calculates the angle between N4 point and camera at this time, opens as right side interference
Beginning angle R1.
Left side interfere start angle and right side interference start between angle for head or so full angle range A content to retain sovereignty over a part of the country (L1,
R1), that is, when the angle being tested between head position and camera is within the scope of this security standpoint, it is believed that subject is look at
Front belongs to normal driving.
Similarly, full angle range content to retain sovereignty over a part of the country is B (U1, D1) up and down.
That is, can determine head or so full angle range A (L1, R1) content to retain sovereignty over a part of the country and up and down full angle content to retain sovereignty over a part of the country through the above way
Range is B (U1, D1), regard the A (L1, R1) of above-mentioned determination and B (U1, D1) as the first security standpoint range.
S3: judgement and execution security warning feedback:
The calculation method in S2 4) calculates deflection angle of each frame head portion relative to camera in real time through the above steps,
And whether an, bn are in A, B angular range in real-time judge head deflection angle Qn (an, bn), if in the first security standpoint model
In enclosing, then without early warning, if not within the scope of the first security standpoint, adding up at any time carries out the early warning of different phase
Feedback.
It, can be with specifically, the feedback stage that above-mentioned head deflection angle is not fed back within the scope of the first security standpoint
Multiple stages are set as, for example, can be set to 4 stages: first stage -- range of stability, second stage -- dangerous 1 rank
Section, phase III in -- dangerous 2 stage and fourth stage -- dangerous 3 stage, wherein the first stage without feedback (because
The duration is shorter at this time, and the possibility caused danger is smaller), the mode of feedback can be freely combined in other three phases, such as:
By one of visualization progress bar, sound, the modes such as vibration or a variety of it can carry out feedback early warning.
For example, subject's head deflection acts certain time, that is, it is in the first stage, within this first stage, no
It gives and feeds back.If subject head deflection movement continues for, the first stage terminates, into second stage；If being tested head
Portion's deflection action continues for, then second stage terminates, into the phase III；If subject head deflection movement continues for,
Then the phase III terminates, into fourth stage.In second stage, phase III, fourth stage, gives and feed back.
When subject head deflection movement terminates, recovery normal angled, then feedback stage timing is zeroed.It is dynamic to be tested head deflection
When occurring again, restart timing.
For example, the duration of default first stage is 1s, the duration 5s of second stage, then when subject head is inclined
Gyration outside the first security standpoint range the duration be 2s when in second stage；If the head deflection time 6s when
It waits and returns within the scope of normal angled i.e. the first security standpoint again, timing zero.Head deviation angle is so calculated again to continue
Total time is then again from 0 timing.
Wherein, above-mentioned second stage, phase III, in fourth stage, the intensity of feedback can be sequentially increased.Selection is visual
Change feedback system, setting visualization interval time, display mode may include that visualization progress bar gradient shows, visualizes flashing
Display, visualization gradual change are shown；Sound is selected to warn feedback system, setting sound warns the feedback interval time；Selection vibration is anti-
Feedback mode, setting vibrational feedback time interval etc..
The duration in each stage in above-mentioned preceding three phases can determine in conjunction with current speed, correspond to different vehicles
Speed, the preceding three phases respective duration can be set to it is different, for example, speed is bigger, corresponding each stage it is lasting when
Between should be shorter, thus it is shorter to enter second stage, phase III, the required duration of fourth stage.
For example, speed is 10km/h, then duration first stage is 2 seconds, and the second stage duration is 3 seconds；And work as
Speed is 50km/h, then duration first stage is 1 second, and the second stage duration is 2 seconds；Because speed is faster, traveling
It is shorter the time required to certain distance, it is shorter the time required to causing danger, more driver is needed to restore normal attitude as early as possible.
Feedback system is tracked by above-mentioned attention, it on the one hand can be by monitoring before different speed head portions deviate just
The duration of the relative skewness angle of side provides a warning feedback to driver, to avoid because of the dry of other stimulations
The attention concentrativeness for influencing driver is disturbed, traffic accident is reduced；On the other hand by acquiring different speeds, setting is different
The safe early warning time range in stage, so as to pass through real-time monitoring speed, root during safe driving and automatic Pilot
Early warning feedback mechanism is provided in real time according to the variation of speed.
Further, it is contemplated that sometimes driver may inclined side body of stretching forward leave normal driving position, go enough to take and drive
Indoor article is sailed, such as: it is placed on co-driver or the article in front of it.It stretches forward to certain position, head is in left-right position
It is fixed, and then has rotation, observes vehicle front or other articles.
In this case, driver's eyes may be to look at front, it is also possible to not look at front, be located at and drive
Member normal driving position immediately ahead of the first camera detection be more than relative to the deflection angle of this camera to driver head
First security standpoint range (that is, the security standpoint range of driver in normal driving position) head completely disengage this first
The visual range of camera will do it feedback to certain time feedback system by above-mentioned early warning rule.Then, if at this time
Driver's eyes are seen forwards, then detect and feedback does not meet determination requirement actually required, that is, above-mentioned feedback
System can not correctly reflect whether driver watches front attentively.
Mounting distance due to camera, eye tracker apart from driver is closer, the detection range of camera, eye tracker itself
And therefore big detection error problem within the vision only detects driver in larger range by a camera, eye tracker
Interior head deflection angle judges whether driver watches front attentively whether within the scope of security standpoint, realizes driver's note
Meaning tracking feedback, is existing defects.For this purpose, additionally provide in this example it is a kind of by mostly set equipment joint-detections in the way of,
To achieve the purpose that accurately to detect.
It may include: the first camera, the first eye tracker, second specifically, providing a tracing system in this example
Camera and the second eye tracker.
Wherein, the first camera and the first eye tracker, the attention for driver's normal driving position are tracked, second camera
The right side of first camera and the first eye tracker is set with the second eye tracker, moves right for driver and is detached from the first safety
After angular range, when into the detection range of second camera and the second eye tracker, pass through second camera and the second eye tracker
It is detected.When entering the detection range of second camera and the second eye tracker, the first camera and the first eye tracker not into
Row feedback, is failure to actuate, second set of equipment is fed back and acted.The left side detection range of second camera and the second eye tracker with
The right side detection range of first camera and the first eye tracker partly overlaps.
First camera and the first eye tracker, second camera and the second eye tracker, drive as long as there is a set of equipment to detect
The person of sailing watches front attentively, then deciding that driver watches front attentively.
Firstly, subordinate act habit, to analyze, when driver is sitting in operator seat realization body inclination side spy, buttocks will not occur
The overall movement of left-right position is tilted by axis of buttocks, and above the waist and head tilts together, also, everyone is upper
Half body height is fixed and invariable, and therefore, when body seat-type water inclined side is visited, is similar to a bar around fixed-axis rotation.Two shoulder lines
Be with body direction it is vertical, body inclination side visit when, by the angle β between two shoulder lines and vertical direction, can be obtained
Inclination angle alpha of the upper part of the body body relative to vertical direction, α=90 °-β.Since the upper part of the body height of human body is to immobilize
, when driver is sitting in operator seat realization body inclination side spy, the overall movement of left-right position will not occur for buttocks, be with buttocks
It is tilted for axis, above the waist and head tilts together, therefore, after being determined above the waist relative to the inclination angle alpha of vertical direction,
The position on head also determines that.
It, can be former using second camera position as second space when carrying out face recognition and head position determines
0 (0,0,0) point of point.Assuming that above the waist relative to vertical direction be tilted to the right angle [alpha] be C when, and driver eye fixation in front of
When, by calibration, N4 point in head is set to N4 at this timeC0(XC0, YC0, ZC0), it is calculated as origin position of the head relative to second camera
It sets.
Specifically, can recorde origin position of the head relative to second camera when angle [alpha] is various values, and such as: α
When respectively 10 °, 15 °, 20 ° ... 80 ° of all angles, origin position N4 of the head relative to second cameraα0(Xα0, Yα0,
Zα0).Wherein, α value interval can determine according to actual needs, for example, can be with value for 5 ° etc., that is, take one at interval of 5 °
Angle determines origin position of the head relative to second camera.
When realizing, for each sampling angle, a second security standpoint range is determined, that is, determine note respectively
Record the second security standpoint range when angle [alpha] is various values.For example, determining the second security standpoint range when α is 10 °, α is 15 °
When the second security standpoint range.
It, can be based on the second determining security standpoint range after determining the second security standpoint range of all angles
Carry out security warning feedback.When due to having determined that multiple angles α, head is relative to the origin position of second camera and
Two security standpoint ranges.Therefore, the angle [alpha] of real-time judge body inclination determines that under the angle, whether head deflection angle is the
Within the scope of two security standpoints, if within the scope of the second security standpoint, without early warning, if not in the second security standpoint model
In enclosing, then the early warning feedback of different phase is carried out.
Further, because in view of due to body inclination, driver's reaction is slow, therefore, can from the first stage into
Row early warning feedback.
Embodiment of the method provided by the embodiment of the present application can be in mobile terminal, terminal or similar operation
It is executed in device.For running on computer terminals, Fig. 4 is a kind of multi-cam safe early warning side of the embodiment of the present invention
The hardware block diagram of the terminal of method.As shown in figure 4, terminal 10 may include it is one or more (in figure only
Showing one) (processor 102 can include but is not limited to Micro-processor MCV or programmable logic device FPGA etc. to processor 102
Processing unit), memory 104 for storing data and the transmission module 106 for communication function.This field is common
Technical staff is appreciated that structure shown in Fig. 4 is only to illustrate, and does not cause to limit to the structure of above-mentioned electronic device.Example
Such as, terminal 10 may also include than shown in Fig. 4 more perhaps less component or with different from shown in Fig. 1
Configuration.
Memory 104 can be used for storing the software program and module of application software, as in the embodiment of the present invention take the photograph more
As the corresponding program instruction/module of head safe early warning method, the software that processor 102 is stored in memory 104 by operation
Program and module realize above-mentioned multi-cam safe early warning side thereby executing various function application and data processing
Method.Memory 104 may include high speed random access memory, may also include nonvolatile memory, as one or more magnetism is deposited
Storage device, flash memory or other non-volatile solid state memories.In some instances, memory 104 can further comprise opposite
In the remotely located memory of processor 102, these remote memories can pass through network connection to terminal 10.It is above-mentioned
The example of network includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Transmission module 106 is used to that data to be received or sent via a network.Above-mentioned network specific example may include
The wireless network that the communication providers of terminal 10 provide.In an example, transmission module 106 includes that a network is suitable
Orchestration (Network Interface Controller, NIC), can be connected by base station with other network equipments so as to
Internet is communicated.In an example, transmission module 106 can be radio frequency (RadioFrequency, RF) module, use
In wirelessly being communicated with internet.
In software view, above-mentioned safety early warning device may include:
First determining module, for determining target object whether in the detection range of the first camera；
First obtains module, for obtaining target in the case where target object is in the detection range of the first camera
First head image of object；
Second determining module, for determining the target object head relative to first according to first head image
Whether the deflection angle of camera is within the scope of the first default security standpoint；
Second obtains module, for obtaining mesh in the case where target object is not in the detection range of the first camera
Mark the second head image of object；
Third determining module, for determining the target object head relative to second according to second head image
Whether the deflection angle of camera is within the scope of the second default security standpoint；
Warning module, for default not first relative to the deflection angle of the first camera on the target object head
Within the scope of security standpoint, and, the target object head is default safe not second relative to the deflection angle of second camera
In the case where in angular range, safe early warning is carried out.
In one embodiment, above-mentioned warning module specifically can be on the target object head relative to the first camera shooting
Head deflection angle not within the scope of the first default security standpoint, and, the target object head is relative to second camera
In the case that duration of the deflection angle not within the scope of the second default security standpoint reaches the first duration, according to the first early warning
Mode carries out early warning；The target object head relative to the first camera deflection angle not in the first default security standpoint
In range, and, the target object head relative to second camera deflection angle not in the second default security standpoint range
In the case that interior duration reaches the second duration, early warning is carried out according to the second alarm mode；Wherein, described second when grow up
In first duration, the intensity of second alarm mode is higher than the intensity of first alarm mode.
In one embodiment, above-mentioned warning module can also be on the target object head relative to the first camera
Deflection angle not within the scope of the first default security standpoint, and, the target object head is relative to the inclined of second camera
In the case that gyration is not within the scope of the second default security standpoint, during carrying out safe early warning, it is current to obtain vehicle
The speed of service；According to the speed of service, first duration and second duration are determined, wherein the speed of service is got over
Height, first duration and second duration are shorter.
In one embodiment, above-mentioned first alarm mode can include but is not limited at least one of: visualization
Progress bar mode, voice mode, vibration mode；Above-mentioned second alarm mode can include but is not limited at least one of: can
Depending on changing progress bar mode, voice mode, vibration mode.
In one embodiment, above-mentioned third determining module can specifically determine the head of the target object to Right deviation
Oblique angle；According to the angle being tilted to the right, the second default safety angle corresponding with the angle being tilted to the right is obtained
Spend range；According to second head image, determine that the target object head is relative to the deflection angle of second camera
It is no within the scope of the second default security standpoint；Wherein, the different angle being tilted to the right, corresponding the second different default safety angle
Spend range.
In one embodiment, the described first default security standpoint range may include: the safety angle on left and right directions
The security standpoint range in range and up and down direction is spent, the second default security standpoint range may include: on left and right directions
Security standpoint range and up and down direction on security standpoint range.
In one embodiment, above-mentioned second determining module determine target object whether the first camera detection
Before in range, can position according to the target object face feature point relative to first camera, to the mesh
It is calibrated the spatial position on head when mark object normal driving and visually front；Control the head of the target object into
Row movement, with the default security standpoint range of determination described first.
Embodiments herein also provides in the multi-cam safe early warning method that can be realized in above-described embodiment all
The specific embodiment of a kind of electronic equipment of step, the electronic equipment specifically include following content:
Processor (processor), memory (memory), communication interface (Communications Interface) and
Bus；
Wherein, the processor, memory, communication interface complete the mutual communication processor by the bus
For calling the computer program in the memory, the processor realizes above-described embodiment when executing the computer program
In Overall Steps, for example, the processor execute the computer program when realize following step:
Step 1: determining target object whether in the detection range of the first camera；
Step 2: in the case where target object is in the detection range of the first camera, obtaining first of target object
Portion's image；
Step 3: according to first head image, determining deflection of the target object head relative to the first camera
Whether angle is within the scope of the first default security standpoint；
Step 4: in the case where target object is not in the detection range of the first camera, obtaining the second of target object
Head image；
Step 5: according to second head image, determining deflection of the target object head relative to second camera
Whether angle is within the scope of the second default security standpoint；
Step 6: the target object head relative to the first camera deflection angle not in the first default safety angle
Spend range in, and, the target object head relative to second camera deflection angle not in the second default security standpoint model
In the case where in enclosing, safe early warning is carried out.
As can be seen from the above description, the embodiment of the present application only deflection angle on target object head relative to the first camera
Not within the scope of the first default security standpoint, and, target object head relative to second camera deflection angle not second
Carry out safe early warning can be just triggered in the case where within the scope of default security standpoint, because by being provided with two cameras, thus
It may be implemented more accurately to judge, can be to avoid because driver be only the problem of being tilted to the right just progress safe early warning, it can be with
User experience is promoted, error prompting is avoided.
Embodiments herein also provides in the multi-cam safe early warning method that can be realized in above-described embodiment all
A kind of computer readable storage medium of step is stored with computer program on the computer readable storage medium, the calculating
Machine program realizes the Overall Steps of the multi-cam safe early warning method in above-described embodiment when being executed by processor, for example, institute
It states when processor executes the computer program and realizes following step:
Step 1: determining target object whether in the detection range of the first camera；
Step 2: in the case where target object is in the detection range of the first camera, obtaining first of target object
Portion's image；
Step 3: according to first head image, determining deflection of the target object head relative to the first camera
Whether angle is within the scope of the first default security standpoint；
Step 4: in the case where target object is not in the detection range of the first camera, obtaining the second of target object
Head image；
Step 5: according to second head image, determining deflection of the target object head relative to second camera
Whether angle is within the scope of the second default security standpoint；
Step 6: the target object head relative to the first camera deflection angle not in the first default safety angle
Spend range in, and, the target object head relative to second camera deflection angle not in the second default security standpoint model
In the case where in enclosing, safe early warning is carried out.
As can be seen from the above description, the embodiment of the present application only deflection angle on target object head relative to the first camera
Not within the scope of the first default security standpoint, and, target object head relative to second camera deflection angle not second
Carry out safe early warning can be just triggered in the case where within the scope of default security standpoint, because by being provided with two cameras, thus
It may be implemented more accurately to judge, can be to avoid because driver be only the problem of being tilted to the right just progress safe early warning, it can be with
User experience is promoted, error prompting is avoided.
For convenience of description, it is divided into various modules when description apparatus above with function to describe respectively.Certainly, implementing this
The function of each module can be realized in the same or multiple software and or hardware when specification embodiment, it can also be by reality
Show the module of same function by the combination realization etc. of multiple submodule or subelement.Installation practice described above is only
Schematically, for example, the division of the unit, only a kind of logical function partition, can there is other draw in actual implementation
The mode of dividing, such as multiple units or components can be combined or can be integrated into another system, or some features can be ignored,
Or it does not execute.Another point, shown or discussed mutual coupling, direct-coupling or communication connection can be by one
The indirect coupling or communication connection of a little interfaces, device or unit can be electrical property, mechanical or other forms.
It is also known in the art that other than realizing controller in a manner of pure computer readable program code, it is complete
Entirely can by by method and step carry out programming in logic come so that controller with logic gate, switch, specific integrated circuit, programmable
Logic controller realizes identical function with the form for being embedded in microcontroller etc..Therefore this controller is considered one kind
Hardware component, and the structure that the device for realizing various functions that its inside includes can also be considered as in hardware component.Or
Person even, can will be considered as realizing the device of various functions either the software module of implementation method can be hardware again
Structure in component.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It will be understood by those skilled in the art that the embodiment of this specification can provide as the production of method, system or computer program
Product.Therefore, in terms of this specification embodiment can be used complete hardware embodiment, complete software embodiment or combine software and hardware
Embodiment form.Moreover, it wherein includes computer available programs that this specification embodiment, which can be used in one or more,
Implement in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of code
The form of computer program product.
This specification embodiment can describe in the general context of computer-executable instructions executed by a computer,
Such as program module.Generally, program module includes routines performing specific tasks or implementing specific abstract data types, journey
Sequence, object, component, data structure etc..This specification embodiment can also be practiced in a distributed computing environment, in these points
Cloth calculates in environment, by executing task by the connected remote processing devices of communication network.In distributed computing ring
In border, program module can be located in the local and remote computer storage media including storage equipment.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for system reality
For applying example, since it is substantially similar to the method embodiment, so being described relatively simple, related place is referring to embodiment of the method
Part explanation.In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ",
The description of " specific example " or " some examples " etc. means specific features described in conjunction with this embodiment or example, structure, material
Or feature is contained at least one embodiment or example of this specification embodiment.In the present specification, to above-mentioned term
Schematic representation be necessarily directed to identical embodiment or example.Moreover, description specific features, structure, material or
Person's feature may be combined in any suitable manner in any one or more of the embodiments or examples.In addition, in not conflicting feelings
Under condition, those skilled in the art by different embodiments or examples described in this specification and different embodiment or can show
The feature of example is combined.
The foregoing is merely the embodiments of this specification embodiment, are not limited to this specification embodiment.It is right
For those skilled in the art, this specification embodiment can have various modifications and variations.It is all in this specification embodiment
Any modification, equivalent replacement, improvement and so within spirit and principle, the right that should be included in this specification embodiment are wanted
Within the scope of asking.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the embodiment of the present invention can have various modifications and variations.All within the spirits and principles of the present invention, made
Any modification, equivalent substitution, improvement and etc. should all be included in the protection scope of the present invention.
Claims (10)
1. a kind of multi-cam safety pre-warning system characterized by comprising
First camera, for obtaining target object in the case where target object is in the detection range of the first camera
First head image；
First eye tracker, for determining the target object head relative to the first camera according to first head image
Deflection angle whether within the scope of the first default security standpoint；
Second camera, for obtaining target object in the case where target object is not in the detection range of the first camera
The second head image；
Second eye tracker, for determining the target object head relative to second camera according to second head image
Deflection angle whether within the scope of the second default security standpoint；
Precaution device, for the target object head relative to the first camera deflection angle not in the first default safety angle
Spend range in, and, the target object head relative to second camera deflection angle not in the second default security standpoint model
In the case where in enclosing, safe early warning is carried out.
2. system according to claim 1, which is characterized in that first camera and first eye tracker setting exist
The front that vehicle master drives, the second camera and second eye tracker are arranged in first camera and described second
The right side of eye tracker.
3. a kind of multi-cam safe early warning method characterized by comprising
Determine target object whether in the detection range of the first camera；
In the case where target object is in the detection range of the first camera, the first head image of target object is obtained；
According to first head image, determine the target object head relative to the first camera deflection angle whether
Within the scope of first default security standpoint；
In the case where target object is not in the detection range of the first camera, the second head image of target object is obtained；
According to second head image, determine the target object head relative to second camera deflection angle whether
Within the scope of second default security standpoint；
The target object head relative to the first camera deflection angle not within the scope of the first default security standpoint,
And deflection angle not situation second default security standpoint within the scope of of the target object head relative to second camera
Under, carry out safe early warning.
4. according to the method described in claim 3, it is characterized in that, on the target object head relative to the first camera
Deflection angle not within the scope of the first default security standpoint, and, deflection of the target object head relative to second camera
In the case that angle is not within the scope of the second default security standpoint, safe early warning is carried out, comprising:
The target object head relative to the first camera deflection angle not within the scope of the first default security standpoint,
And the target object head does not continue relative to the deflection angle of second camera within the scope of the second default security standpoint
In the case that duration reaches the first duration, early warning is carried out according to the first alarm mode；
The target object head relative to the first camera deflection angle not within the scope of the first default security standpoint,
And the target object head does not continue relative to the deflection angle of second camera within the scope of the second default security standpoint
In the case that duration reaches the second duration, early warning is carried out according to the second alarm mode；
Wherein, second duration is greater than first duration, and the intensity of second alarm mode is higher than first early warning
The intensity of mode.
5. according to the method described in claim 4, it is characterized in that, on the target object head relative to the first camera
Deflection angle not within the scope of the first default security standpoint, and, deflection of the target object head relative to second camera
In the case that angle is not within the scope of the second default security standpoint, during carrying out safe early warning, further includes:
Obtain the current speed of service of vehicle；
According to the speed of service, first duration and second duration are determined, wherein the speed of service is higher, institute
It states the first duration and second duration is shorter.
6. according to the method described in claim 3, it is characterized in that, determining the target pair according to second head image
As head relative to second camera deflection angle whether within the scope of the second default security standpoint, comprising:
Determine the angle that the head of the target object is tilted to the right；
According to the angle being tilted to the right, the second default security standpoint model corresponding with the angle being tilted to the right is obtained
It encloses；
According to second head image, determine the target object head relative to second camera deflection angle whether
Within the scope of second default security standpoint；
Wherein, the different angle being tilted to the right, corresponding the second different default security standpoint range.
7. according to the method described in claim 3, it is characterized in that, the first default security standpoint range includes: right and left
Security standpoint range in upward security standpoint range and up and down direction, the second default security standpoint range includes: a left side
Security standpoint range in right direction and the security standpoint range in up and down direction.
8. according to the method described in claim 3, it is characterized in that, determine target object whether the first camera detection
Before in range, the method also includes:
Position according to the target object face feature point relative to first camera, normally drives the target object
It is calibrated the spatial position on head when immediately ahead of sailing and being visual；
The head for controlling the target object is moved, with the default security standpoint range of determination described first.
9. a kind of vehicle characterized by comprising multi-cam safe early warning system described in any one of claims 1 to 2
System.
10. a kind of terminal device, including processor and for the memory of storage processor executable instruction, the processor
The step of any one of claim 5 to 11 the method is realized when executing described instruction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910092790.3A CN109672868B (en) | 2019-01-30 | 2019-01-30 | Multi-camera safety early warning system and method, vehicle and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910092790.3A CN109672868B (en) | 2019-01-30 | 2019-01-30 | Multi-camera safety early warning system and method, vehicle and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109672868A true CN109672868A (en) | 2019-04-23 |
| CN109672868B CN109672868B (en) | 2020-10-27 |
Family
ID=66150008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910092790.3A Active CN109672868B (en) | 2019-01-30 | 2019-01-30 | Multi-camera safety early warning system and method, vehicle and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109672868B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113923349A (en) * | 2021-09-02 | 2022-01-11 | 深圳壹秘科技有限公司 | Camera privacy protection method, device, equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1569819B1 (en) * | 2002-12-12 | 2008-10-08 | Pano Yiotis Patrikakis | Vehicle mirror system |
| DE102012222519A1 (en) * | 2011-12-09 | 2013-06-13 | Denso Corporation | DRIVING STATE DETERMINATION DEVICE |
| WO2014006909A2 (en) * | 2012-07-06 | 2014-01-09 | Yazaki Corporation | Driver condition detecting device and driver condition informing device |
| CN203651621U (en) * | 2013-12-31 | 2014-06-18 | 北京兴科迪科技有限公司 | Inner rearview mirror with fatigue driving prompting device and vehicle with the inner rearview mirror |
| CN105469035A (en) * | 2015-11-17 | 2016-04-06 | 中国科学院重庆绿色智能技术研究院 | Driver's bad driving behavior detection system based on binocular video analysis |
| CN106709420A (en) * | 2016-11-21 | 2017-05-24 | 厦门瑞为信息技术有限公司 | Method for monitoring driving behaviors of driver of commercial vehicle |
| US20170210289A1 (en) * | 2016-01-22 | 2017-07-27 | Arjun Kundan Dhawan | Driver Focus Analyzer |
| JP2018127084A (en) * | 2017-02-08 | 2018-08-16 | トヨタ自動車株式会社 | Automatic drive vehicle |
-
2019
- 2019-01-30 CN CN201910092790.3A patent/CN109672868B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1569819B1 (en) * | 2002-12-12 | 2008-10-08 | Pano Yiotis Patrikakis | Vehicle mirror system |
| DE102012222519A1 (en) * | 2011-12-09 | 2013-06-13 | Denso Corporation | DRIVING STATE DETERMINATION DEVICE |
| WO2014006909A2 (en) * | 2012-07-06 | 2014-01-09 | Yazaki Corporation | Driver condition detecting device and driver condition informing device |
| CN203651621U (en) * | 2013-12-31 | 2014-06-18 | 北京兴科迪科技有限公司 | Inner rearview mirror with fatigue driving prompting device and vehicle with the inner rearview mirror |
| CN105469035A (en) * | 2015-11-17 | 2016-04-06 | 中国科学院重庆绿色智能技术研究院 | Driver's bad driving behavior detection system based on binocular video analysis |
| US20170210289A1 (en) * | 2016-01-22 | 2017-07-27 | Arjun Kundan Dhawan | Driver Focus Analyzer |
| CN106709420A (en) * | 2016-11-21 | 2017-05-24 | 厦门瑞为信息技术有限公司 | Method for monitoring driving behaviors of driver of commercial vehicle |
| JP2018127084A (en) * | 2017-02-08 | 2018-08-16 | トヨタ自動車株式会社 | Automatic drive vehicle |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113923349A (en) * | 2021-09-02 | 2022-01-11 | 深圳壹秘科技有限公司 | Camera privacy protection method, device, equipment and storage medium |
| CN113923349B (en) * | 2021-09-02 | 2024-06-11 | 深圳壹秘科技有限公司 | Camera privacy protection method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109672868B (en) | 2020-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200334477A1 (en) | State estimation apparatus, state estimation method, and state estimation program | |
| JP6364627B2 (en) | Gaze direction detection device and gaze direction detection method | |
| EP2649932A1 (en) | Method for detecting point of gaze and device for detecting point of gaze | |
| CN109791739A (en) | Carsick estimation device, carsick anti-locking apparatus and carsick estimation method | |
| US20150173665A1 (en) | State estimation device and state estimation program | |
| JP2002505134A (en) | Exercise pattern evaluation method and apparatus | |
| JP2008067362A (en) | Image processor and image processing method | |
| US10969579B2 (en) | Augmented reality glasses, method for determining a pose of augmented reality glasses, and transportation vehicle suitable for using the augmented reality glasses or the method | |
| WO2019176492A1 (en) | Calculation system, information processing device, driving assistance system, index calculation method, computer program, and storage medium | |
| JP2018048839A (en) | Three-dimensional data generator, three-dimensional data generation method, and monitoring system equipped with three-dimensional data generator | |
| JP6762157B2 (en) | Methods and devices for predicting the direction of the occupant's line of sight | |
| CN110448312A (en) | Data processing equipment, wakes up system, data processing method at monitoring system | |
| CN110263615A (en) | Interaction processing method, device, equipment and client in vehicle shooting | |
| CN110525444A (en) | A kind of processing method and processing device for driver's body abnormality situation | |
| CN106101857B (en) | A kind of display methods and device of video pictures | |
| CN109672868A (en) | Multi-cam safety pre-warning system, method, vehicle and terminal device | |
| US11810372B2 (en) | Multiple in-cabin cameras and lighting sources for driver monitoring | |
| CN109859438A (en) | Safe early warning method, system, vehicle and terminal device | |
| JP2019028640A (en) | Visual line detection device | |
| JP2011159214A (en) | Action detection device | |
| JP7073769B2 (en) | Driving support system and driving support method | |
| CN110159495A (en) | Blower fan tower barrel method for early warning, apparatus and system | |
| CN113049264B (en) | Test system and method for advanced driving assistance system of vehicle | |
| JP2009125518A (en) | Driver's blink detection method, driver's awakening degree determination method, and device | |
| JP2012011810A (en) | Face image pickup apparatus for vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |