Drowsiness Detection Using Opencv Final

DROWSINESS DETECTION USING OPENCV
A project report
Submitted in partial fulfillment of the requirements for the award of the degree of
BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE & ENGINEERING
Submitted By
P.DURGA PRASADU 176R1A0525
B.AMANI 176R1A0505
J.TEJASSIE 176R1A0514
K.SIVA GANGA KRISHNA 176R1A0517
Under the esteemed guidance of
Mr. N. Rakesh Kumar , M.Tech.,

Assistant Professor,
Department of CSE.
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING
GIET COLLEGE OF ENGINEERING

(Approved by A.I.C.T.E, Affiliated to JNTU, Kakinada)
NH-16,Chaitanya Knowledge City,Rajamahendravaram,E.G.Dist,A.P,Pin-533233
(2021)
GIET COLLEGE OF ENGINEERING
(Approved by A.I.C.T.E, Affiliated to JNTU, Kakinada)
NH-16,Chaitanya Knowledge City,Rajamahendravaram,E.G.Dist,A.P,Pin-533233
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING
CERTIFICATE
This is to certify that the project entitled “DROWSINESS DETECTION USING
OPENCV” is being carried
By
B.AMANI 176R1A0505
J.TEJASSIE 176R1A0514
In partial fulfillment of the requirements for the award of the degree BACHELOR OF
TECHNOLOGY in COMPUTER SCIENCE AND ENGINEERING, is record of
bonafide work carried out by them under my guidance and supervision.
The results embodied in this project have not submitted to any other university or
institution for the award of any degree or diploma.
Internal Guide Head of the Department

Mr. N. Rakesh Kumar , M.Tech Dr. SK.MEERA SHARIEF, Ph.D.
Assistant Professor Professor & HOD-CSE
External Examiner
i
ACKNOWLEDGEMENT
We take immense pleasure in conveying our thanks and deep sense of gratitude to our Guide
Mr. N. RAKESH KUMAR for his exhilarating supervision, timely suggestions and encouragement
during all phases of this work.
We wish to thank with deep sense of acknowledge to the management of GIET College of
engineering and to our principal, Dr. S.SRI GOWRI REDDY for extending all facilities.
We wish to express our sincere thanks to Dr. SK.MEERA SHARIEF, Head of the
Department of Computer Science and Engineering for his excellent encouragement in course of this
work.
We wish to express our sincere thanks to Mr. SREENIVASU, Academic and Project
Coordinator for his excellent encouragement in course of his work .
We would like to thank all the teaching and non-teaching faculty members of our department
for advising us whenever in need, co-operating with us and arranging the necessary facilities.
We would like to convey gratitude to our Parents whose blessings were always with us. Last
but not the least we would like to thank our Friends and Others who directly or indirectly helped
us in successful completion of this work.
B.AMANI 176R1A0505
J.TEJASSIE
176R1A0514
ii
DECLARATION
We hereby declare that the project entitled “DROWSINESS DETECTION USING

OPENCV” has been undertaken by us and this work has been submitted to GIET COLLEGE OF
ENGINEERING affiliated to JNTU Kakinada in partial fulfillment of the requirements for the
award of the degree of Bachelor of Technology in COMPUTER SCIENCE & ENGINEERING.
PLACE:RAJAMAHENDRAVARAM
DATE:
B.AMANI 176R1A0505
J.TEJASSIE
176R1A0514
iii
INDEX
S.NO CONTENTS PAGE NO
CERTIFICATES i
ACKNOWLEDGEMENT ii
DECLARATION iii
LIST OF FIGURES iv
LIST OF TABLES v
ABSTRACT vi
1. INTRODUCTION 1
1.1 PROJECT PURPOSE 1
1.2 PROBLEM STATEMENT 3
1.3 PROJECT SCOPE 4
1.4 PROJECT OBJECTIVES 4
2. LITERATURE SURVEY 5
2.1 RESEARCH OF STUDIES 5
2.2 EXISTING SYSTEM 7
2.1.1 DISADVANTAGES 7
2.3 FEASIBILITY STUDY 7
2.3.1 ECONOMIC FEASIBILITY 8
2.3.2 TECHNICAL FEASIBILITY 8
2.3.3 SOCIAL FEASIBILITY 8
3. SOFTWARE REQUIREMENT & ANALYSIS 9

3.1 FUNCTIONAL REQUIREMENTS 9
3.2 NON-FUNCTIONAL REQUIREMENTS 9

3.3 HARDWARE REQUIREMENTS 10
3.4 SOFTWARE REQUIREMENTS 10
3.5 TECHNOLOGIES USED 11
3.5.1 PYTHON 11
3.5.2 OPENCV 12
3.5.3 CONVOLUTIONAL NEURAL NETWORKS(CNN) 16
3.5.4 SPYDER IDE 17
3.5.5 TWILIO 18
3.5.6 PYGAME.MIXER 19
3.5.7 HAAR CASCADE CLASSIFIER IN OPENCV 20
3.5.8 EYE ASPECT RATIO 21
3.5.9 68 FACE LANDMARK DETECTOR 23
3.5.10 LIBRARIES IN PYTHON 24
3.5.11 DLIB 26
3.6 DEVELOPMENT ENVIRONMENT 27
3.6.1 PYTHON ENVIRONMENT SETUP 27
3.6.2 SPYDER IDE INSTALLATION 31
4. SOFTWARE DESIGN 33
4.1 FACE RECOGNITION 33
4.2 CASCADE CLASSIFIER 36
4.3 UML DIAGRAMS 38
4.3.1 USE CASE DIAGRAM 38
4.3.2 CLASS DIAGRAM 40
4.3.3 SEQUENCE DIAGRAM 42
4.3.4 ACTIVITY DIAGRAM 43
4.4 CONTROL FLOW DIAGRAM 45

5. PROPOSED SYSTEM 46
5.1 PROPOSED METHODOLOGY 46
5.2 SYSTEM MODULES 47
5.3 ADVANTAGES 51
6. CODING 52
6.1 DIRECTORIES 52
6.2 SOURCE CODE 53
6.2.1 DROWSINESS_DETECT.PY 53
6.2.2 FACE_AND_EYE_DETECTOR_WEBCAM_VIDEO.PY 58
7. TESTING AND VALIDATIONS 59

7.1 INTRODUCTION 59
7.2 TESTING AND VALIDATION 59
7.2.1 UNIT TESTING 59
7.2.2 INTEGRATION TESTING 59
7.2.3 FUNCTIONAL TESTING 60
7.2.4 SYSTEM TESTING 60
7.2.5 WHITE BOX TESTING 61
7.2.6 BLACK BOX TESTING 61
7.2.7 ACCEPTANCE TESTING 61
7.3 TEST RESULTS 62
8. OUTPUT SCREENS 63
9. CONCLUSION AND FUTURE WORK 66
10 . REFERENCES 68
10.1 BOOKS
10.2 PAGES
10.3 WEBSITES
LIST OF FIGURES
S.NO FIG NO FIG NAME PAGE NO
1. 1.1.1 THE STATISTIC OF ROAD ACCIDENTS OF 1

ASIAN NATIONS IN 2013 AND 2016
2. 1.1.2 EXAMPLE OF DROWSINESS CONDITION 2
WHILE DRIVING
3. 1.2.1 DROWSINESS 3
4. 3.5.2.1 OPENCV LIBRARY 12
5. 3.5.2.2 PYTHON WITH OPENCV 13
6. 3.5.2.3.1 PARTS OF OPENCV 14
7. 3.5.3.1 CNN LAYERS 17
8. 3.5.5.1 TWILIO REGISTRATION PAGE 18
9. 3.5.5.2 TWILIO INSTALLALATION 19
10. 3.5.8.1 EYE ASPECT RATIO FORMULA 21
11. 3.5.8.2 INDEXES OF EYE 22
12. 3.5.9.1 VISUALIZING THE 68 FACIAL 23

LANDMARK COORDINATES FROM
THE IBUG 300-W DATASET
13. 3.6.1.1.1 PYTHON TOOL FOR DOWNLOAD 27
14. 3.6.1.1.2 DIFFERENT VERSIONS DOWNLOAD 27

PYTHON FOR WINDOWS 64 BIT
15. 3.6.1.1.3 PYTHON INSTALLATION WIZARD1 28

S.NO FIG NO FIG NAME PAGE NO
19 3.6.2.2.1 ANACONDA NAVIGATOR SHOWING 30

SPYDER
20. 4.1.1.1 FACE RECOGNITION 34
19. 4.3.1.1 USECASE DIAGRAM DROWSINESS 38

DETECTION USING OPENCV
20. 4.3.2.1 CLASS DIAGRAM FOR DROWSINESS 40

21. 4.3.3.1 SEQUENCE DIAGRAM FOR DROWSINESS 41

22. 4.3.4.1 ACTIVITY DIAGRAM FOR DROWSINESS 42

25. 4.4.1 CONTROL FLOW DIAGRAM 43
26. 5.2.1.1 HAAR CASCADE FEATURES 45
27. 5.2.3.1.1 CONVERSION OF COLOORED IMAGE

TO GRAYSCALE
28. 6.1.1 PROJECT DIRECTORIES
29. 8.1.1 VIDEO FEED AND DRAWING

CIRCLE AROUND EYES
30. 8.1.2 DROWSY SCREEN ALERT
31. 8.1.3 ALERT MESSAGEIN PHONE
iv
LIST OF TABLES
S.NO TABLE NO TABLE NAME PAGE NO
1. 1 TEST CASES 60
v
ABSTRACT
Many of the accidents occur due to drowsiness of drivers. It is one of the critical causes of
roadways accidents now-a-days. Latest statistics say that many of the accidents were
caused because of drowsiness of drivers. Vehicle accidents due to drowsiness in drivers are
causing death to thousands of lives. More than 30% accidents occur due to drowsiness. For
the prevention of this, a system is required which detects the drowsiness and alerts the
driver which saves the life. A computer vision based thoughts have been used for the
creation of a Drowsy Driver Detection System. The little camera has been utilized by
framework that concentrates straight towards the essence of driver and checks the driver's
eyes with a particular ultimate objective to perceive weakness. A notice sign is issued to
alert the driver, in such circumstance when exhaustion is perceived. The framework
oversees using information picked up for the picture to find the facial tourist spots, which
gets the area where the eyes of an individual may exist. On the off chance that the eyes of
driver are discovered close for a specific measure of casings, the proposed framework
accept that the driver is falling asleep and an alarm of caution has been issued. The
structure can work just when the eyes are found, and works in encompassing lighting
conditions too. In this, the driver is continuously monitored through webcam. This model
uses image processing techniques which mainly focuses on face and eyes of the driver. The
model extract the drivers face and predicts the blinking of eye from eye region. We use an
algorithm to track and analyze drivers face and eyes to measure eye aspect ratio. If the
blinking rate is high then the system alerts the driver with a sound.
vi
1. INTRODUCTION
1.1 PROJECT PURPOSE
Drowsiness is the state where person have strong desire to sleep. It is the state where person
has the high desire to sleep. It has two definite meanings, referring both to the state foregoing
falling asleep and chronic condition referring to being in that state independent of the daily
rhythm. While performing the tasks that needed the constant concentration such as driving can
be dangerous if it is done in a drowsiness state. Person can experienced drowsiness if they had
a sufficient fatigue and this can leads to road accidents.
FIGURE 1.1.1 : THE STATISTIC OF ROAD ACCIDENTS OF ASIAN NATIONS IN 2013

AND 2016
The major challenge in the field of accident avoidance system faced by the developer and
researcher is in developing the technologies for detecting or preventing drowsiness among the
drivers. Solutions need to be done in order to counteract the presence hazards of drowsiness
on a road.
GIET COLLEGE OF ENGINEERINGPage 1

The purpose of this project is to develop the simulation of drowsiness detection system. The focus
of the project is to design a system that will detect the drowsiness by detecting the closed eyes
of the driver. By monitoring the state of the eyes, it is believed can detect the early symptom
of the driver’s drowsiness, to avoid car accidents. The process of detecting the drowsiness
between drivers is to detect the open and closed of the eyes.
FIG 1.1.2 : EXAMPLE OF DROWSINESS CONDITION WHILE DRIVING
Sometimes, people needs an alertness and warn to prevent them from sleeping while driving.
There are plenty driver drowsiness detection system were developed by many researcher. One
of the example is the use a computer vision or OpenCV to detect image or video processing.
This alternative will help developer to developed computer vision based system.
The purpose of the project is to design the simulation of drowsiness detection system. The system
will accurately monitor the state of driver’s eyes whether it is open or closed. The road
accidents believed can be avoided by monitoring the early stage of drivers’ drowsiness.
Face recognition is one of the application besides human identification and tracking for security
systems becomes popular research area by analyzing the face images. This project is focused
on 3 the localization of the eyes, which involves the both eyes and face by applying the
existed image processing algorithm. The detection of the drowsiness will be determined once
the position of the eyes located.

1.2 PROBLEM STATEMENT
FIG 1.2.1 DROWSINESS
Current drowsiness detection systems monitoring the driver’s condition requires complex
computation and expensive equipment, not comfortable to wear during driving and is not
suitable for driving conditions; for example, Electroencephalography (EEG) and
Electrocardiography (ECG), i. e. detecting the brain frequency and measuring the rhythm of
heart, respectively.
A drowsiness detection system which use a camera placed in front of the driver is more suitable to
be use but the physical signs that will indicate drowsiness need to be located first in order to
come up with a drowsiness detection algorithm that is reliable and accurate. Lighting intensity
and while the driver tilt their face left or right are the problems occur during detection of
eyes .
Therefore, this project aims to analyze all the previous research and method, hence propose a
method to detect drowsiness by using video or webcam. It analyzes the video images that
have been recorded and come up with a system that can analyze each frame of the video.

1.3 PROJECT SCOPE
In the project, the main focused will be based on :

• Concept of drowsiness detection system.
• Determine drowsiness from these parameters:
a) Blink eye.
b) Eye of Ratio (EAR).
• Coding development and testing.
• Complete testing and improvement
1.4. PROJECT OBJECTIVES
The project focuses on these objectives,which are:

 To suggest ways to detect fatigue and drowsiness while driving.
 To study on eyes from the video images of participants in the experiment of driving
simulation conducted by MIROS that can be used as an indicator of fatigue and
drowsiness.
 To investigate the physical changes of fatigue and drowsiness.
 To develop a system that use eyes closure as a way to detect drowsiness.

2. LITERATURE SURVEY
There are many research papers that focusing on detecting the drowsiness among the
drivers that developer can refer in order to develop the drowsiness detection system.
Literature review focusing on what have been done by other authors.
2.1 RESEARCH OF STUDIES
2.1.1 AUTOMATIC DRIVER DROWSINESS DETECTON AND ACCIDENT

PREVENTION SYSTEM USING IMAGE PROCESSING
In this article, sleep is a primary human need. Basically, human needs to sleep at least eight
hours a day by schedule for the day. If someone needs adequate relief the frame does not have
a proper characteristic (function). The first to do so drivers all need enough sleep to boost
their work. If we don't sleep enough, we might get drowsy and fall asleep. A few seconds of
drowsiness will also cost lives for both drivers and travelers. This system gives you a vibrant
monitoring assistant who analyses blinking of the drivers' eyes action .
2.1.2 UNDERSTANDING OF A CONVOLUTIONAL NEURAL NETWORK
In this paper, deep learning or deep neural network refers to Artificial Neural Networks
(ANN) with multiple layers. It has been considered one of the most recent decades. Most
efficient devices, and has been very popular literature as it is capable of processing a large
volume of data. This is the interest in having deeper hidden layers has recently begun. Surpass
the performance of classical methods in different fields; especially in recognition of patterns.
One of the deepest, most popular the neural networks are the Convolutional Neural Network
(CNN). Taking its name from the mathematical linear operation between matrixes are called
convolution. CNN has several layers, like a convolutionary layer, a non-linear layer, a pooling
layer and a completely connected layer. Convolutionary and fully connected layers have
parameters but there are no pooling and non-linear layers its’ parameters. CNN has an
outstanding computer efficiency things of learning. Applications dealing with, in particular,

image data, such as the largest image classification data set (Image Net), computer vision and
natural language processing (NLP) and the results obtained have been very wonderful. This
paper discusses and describes all the elements and issues of significance related to CNN, and
how it works. In addition, we are this will also state the parameters that have an impact on
CNN performance. This paper assumes that readers have sufficient knowledge of both
machine learning and artificial neural networks.
2.1.3 A SURVEY PAPER ON DROWSINESS DETECTION & ALARM SYSTEM

FOR DRIVERS
In this research paper, our safety is the first priority while traveling or driving. A driver's
mistake can lead to a serious mistake. Physical injury, mortality and major economic losses.
There are several applications available on the market today, such as navigation devices,
multiple sensors, etc. Easy to work. There are numerous reasons, particularly human’s faults
that give rise to road accidents. Studies say that there is a huge rise in the number of road
accidents in our country has been in existence for the last few years. The key explanation that
is happening on the highway injuries are somnolence and sleepiness driver as he's driving.
2.1.4 REAL-TIME DROWSY DRIVER DETECTION USING HAAR CASCADE

SAMPLES
In this paper, with population increase, car injuries have also occurred growth. A thorough
study reveals that there are over half of a million incidents in India per year. However, owing
to driver exhaustion, about 60 percent of such injuries are induced. Tired rider influences
driving skill in the following 3 regions, (a) hinder balance, (b) affect slower driving abilities
reaction cycles, and, c) Perception loss. We have a real-time via this paper monitoring device
of image recognition technologies, facial / head identification. Even, to ensure real-time
computation, Haarcascade tests are used to discern between a blink of the eye and detection in
drowsiness / exhaustion.

2.2 EXISTING SYSTEM
 There are various methods like detecting objects which are near to vehicle and front and rear
cameras for detecting vehicles approaching near to vehicle and airbag system which can save
lives after an accident is accorded.
 In existing systems ,there are some techniques to detect condition of a person by calculating
the brain frequency and heart rate.
2.2.1 DISADVANTAGES
 Most of the existing systems use external factors and inform the user about the problem and
save users after an accident is accord but from research most of the accidents are due to faults
in users like drowsiness, sleeping while driving.
 In existing systems , to detect drowsiness it uses complex methods.
 Existing methods are not comfortable while driving.
2.4 FEASIBILITY STUDY
The feasibility of the project is analyzed in this phase and business proposal is put forth
with a very general plan for the project and some cost estimates. During system analysis the
feasibility study of the proposed system is to be carried out. This is to ensure that the proposed
system is not a burden to the company. For feasibility analysis, some understanding of the
major requirements for the system is essential
Three key considerations involved in the feasibility analysis are
 ECONOMICAL FEASIBILITY
 TECHNICAL FEASIBILITY
 SOCIAL FEASIBILITY

2.4.1 ECONOMICAL FEASIBILITY
This study is carried out to check the economic impact that the system will have on the
organization. The amount of fund that the company can pour into the research and
development of the system is limited. The expenditures must be justified. Thus the developed
system as well within the budget and this was achieved because most of the technologies used
are freely available. Only the customized products had to be purchased.
2.4.2 TECHNICAL FEASIBILITY
This study is carried out to check the technical feasibility, that is, the technical requirements
of the system. Any system developed must not have a high demand on the available technical
resources. This will lead to high demands on the available technical resources. This will lead
to high demands being placed on the client. The developed system must have a modest
requirement, as only minimal or null changes are required for implementing this system.
2.4.3 SOCIAL FEASIBILITY
The aspect of study is to check the level of acceptance of the system by the user. This
includes the process of training the user to use the system efficiently. The user must not feel
threatened by the system, instead must accept it as a necessity. The level of acceptance by the
users solely depends on the methods that are employed to educate the user about the system
and to make him familiar with it. His level of confidence must be raised so that he is also able
to make some constructive criticism, which is welcomed, as he is the final user of the system.

3. SOFTWARE REQUIREMENT ANALYSIS
A Software Requirements Specification (SRS) is a detailed description of a software

system to be developed with its functional and non-functional requirements. The SRS is
developed based the agreement between customer and contractors. It may include the use
cases of how user is going to interact with software system. The software requirement
specification document consistent of all necessary requirements required for project
development. To develop the software system we should have clear understanding of
Software system. To achieve this we need to continuous communication with customers to
gather all requirements.
A good SRS defines the how Software System will interact with all internal
modules, hardware, communication with other programs and human user interactions
with wide range of real life scenarios. Using the Software requirements specification
(SRS) document on QA lead, managers creates test plan. It is very important that testers
must be cleared with every detail specified in this document in order to avoid faults in
test cases and its expected results. It is highly recommended to review or test SRS
documents before start writing test cases and making any plan for testing.
3.1 FUNCTIONAL REQUIREMENTS
These are the requirements that the end user specifically demands as basic facilities
that the system should offer. All these functionalities need to be necessarily incorporated
into the system as a part of the contract. These are represented or stated in the form of input
to be given to the system, the operation performed and the output expected. They are
basically the requirements stated by the user which one can see directly in the final product,
unlike the non-functional requirements.
3.2 NON- FUNCTIONAL REQUIREMENTS

These are basically the quality constraints that the system must satisfy according to

the project contract. The priority or extent to which these factors are implemented varies
from one project to other. They are also called non-behavioral requirements. They basically
deal with issues like Portability, Security, Maintainability, Reliability, Scalability,
Performance, Reusability, Flexibility.
3.2 HARDWARE REQUIREMENTS
For developing the project the following are the Hardware requirements:
 System : Intel i3 or i5 or i7 processor based system
 Hard Disk : 1 TB
 Monitor : 14 or 15.6 inches Colour Monitor
 Mouse : Optical Mouse
 RAM : 4 GB
 Webcam : Standard web camera
 Keyboard : Standard keyboard
3.4 SOFTWARE REQUIREMENTS
The recommended software specifications to install and effectively operate Drowsiness

detection using opencv includes :
 Coding Language : Python 3.8 or 3.9

 Operating System : Windows 7 or 10
 Tools : Anaconda, Spyder ide
 Libraries : OpenCV, Twilio and pygame.mixer
 Algorithm : Convolutional neural networks(CNN)
 Data set : Haar cascade classifier, 68 face landmark detector

3.5 TECHNOLOGIES USED
3.5.1 PYTHON
3.5.1.1 INTRODUCTION TO PYTHON
PYTHON language is used to code a nonlinear clustering algorithm and implement it

on a dataset to form clusters to get similar groups of data. Python is a general-purpose
interpreted, interactive, object-oriented, and high-level programming language. It was created
by Guido van Rossum during 1985- 1990. Like Perl, Python source code is also available
under the GNU General Public License (GPL). This tutorial gives enough understanding on
Python programming language. Python is a high-level, interpreted, interactive and object-
oriented scripting language. Python is designed to be highly readable. It uses English
keywords frequently where as other languages use punctuation, and it has fewer syntactical
constructions than other languages
 Python is interpreted : Python is processed at runtime by the interpreter. You do

not need to compile your program before executing it. This is similar to PERL and PHP.
 Python is Interactive : You can actually sit at a Python prompt and interact with
the interpreter directly to write your programs.
 Python is Object-Oriented : Python supports Object-Oriented style or technique of
programming that encapsulates code within objects.
 Python is a Beginner's Language: Python is a great language for the beginner-level
programmers and supports the development of a wide range of applications from simple text
processing to WWW browsers to games
.
3.5.1.2 PYTHON FEATURES
Python's features include:

 Easy-to-learn: Python has few keywords, simple structure, and a clearly defined syntax. This
allows the student to pick up the language quickly.
 Easy-to-read: Python code is more clearly defined and visible to the eyes.
 Easy-to-maintain: Python's source code is fairly easy-to-maintaining.
 Interactive Mode: Python has support for an interactive mode which allows interactive
testing and debugging of snippets of code.
 Portable: Python can run on a wide variety of hardware platforms and has the same interface
on all platforms.
 Extendable: You can add low-level modules to the Python interpreter. These modules enable
programmers to add to or customize their tools to be more efficient.
 Databases: Python provides interfaces to all major commercial databases.
 A broad standard library: Python's bulk of the library is very portable and cross- platform
compatible on UNIX, Windows, and Macintosh.
3.5.2 OPENCV
OpenCv is predominantly a technique for real time image processing which has free of cost
implementations on latest computer vision algorithms. It has all required computer vision
algorithms.

FIG 3.5.2.1. OPENCV LIBRARY
OpenCV is an open source computer vision library accessible in python coding language to
code for visionary capabilities of our smart pc.
OpenCV was expected for computational capability and having a high focus on
ongoing picture location and distinguishing proof. OpenCV is coded with streamlined C and
can take work with multicore processors. If we need progressively programmed improvement
utilizing Intel models [Intel], you can purchase Intel's Integrated Performance Primitives (IPP)
libraries [IPP]. These comprise of low-level schedules in different algorithmic regions which
are streamlined. OpenCV consequently utilizes the IPP library, at runtime if that library is
introduced.
FIG 3.5.2.2 : PYTHON WITH OPENCV
3.5.2.1 THE COMPUTER’S VISION
PC's vision is the change of information from a still, or camcorder into either a depiction or
another choice. Each and every such changes are performed to achieve a particular target. A
Computer gains a cross section of numbers from a camera or from the circle, and it's just as
simple as that. For the most part, there is no worked in example acknowledgment or
programmed control of center and gap, no cross-relationship with long periods of experience.
Generally, vision frameworks are still reasonably gullible.
3.5.2.2 THE ORIGIN OF OPENCV

OpenCV left an Intel Research action proposed to drive CPU-raised applications.Toward

this end, Intel moved various endeavors that included constant beam following and moreover
3D show dividers. One of the product engineers working for Intel at the time was visiting
schools. He saw that several top school social affairs, like the MIT Media Lab, used to have
well-made similarly as inside open PC vision frameworks—code which was supplied starting
with one understudy then onto the next and which gave each resulting understudy an
important establishment while building up his own vision application. Rather than rehashing
the fundamental capacities from starting, another understudy may begin by adding to that
which preceded .
3.5.2.3 OPENCV STRUCTURE AND CONTENT
OpenCV left an Intel Research movement planned to drive CPU-raised applications.

Toward this end, Intel pushed various endeavors that included continuous beam following and
moreover 3D show dividers. One of the product engineers working for Intel at the time was
visiting schools. He saw that two or three top school social events, like the MIT Media Lab,
used to have well-made similarly as inside open PC vision foundations

FIG 3.5.2.3.1 : PARTS OF OPENCV
3.5.2.4 WHY OPEN CV?
3.5.2.4.1 SPECIFIC
OpenCV was planned for picture handling. Each capacity and information structure has
been arranged in perspective on an Image Processing application. Then, Matlab, is very
conventional. You can get almost everything on the planet by methods for tool compartments.
It may be money related tool stash or then again concentrated DNA tool compartments .
3.5.2.4.2 SPEEDY
Matlab is just excessively moderate. Matlab itself depended on Java. Similarly Java
depended on C. So when we run a Matlab program, our PC gets caught up with attempting to
translate and assemble all that convoluted Matlab code. At that point it is transformed into
Java, lastly executes the code. In case we use C/C++, we don't waste such time. We direct
give machine language code to the PC, and it gets executed. So in the end we get more picture
taking care of, and not additionally interpreting. Ensuing to doing some constant picture
handling with both Matlab and OpenCV, we typically got low speeds, a point of confinement
of around 4-5 outlines arranged each second with Matlab. With OpenCV in any case, we get
genuine persistent dealing with at around 30 outlines being handled every second. 22 Beyond
any confusion we give the prize for speediness –a progressively enigmatic language to handle,
yet it's unquestionably of true worth .We can complete a lot more work, as calculate some
extremely perplexing arithmetic on pictures utilizing C and still pull off adequate speeds for
your application.
3.5.2.4.3 EFFICIENT
Matlab utilizes just an excessive amount of system assets. With OpenCV, we can pull off
as pitiful as 10mb RAM for a constant application. Not with standing the way that with the
present PCs, the RAM factor is surely not a noteworthy thing to be worried over.
In any case, our tiredness identification framework is to be used inside a vehicle in a way that
is non-meddlesome and little; so a low handling necessity is vital. Subsequently we shall
perceive as to how OpenCV is superior for a real-time drowsiness detection system .
3.5.3 CONVOLUTIONAL NEURAL NETWORKS (CNN)
In deep learning, a convolutional neural network (CNN, or Convent) is a class of deep

neural network, most commonly applied to analyze visual imagery. They are also known as
shift invariant or space invariant artificial neural networks (SIANN), based on the shared-
weight architecture of the convolution kernels or filters that slide along input features and
provide translation equivariant responses known as feature maps. Counter-intuitively, most
convolutional neural networks are only equivariant, as opposed to invariant, to translation.
They have applications in image and video recognition, recommender systems, image
classification, image segmentation, medical image analysis, natural language processing,
brain-computer interfaces, and financial time series.
CNNs are regularized versions of multilayer perceptron’s. Multilayer perceptron’s usually

mean fully connected networks, that is, each neuron in one layer is connected to all neurons in
the next layer. The “full connectivity” of these networks makes them prone to overfitting data.
Typical ways of regularization, or preventing overfitting, include: penalizing parameters
during training (such as weight decay) or trimming connectivity (skipped connections,
dropout, etc.) CNNs take a different approach towards regularization: they take advantage of
the hierarchical pattern in data and assemble patterns of increasing complexity using smaller
and simpler patterns embossed in their filters.
Therefore, on a scale of connectivity and complexity, CNNs are on the lower extreme.
Convolutional networks were inspired by biological processes in that the connectivity pattern
between neurons resembles the organization of the animal visual cortex. Individual cortical
neurons respond to stimuli only in a restricted region of the visual field known as the
receptive field. The receptive fields of different neurons partially overlap such that they cover
the entire visual field. CNNs use relatively little pre-processing compared to other image
classification algorithms. This means that the network learns to optimize the filters (or
kernels) through automated learning, whereas in traditional algorithms these filters are hand-
engineered. This independence from prior knowledge and human intervention in feature
extraction is a major advantage.
FIG 3.5.3.1 : CNN LAYERS
3.5.4 SPYDER IDE:
Spyder is an open-source cross-platform integrated development environment (IDE) for

scientific programming in the Python language. Spyder integrates with a number of prominent
packages in the scientific Python stack, including NumPy, SciPy, Matplotlib, pandas, IPython,
SymPy and Cython, as well as other open-source software. It is released under the MIT
license.
Initially created and developed by Pierre Raybaut in 2009, since 2012 Spyder has been
maintained and continuously improved by a team of scientific Python developers and the
community.
Spyder is extensible with first-party and third-party plugins, includes support for interactive

tools for data inspection and embeds Python-specific code quality assurance and introspection
instruments, such as Pyflakes, Pylint and Rope. It is available cross-platform through
Anaconda, on Windows, on macOS through MacPorts, and on major Linux distributions such
as Arch Linux, Debian, Fedora, Gentoo Linux, openSUSE and Ubuntu.
Spyder uses Qt for its GUI and is designed to use either of the PyQt or PySide Python
bindings. QtPy, a thin abstraction layer developed by the Spyder project and later adopted by
multiple other packages, provides the flexibility to use either backend.
3.5.5 TWILIO :
Twilio Messaging is an API to send and receive sms, mms messages globally. It uses
intelligent sending features to ensure messages reliably reach end users wherever they are.
The Twilio Python Helper Library makes it easy to interact with the Twilio API from
our Python application.Applications are simple to build and scalable.
The Twilio Python Helper Library makes it easy to interact with the Twilio API from your
Python application. The most recent version of the library can be found on PyPi. The Twilio
Python Helper Library supports Python applications written in Python 2.7 and above. If you
are interested in migrating to the newer 6.x version of the Twilio Python Helper Library from
the 5.x version, check out this guide.

FIG 3.5.5.1 : TWILIO REGISTRATION PAGE
3.5.1 INSTALL THE LIBRARY :

The easiest way to install the library is from PyPi using pip, a package manager for Python.
Simply run this in the terminal:
pip install twilio
FIG 3.5.5.2 : TWILIO INSTALLALATION
If you get a pip: command not found error, you can also use easy_install. Run this in your
terminal:
easy_install twilio
3.5.2 MANUAL INSTALLATION
you can download the source code (ZIP) for twilio-python, and then install the library by
running:
python setup.py install

3.5.6 PYGAME.MIXER:
Game programming is very rewarding nowadays and it can also be used in advertising and as
a teaching tool too. Game development includes mathematics, logic, physics, AI and much
more and it can be amazingly fun. In python, game programming is done in pygame and it is
one of the best modules for doing so.
In order to play music/audio files in pygame, pygame.mixer is used (pygame module for
loading and playing sounds). This module contains classes for loading Sound objects and
controlling playback. There are basically four steps in order to do so:
Starting the mixer : mixer.init()
Loading the song : mixer.music.load("song.mp3")
Setting the volume : mixer.music.set_volume(0.7)
Start playing the song : mixer.music.play()
3.5.7 HAAR-CASCADE DETECTION IN OPENCV:
Object Detection using Haar feature-based cascade classifiers is an effective method proposed
by Paul Viola and Michael Jones in the 2001 paper, "Rapid Object Detection using a Boosted
Cascade of Simple Features". It is a machine learning based approach in which a cascade
function is trained from a lot of positive and negative images. It is then used to detect objects
in other images.
Here we will work with face detection. Initially, the algorithm needs a lot of positive images
(images of faces) and negative images (images without faces) to train the classifier. Then we
need to extract features from it.
Here we will deal with detection. OpenCV already contains many pre-trained classifiers for
face, eyes, smile etc. Those XML files are stored in opencv/data/haarcascades/ folder. Let's
create a face and eye detector with OpenCV.
We use the function:

detectMultiScale (image, objects, scaleFactor = 1.1, minNeighbors = 3, flags = 0, minSize =
new cv.Size(0, 0), maxSize = new cv.Size(0, 0))
3.5.7.1 PARAMETERS
Image : Matrix of the type CV_8U containing an image where objects are detected.
Objects : Vector of rectangles where each rectangle contains the detected object. The
rectangles may be partially outside the original image.
ScaleFactor : Parameter specifying how much the image size is reduced at each image
scale.
MinNeighbors : Parameter specifying how many neighbors each candidate rectangle should
have to retain it.
Flags :Parameter with the same meaning for an old cascade as in the function
CvHaarDetectObjects : It is not used for a new cascade.
MinSize : Minimum possible object size. Objects smaller than this are ignored.
MaxSize : Maximum possible object size. Objects larger than this are ignored. If .
maxSize == minSize model is evaluated on single scale
3.5.8 EYE ASPECT RATIO
The Eye Aspect Ratio is an estimate of the eye opening state. After detecting the face of
the driver, the calculation of drowsiness level of the driver is based on eye blink rate. The Eye

Aspect Ratio (EAR) formula, which was proposed is able to detect the eye blink using the
scalar value.
For instance, if driver blinks eyes more frequently, it means that the drivers are in the state of
drowsiness. Thus, it is necessary to detect the eyes shape accurately in order to calculate the
eye blink frequency. From the landmarks detected in the image with face, the EAR is used as
an estimate of the eye openness state. For every video frame, the eye landmarks are detected
between height and width of the eye that had been computed. The eye aspect ratio can be
defined by the Equation
FIG 3.5.8.1 : EYE ASPECT RATIO FORMULA
It shows the eye aspect ratio formula where p1 until p6 are the 2D landmark locations. The
p2, p3, p5 and p6 are used to measure the height whereas p1 and p4 are used to measure width
of the eyes in meter (m)
The eye aspect ratio is a constant value when the eye is opened, but rapidly falls
approximately to 0 when the eye is closed
The values of P will be calculated by the

FIG 3.5.8.2 : INDEXES OF EYE
We observe the aspect ratio of the eye remains constant for a period of time indicating that the
eye was open, then it falls rapidly to zero and then increases again which indicates the person
blinked .
We will be observing this eye aspect ratio in our drowsiness detector case to see if the value
remains constant or falls to zero but not increases again implying that the driver has closed his
eyes for extended period
3.5.9 68 FACE LANDMARK DETECTOR
The pre-trained facial landmark detector inside the dlib library is used to estimate the location
of 68 (x, y)-coordinates that map to facial structures on the face.
The indexes of the 68 coordinates can be visualized on the image below:

FIG 3.5.9.1 : VISUALIZING THE 68 FACIAL LANDMARK COORDINATES FROM THE

IBUG 300-W DATASET (HIGHER RESOLUTION).
These annotations are part of the 68 point iBUG 300-W dataset which the dlib facial landmark
predictor was trained on.
It’s important to note that other flavors of facial landmark detectors exist, including the 194
point model that can be trained on the Helen dataset.
Regardless of which dataset is used, the same dlib framework can be leveraged to train a
shape predictor on the input training data — this is useful if you would like to train facial
landmark detectors or custom shape predictors of your own.
3.5.10 LIBRARIES IN PYTHON :

We use the following libraries in our project ,they are

 Numpy
 Scipy
 Imutils
3.5.10.1 NUMPY
NumPy is the fundamental package for scientific computing with Python. It contains among
other things:
 A powerful N-dimensional arrayobject
 Sophisticated (broadcasting)functions
 Tools for integrating C/C++ and Fortrancode
 Useful linear algebra, Fourier transform, and random number capabilities
NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary
data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a
wide variety of databases.
3.5.10.2 SCIPY
SciPy is an open-source Python library which is used to solve scientific and mathematical
problems. It is built on the NumPy extension and allows the user to manipulate and visualize
data with a wide range of high-level commands. As mentioned earlier, SciPy builds on
NumPy and therefore if you import SciPy, there is no need to import NumPy.
3.5.10.3 IMUTILS
A series of convenience functions to make basic image processing functions such as

translation, rotation, resizing, skeletonization, and displaying Matplotlib images easier with
OpenCV and both Python 2.7 and Python 3.

3.5.10.3.1 INSTALLATION
Provided you already have NumPy, SciPy, Matplotlib, and OpenCV already installed,
the imutils package is completely pip-installable:
$ pip install imutils
Finding function OpenCV functions by name
OpenCV can be a big, hard to navigate library, especially if you are just getting started
learning computer vision and image processing. The find_function method allows you to
quickly search function names across modules (and optionally sub-modules) to find the
function you are looking for.
3.5.10.3.2 TRANSLATION
Translation is the shifting of an image in either the x or y direction. To translate an image in

OpenCV you would need to supply the (x, y)-shift, denoted as (tx, ty) to construct the
translation matrix M:
And from there, you would need to apply the cv2.warpAffine function.
Instead of manually constructing the translation matrix M and calling cv2.warpAffine, you
can simply make a call to the translate function of imutils.
3.5.10.3.3 ROTATION

Rotating an image in OpenCV is accomplished by making a call

to cv2.getRotationMatrix2D and cv2.warpAffine. Further care has to be taken to supply
the (x, y)-coordinate of the point the image is to be rotated about. These calculation calls can
quickly add up and make your code bulky and less readable. The rotate function
in imutils helps resolve this problem.
3.5.10.3.4 RESIZING
Resizing an image in OpenCV is accomplished by calling the cv2.resize function. However,

special care needs to be taken to ensure that the aspect ratio is maintained.
This resize function of imutils maintains the aspect ratio and provides the keyword
arguments width and height so the image can be resized to the intended width/height while (1)
maintaining aspect ratio and (2) ensuring the dimensions of the image do not have to be
explicitly computed by the developer.
Another optional keyword argument, inter, can be used to specify interpolation method as
well.
3.5.11 DLIB
Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating
complex software in C++ to solve real world problems. It is used in both industry and
academia in a wide range of domains including robotics, embedded devices, mobile phones,
and large high performance computing environments.
Dlib's open source licensing allows you to use it in any application, free of charge. Open
Source Dib library is used by author for implementation of CNN(Neural Networks).
Highly optimized Pre-learned facial shape predictor and detectors functions are used by
author for detection of facial landmarks. Facial landmarks were further used for extracting eye
coordinates.

3.6 DEVELOPMENT ENVIRONMENT
3.6.1 PYTHON ENVIRONMENT SETUP
Python is available for use with Windows, Linux, Mac OS as well as certain other
platforms such as IBM AS/400, iOS, Solaris, etc.
To install Python on your local machine, get a copy of the standard distribution of Python
software from https://www.python.org/downloads based on your operating system, hardware
architecture and version of your local machine.
3.6.1.1 INSTALL PYTHON ON WINDOWS
To install Python on a Windows platform, you need to download the installer. A web-
based installer, executable installer and embeddable zip files are available to install Python on
Windows. Visit https://www.python.org/downloads/windows and download the installer
based on your local machine's hardware architecture.
The web-based installer needs an active internet connection. So, you can also download the
standalone executable installer. Visit https://www.python.org/ and click on the
Download Python 3.7.0 button as shown below. (3.7.0 is the latest version as of this writing.)

FIG 3.6.1.1.1 : PYTHON TOOL FOR DOWNLOAD
This will download python-3.7.0.exe for 32 bit. For the 64 bit installer, go
to https://www.python.org/downloads/windows and select the appropriate 64bit
installer, as shown below.
FIG 3.6.1.1.2: DIFFERENT VERSIONS DOWNLOAD PYTHON FOR WINDOWS 64 BIT
Download the Windows x86-64 executable installer and double click on it to start the python
installation wizard as shown below.

FIG 3.6.1.1.3 : PYTHON INSTALLATION WIZARD1
Installation is a simple wizard-based process. As you can see in the above figure, the default
installation folder will be path for Python 3.7.0 64 bit. Check the Add Python
3.7 to PATH checkbox, so that you can execute python scripts from any path. You may
choose the installation folder or feature by clicking on Customize installation. This will go to
the next step of optional features, as shown below.

Click Next to continue.

In Advanced Options, select the Install for all users option so that any user of your local
machine can execute Python scripts. Also, choose the installation folder to make a shorter
path for Python executable (something like C:\python37), keeping the rest of the choices to
default and finally click on the Install button.
After successful installation, you can start working with Python on your local machine.
3.6.2 SPYDER IDE INSTALLATION
Spyder is relatively easy to install on Windows, Linux and macOS. Just make sure to read and
follow these instructions with care.
3.6.2.1 USING ANACONDA
Spyder is included by default in the Anaconda Python distribution, which comes with
everything you need to get started in an all-in-one package, and is our recommended
installation method on Linux (and supported on the other platforms too).
3.6.2.2 RUNNING WITH ANACONDA
To run Spyder after installing it with Anaconda, the recommended method on Windows is to
launch it via the Start menu shortcut. On other platforms, open Anaconda Navigator, scroll to
Spyder under Home, and click Launch.
FIG 3.6.2.2.1 : ANACONDA NAVIGATOR SHOWING SPYDER

If Spyder does not launch via this method or you prefer to use the command line, open
Anaconda Prompt (Windows) or your terminal (other platforms), type conda activate base
then spyder.
3.6.2.3 NEW CONDA ENVIRONMENT

If you would like to have Spyder in a dedicated environment to update it separately from your
other packages and avoid any conflicts, you can. Just run the following command in your
Anaconda Prompt (Windows) or terminal (other platforms), for a minimal install of Spyder
into a new environment called spyder-env:
conda create -n spyder-env spyder=4
To install Spyder’s optional dependencies as well for full functionality, use the following
command instead:
conda create -n spyder-env spyder=4 numpy scipy pandas matplotlib sympy cython
4. SOFTWARE DESIGN
4.1 FACE RECOGNITION
A face Recognition system is a technology capable of identifying or verifying a person

from a digital image or a video frame from a video source. There are multiple methods in
which facial recognition systems work, but in general, they work by comparing selected facial
features from given image with faces within a database. It is also described as a Biometric
Artificial Intelligence based application that can uniquely identify a person by analyzing
patterns based on the person's facial textures and shape.

While initially a form of computer application, it has seen wider uses in recent times on
mobile platforms and in other forms of technology, such as robotics. It is typically used as
access control in security systems and can be compared to other biometrics such as
fingerprint or eye iris recognition systems. Although the accuracy of facial recognition
system as a biometric technology is lower than iris recognition and fingerprint
recognition, it is widely adopted due to its contactless and non-invasive process. Recently, it
has also become popular as a commercial identification and marketing tool. Other applications
include advanced human-computer interaction, video surveillance, automatic indexing of
images, and video database, among others.
During 1964 and 1965, Bledsoe, along with Helen Chan and Charles Bisson, worked on using
the computer to recognize human faces (Bledsoe 1966a, 1966b; Bledsoe and Chan 1965). He
was proud of this work, but because the funding was provided by an unnamed intelligence
agency that did not allow much publicity, little of the work was published. Based on the
available references, it was revealed that the Bledsoe's initial approach involved the manual
marking of various landmarks on the face such as the eye centers, mouth, etc., and these were
mathematically rotated by computer to compensate for pose variation. The distances between
landmarks were also automatically computed and compared between images to determine
identity.
Object Detection using Haar feature-based cascade classifiers is an effective object detection
method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection
using a Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach
where a cascade function is trained from a lot of positive and negative images. It is then used
to detect objects in other images. Here we will work with face detection. Initially, the
algorithm needs a lot of positive images (images of faces) and negative images (images
without faces) to train the classifier. Then we need to extract features from it. For this, Haar
features shown in the below image are used. They are just like our convolutional kernel. Each
feature is a single value obtained by subtracting sum of pixels under the white rectangle from
sum of pixels under the black rectangle.

4.1.1 HOW FACE WILL BE RECOGNIZED
Some face recognition algorithms identify facial features by extracting landmarks, or features,
from an image of the subject's face. For example, an algorithm may analyze the relative
position, size, and/or shape of the eyes, nose, cheekbones, and jaw. These features are then
used to search for other images with matching features.
Other algorithms normalize a gallery of face images and then compress the face data, only
saving the data in the image that is useful for face recognition. A probe image is then
compared with the face data. One of the earliest successful systems is based on template
matching techniques applied to a set of salient facial features, providing a sort of compressed
face representation.
Recognition algorithms can be divided into two main approaches: geometric, which looks at
distinguishing features, or photometric, which is a statistical approach that distills an image
into values and compares the values with templates to eliminate variances. Some classify
these algorithms into two broad categories: holistic and feature- based models. The former
attempts to recognize the face in its entirety while the feature- based subdivide into
components such as according to features and analyze each as well as its spatial location with
respect to other features.

FIG 4.1.1.1 : FACE RECOGNITION
4.1.2 NEED OF FACE RECOGNITION
While face recognition has been around in one form or another since the 1960s, recent
technological developments have led to a wide proliferation of this technology. This
technology is no longer seen as something out of science fiction movies like Minority Report.
With the release of the iPhone X, millions of people now literally have face recognition
technology in the palms of their hands, protecting their data and personal information. While
mobile phone access control might be the most recognizable way face recognition is being
used, it is being employed for a wide range of use cases including preventing crime,
protecting events and making air travel more convenient.
We’ve compiled a list of 21 ways that face recognition is currently being used to make
the world safer, smarter and more convenient.
Some of the technologies are:
1. Unlock phone
2. Find missing persons
3. Help the blind
4. Identify people in social media
5.Recognize Drivers
4.2 CASCADE CLASSIFIER

Cascading classifiers are trained with several hundred "positive" sample views of a particular
object and arbitrary "negative" images of the same size. After the classifier is trained it can be
applied to a region of an image and detect the object in question. To search for the object in
the entire frame, the search window can be moved across the image and check every location
for the classifier. This process is most commonly used in image processing for object
detection and tracking, primarily facial detection and recognition.
4.2.1 HOW IT WORKS
You might be good at recognizing faces. You probably find it a cinch to identify the
face of a family member, friend, or acquaintance. You’re familiar with their facial features —
their eyes, nose, mouth — and how they come together.
That’s how a facial recognition system works, but on a grand, algorithmic scale. Where
you see a face, recognition technology sees data. That data can be stored and accessed. For
instance, half of all American adults have their images stored in one or more facial-
recognition databases that law enforcement agencies can search, according to a Georgetown
University study.
So how does facial recognition work? Technologies vary, but here are the basic steps:
Step1. A picture of your face is captured from a photo or video. Your face might appear alone
or in a crowd. Your image may show you looking straight ahead or nearly in profile.
Step2. Facial recognition software reads the geometry of your face. Key factors include the
distance between your eyes and the distance from forehead to chin. The software identifies
facial landmarks — one system identifies 68 of them — that are key to distinguishing your
face. The result: your facial signature.
Step3. Your facial signature — a mathematical formula — is compared to a database of

known faces. And consider this: at least 117 million Americans have images of their faces in
one or more police databases. According to a May 2018 report, the FBI has had access to 412
million facial images for searches.
Step4. A determination is made. Your face print may match that of an image in a facial
recognition system database
4.2.2 HOW IT CLASSIFY LANDMARKS
As computer vision engineers and researchers we have been trying to understand the
human face since the very early days. The most obvious application of facial analysis is Face
Recognition. But to be able to identify a person in an image we first need to find where in the
image a face is located. Therefore, face detection — locating a face in an image and returning
a bounding rectangle / square that contains the face — was a hot research area. In 2001, Paul
Viola and Michael Jones pretty much nailed the problem with their seminal paper titled
“Rapid Object Detection using a Boosted Cascade of Simple Features.” In the early days of
OpenCV and to some extent even now, the killer application of OpenCV was a good
implementation of the Viola and Jones face detector.
Once you have a bounding box around the face, the obvious research problem is to see if you
can find the location of different facial features ( e.g. corners of the eyes, eyebrows, and the
mouth, the tip of the nose etc ) accurately. Facial feature detection is also referred to as “facial
landmark detection”, “facial keypoint detection” and “face alignment” in the literature, and
you can use those keywords in Google for finding additional material on the topic.
4.3 DIAGRAMS
A UML diagram is a diagram based on the UML (Unified Modeling Language) with the
purpose of visually representing a system along with its main actors, roles, actions, artifacts or
classes, in order to better understand, alter, maintain, or document information about the
system.
UML is an acronym that stands for Unified Modeling Language. Simply put, UML is a
modern approach to modeling and documenting software. In fact, it’s one of the most popular
business process modeling techniques.

It is based on diagrammatic representations of software components .As the old proverb

says: “a picture is worth a thousand words”. By using visual representations, we are able to
better understand possible flaws or errors in software or business processes.
4.3.1 USECASE DIAGRAM
A use case diagram at its simplest is a representation of a user's interaction with the
system that shows the relationship between the user and the different use cases in which the
user is involved. A use case diagram can identify the different types of users of a system and
the different use cases and will often be accompanied by other types of diagrams as well. The
use cases are represented by either circles or ellipses.

FIG 4.3.1.1 USECASE DIAGRAM DROWSINESS DETECTION USING OPENCV

4.3.2 CLASS DIAGRAM
In software engineering, a class diagram in the Unified Modeling Language

(UML) is a type of static structure diagram that describes the structure of a system by showing
the system's classes, their attributes, operations (or methods), and the relationships among
objects.
The class diagram is the main building block of object-oriented modeling. It is used for
general conceptual modeling of the structure of the application, and for detailed modeling
translating the models into programming code. Class diagrams can also be used for data
modelling.
The classes in a class diagram represent both the main elements, interactions in the
application, and the classes to be programmed. In the diagram, classes are represented with
boxes that contain three compartments:
 The top compartment contains the name of the class. It is printed in bold and centered, and the
first letter is capitalized.
 The middle compartment contains the attributes of the class. They are left-aligned and the first
letter is lowercase.
 The bottom compartment contains the operations the class can execute. They are also left-
aligned and the first letter is lowercase.
 In the design of a system, a number of classes are identified and grouped together in a class
diagram that helps to determine the static relations between them. With detailed modeling, the
classes of the conceptual design are often split into a number of subclasses.
 In order to further describe the behavior of systems, these class diagrams can be
complemented by a state diagram or UML state machine.

FIG 4.3.2.1 : CLASS DIAGRAM FOR DROWSINESS DETECTION USING OPENCV

4.3.3 SEQUENCE DIAGRAM
A sequence diagram simply depicts interaction between objects in a sequential order i.e.
the order in which these interactions take place. We can also use the terms event diagrams or
event scenarios to refer to a sequence diagram. Sequence diagrams describe how and in what
order the objects in a system function. These diagrams are widely used by businessmen and
software developers to document and understand requirements for new and existing systems.
FIG 4.3.3.1 : SEQUENCE DIAGRAM FOR DROWSINESS DETECTION USING

OPENCV

4.3.4 ACTIVITY DIAGRAM
We use Activity Diagram to illustrate the flow of control in a system and refer to the
steps involved in the execution of a use case. We model sequential and concurrent activities
using activity diagrams. So, we basically depict workflows visually using an activity diagram.
An activity diagram focuses on condition of flow and the sequence in which it happens. We
describe or depict what causes a particular event using an activity diagram.
UML models basically three types of diagrams, namely, structure diagrams, interaction
diagrams, and behavior diagrams. An activity diagram is a behavioral diagram i.e. it depicts
the behavior of a system.
An activity diagram portrays the control flow from a start point to a finish point
showing the various decision paths that exist while the activity is being executed. We can
depict both sequential processing and concurrent processing of activities using activity
diagram. They are used in business and process modelling where their primary use is to depict
the dynamic aspects of a system.

FIG 4.3.4.1 : ACTIVITY DIAGRAM FOR DROWSINESS DETECTION USING OPENCV

4.4 CONTROL FLOW DIAGRAMS
A control flow diagram helps us understand the detail of a process. It shows us where
control starts and ends and where it may branch off in another direction, given certain
situations. Let's say you are working on software to start a machine. What happens if the
engine is flooded, or a spark plug is broken? Control then changes the flow to other parts of
the software.
We can represent these branches with a diagram. The flow diagram is helpful because it
can be understood by both stakeholders and systems professionals. Although some of the
symbols might not be fully understood by the layperson, they can still grasp the general
concept.
FIG 4.4.1 : CONTROL FLOW DIAGRAM

5. PROPOSED SYSTEM
To deal with the drowsiness problem and to provide an effective system a drowsiness
detection system can be developed which can be placed inside any vehicle which will take
live video of the driver as input and if the driver is showing any symptoms of drowsiness
system will automatically detect and raise an alarm which will alert the driver and other
passengers.
After continuously alerting the driver ,if the driver is not recovered from drowsiness then the
alert message will be sent to the owner of the vehicle .so then owner will help to alert the
driver.
5.1 PROPOSED METHODOLOGY
A different type of methodologies has been developed to find out drowsiness.
5.1.1 PHYSIOLOGICAL LEVEL APPROACH
This technique is an intrusive method wherein electrodes are used to obtain pulse rate, heart
rate and brain activity information. ECG is used to calculate the variations in heart rate and
also detect different conditions for drowsiness. The correlation between different signals such
as ecg (electrocardiogram), EEG (electroencephalogram), and EMG (electromyogram) are
made and then the output is generated whether the person is drowsy or not.
5.1.2 BEHAVIORAL BASED APPROACH
In this technique eye blinking frequency, head pose, etc. of a person is monitored through a
camera and the person is alerted if any of these drowsiness symptoms are detected.

5.2 SYSTEM MODULES
5.2.1 FACE DETECTION
Object Detection using Haar feature-based cascade classifiers is an effective object detection
method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection
using a Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach
where a cascade function is trained from a lot of positive and negative images.
It is then used to detect objects in other images. Here we will work with face detection.
Initially, the algorithm needs a lot of positive images (images of faces) and negative images
(images without faces) to train the classifier. Then we need to extract features from it. For
this, Haar features shown in the below image are used. They are just like our convolutional
kernel. Each feature is a single value obtained by subtracting sum of pixels under the white
rectangle from sum of pixels under the black rectangle.
FIG 5.2.1.1 : HAAR CASCADE FEATURES

A cascaded Adaboost classifier with the Haar-like features is exploited to find out the face
region. First, the compensated image is segmented into numbers of rectangle areas, at any
position and scale within the original image. Due to the difference of facial feature, Haar-like
feature is efficient for real-time face detection. These can be calculated according to the
difference of sum of pixel values within rectangle areas. The features can be represented by
the different composition of the black region and white region.
A cascaded Adaboost classifier is 7 | P a g e a strong classifier which is a combination of

several weak classifiers. Each weak classifier is trained by Adaboost algorithm. If a candidate
sample passes through the cascaded Adaboost classifier, the face region can be found. Almost
all of face samples can pass through and nonface samples can be rejected.
5.2.2 EYE DETECTION
Object detection using Haar feature-based cascade classifiers is more than a decade and a half
old. OpenCV framework provides a pre-built Haar and LBP based cascade classifiers for face
and eye detection which are of reasonably good quality. pre-built Haar/LBP cascades have a
high false positive rate which might make them unsuitable for many use-cases. Fortunately, it
is possible to train an eye detector with very high accuracy and low false positive rates for
many cases with OpenCV.
5.2.2.1 RECOGNITION OF EYE'S STATE
The characteristic feature of the eye is extracted to recognize the eye's state. In general, the
state of left eye is equal to right one at the same time. Therefore, we only consider one eye’s
state in one frame. This consideration is also useful to the reduction of computational
complexity. In this step, two schemes are adopted: (1) binary pattern and (2) the Canny’s edge
detection
The eye image is converted to binary pattern based on the threshold value T.
𝑇 = {∑ Xi 𝑛 𝑖=1 }/𝑛

Here n is the number of pixels in the eye region and xi is the pixel value of the position i in
the region. There are n pixels in the eye region. If the pixel value of P is greater than the
threshold T, P will be set as white, 1. Otherwise P will be set as 0.
When the conversion of eye image is completed, the height of the eyelids is utilized to
determine the eye's state.
Then, the magnitude and orientation of the edge are calculated by the differential filter. The
final edge image is obtained by edge information of multiple scale σ. Finally, the numbers of
edge points are summed for recognizing the eye's state.
5.2.3 EYE BLINK DETECTION
5.2.3.1 GRAYSCALE CONVERSION:
The coloured eye image is first converted to grayscale. Gray scale conversion algorithms use
to convert the coloured image to grayscale.
FIG 5.2.3.1.1 : CONVERSION OF COLOORED IMAGE TO GRAYSCALE
5.2.3.2 MIDPOINT CALCULATION
A midpoint is defined as the middle or Centre point of a line segment. Once all the required
points have been found, the next step would be to find midpoint between the two upper corner
points.

Let (X1, Y1) be the coordinates of upper left corner and (X2,Y2) be the coordinates of the
upper right corner. A line segment is drawn between these two points. The midpoint of this
line segment can be calculated using the following formula.
5.2.3.3 DISTANCE CALCULATION
Distance is a mathematical description of how far objects are from each other. As next step,
we find distance of the midpoint from the point at lower eyelid. In analytic geometry, distance
between two or more points is calculated by using the distance formula given by the
Pythagorean Theorem.
5.2.3.4 EYE STATE DETERMINATION
Finally, the decision for the eye state is made on the basis of distance ’d’ calculated in the
previous step. If the distance is zero or is close to zero, the eye state is classified as “closed”
otherwise the eye state is identified as “open”.
5.2.4 DROWSINESS DETECTION:
The last step of the algorithm is to determine the person’s condition on the basis of a pre-set
condition for drowsiness. The average blink duration of a person is 100-400 milliseconds.
This is 0.1-0.4 of a second. Hence if a person is drowsy his eye closure must be beyond this
interval. We set a time frame of 5 seconds. If the eyes remain closed for five or more seconds,
drowsiness is detected and alert pop regarding this is triggered
5.2.4 SENDING ALERT MESSAGE
 Alert message will be send when the driver is detected with the drowsiness,,so we use Twilio
application programming interface.
 Twilio Messaging is an API to send and receive sms, mms messages globally. It uses

intelligent sending features to ensure messages reliably reach end users wherever they are.
 The Twilio Python Helper Library makes it easy to interact with the Twilio API from
our Python application.
 Applications are simple to build and scalable.
5.2.5 PLAYING ALERT SOUND
 Alert sound will be played when drowsiness is detected ,it uses playgame.mixer library in
python
 In order to play music/audio files in pygame , pygame.mixer is used (pygame module for
loading and playing sounds). This module contains classes for loading Sound objects and
controlling playbacks.
5.3 ADVANTAGES
 This method will detect a drowsiness of driver before any problem accord and inform the driver
and other passengers by raising an alarm.
 These detection methods are comfortable while driving.

6. CODING
6.1 DIRECTORIES
FIG-6.1.1 PROJECT DIRECTORIES

6.2 SOURCE CODE
6.2.1 DROWSINESS_DETECT.PY
#Import necessary libraries
from scipy.spatial import distance

from imutils import face_utils
import numpy as np
import pygame #For playing sound
import time
import dlib
import cv2
from twilio.rest import Client
#Initialize Pygame and load music
pygame.mixer.init()
pygame.mixer.music.load(r'C:/Users/RELIANCE/Downloads/drowsiness
(1)/drowsiness/audio/alert.wav')
#Minimum threshold of eye aspect ratio below which alarm is triggerd
EYE_ASPECT_RATIO_THRESHOLD = 0.3
#Minimum consecutive frames for which eye ratio is below threshold for alarm to be
triggered

EYE_ASPECT_RATIO_CONSEC_FRAMES = 50
#COunts no. of consecutuve frames below threshold value
COUNTER = 0
#Load face cascade which will be used to draw a rectangle around detected faces.
face_cascade = cv2.CascadeClassifier(r"C:/Users/RELIANCE/Downloads/drowsiness
(1)/drowsiness/haarcascades/haarcascade_frontalface_default.xml")
#This function calculates and return eye aspect ratio
def eye_aspect_ratio(eye):
A = distance.euclidean(eye[1], eye[5])
B = distance.euclidean(eye[2], eye[4])
C = distance.euclidean(eye[0], eye[3])
ear = (A+B) / (2*C)
return ear
#Load face detector and predictor, uses dlib shape predictor file
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(r'C:/Users/RELIANCE/Downloads/drowsiness
(1)/drowsiness/haarcascades/shape_predictor_68_face_landmarks.dat')
#Extract indexes of facial landmarks for the left and right eye
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS['left_eye']

(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS['right_eye']
#Start webcam video capture
video_capture = cv2.VideoCapture(0)
#Give some time for camera to initialize(not required)

time.sleep(2)
while(True):
#Read each frame and flip it, and convert to grayscale
ret, frame = video_capture.read()

frame = cv2.flip(frame,1)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = detector(gray, 0)
#Detect faces through haarcascade_frontalface_default.xml

face_rectangle = face_cascade.detectMultiScale(gray, 1.3, 5)
#Draw rectangle around each face detected
for (x,y,w,h) in face_rectangle:

cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
#Detect facial points
for face in faces:

shape = predictor(gray, face)
shape = face_utils.shape_to_np(shape)

#Get array of coordinates of leftEye and rightEye

leftEye = shape[lStart:lEnd]
rightEye = shape[rStart:rEnd]
#Calculate aspect ratio of both eyes
leftEyeAspectRatio = eye_aspect_ratio(leftEye)
rightEyeAspectRatio = eye_aspect_ratio(rightEye)
eyeAspectRatio = (leftEyeAspectRatio + rightEyeAspectRatio) / 2
#Use hull to remove convex contour discrepencies and draw eye shape around eyes
leftEyeHull = cv2.convexHull(leftEye)
rightEyeHull = cv2.convexHull(rightEye)
cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1
#Detect if eye aspect ratio is less than threshold
if(eyeAspectRatio < EYE_ASPECT_RATIO_THRESHOLD):

COUNTER += 1
#If no. of frames is greater than threshold frames,
if COUNTER >= EYE_ASPECT_RATIO_CONSEC_FRAMES):

pygame.mixer.music.play(-1)
cv2.putText(frame, "You are Drowsy", (150,200),
cv2.FONT_HERSHEY_SIMPLEX, 1.5,0,0,255), 2)

# the following line needs your Twilio Account SID and Auth Token
client = Client("AC758b3560659d53c0824b81eddbc4a210",
"4e933c9d2c328d53bcc44f0a2337fd0c")
# change the "from_" number to your Twilio number and the "to" number
# to the phone number you signed up for Twilio with, or upgrade your
# account to send SMS to any phone number
client.messages.create(to="+918187808068",
from_="+16822771585",
body="ALERT the Driver is Sleepy !!!!")
else:
pygame.mixer.music.stop()
COUNTER = 0
#Show video feed
cv2.imshow('Video', frame)
if(cv2.waitKey(1) & 0xFF == ord('q')):
break
#Finally when video capture is over, release the video capture and destroyAllWindows
video_capture.release()
cv2.destroyAllWindows(

6.3 FACE_AND_EYE_DETECTOR_WEBCAM_VIDEO.PY
#Import necessary libraries
import cv2 as cv
import numpy as np
#Load face cascade and hair cascade from haarcascades folder
face_cascade = cv.CascadeClassifier("haarcascades/haarcascade_frontalface_default.xml")
eye_cascade = cv.CascadeClassifier("haarcascades/haarcascade_eye.xml")
#Capture video from webcam
video_capture = cv.VideoCapture(0)
#Read all frames from webcam
while True:
ret, frame = video_capture.read()
frame = cv.flip(frame,1) #Flip so that video feed is not flipped, and appears mirror like.
gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:

cv.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = frame[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(roi_gray)
for (ex,ey,ew,eh) in eyes:

cv.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
cv.imshow('Video', frame)
if(cv.waitKey(1) & 0xFF == ord('q')):

break
#Finally when video capture is over, release the video capture and destroyAllWindows

video_capture.release()
cv.destroyAllWindows()
7. TESTING AND VALIDATIONS
7.1 INTRODUCTION
The purpose of testing is to discover error. Testing is the Process of trying to discover
every conceivable fault or weakness in a product. It provides a way to check the functionality
of components, sub assembles and/ or a finished product. It is the process of exercising
software with the intent of ensuring that the software system meets its requirements and user
expectations and does not fail in an unacceptable manner. There are various types of test.
Each test type addresses a specific testing requirement.
7.2 TESTING AND VALIDATION
7.2.1 UNIT TESTING
Unit testing involves the design of test cases that validate that the internal program logic
is functioning properly, and that program inputs produce valid outputs. All decision branches
and internal code flow should be validates. It is the testing of individual software units of the
application. It is done after the completion of an individual unit before integration. This is a
structural testing, that relies on knowledge of its construction and is invasive. Unit tests
perform basic test at component level and test a specific business process, application, and/or
system configuration. Unit tests ensure that each unique path of a business process performs
accurately to the documented specifications and contain clearly defined inputs and expected
results.
7.2.2 INTEGRATION TESTING
Integration tests are designed to test integrated software components to determine if

they actually run as one program. Testing is event driven and is more concerned with the
basic outcome of screens or fields. Integration tests demonstrate that although the components
were individually satisfaction, as shown by successful unit testing, combination of
components is correct and consistent. Integration testing is specifically aimed at exposing the
problems that arise from the combination of components.
7.2.3 FUNCTIONAL TESTING
Functional tests provide systematic demonstrations that functions tested are available as
specified by the business and technical requirements, system documentation, and user
manuals.
Functional testing is centered on the following items:
Valid Input : identified classes of valid input must be accepted.

Invalid Input : identified classes of invalid input must be rejected.
Functions : identified functions must be exercised.
Output : identified classes of application outputs must be exercised.
Systems/Procedures : interfacing systems or procedures must be invoked.
Organization and preparation of functional tests is focused on requirements, key functions, or

special test cases. In addition, systematic coverage pertaining to identify Business process
flows; data fields, predefined processes, and successive processes must be considered for
testing. Before functional testing is complete, additional tests are identified and the effective
value of current tests is determined.
7.2.4 SYSTEM TESTING
System testing ensures that the entire integrated software system meets requirements.
It tests a configuration to ensure known and predictable results. An example of system testing
is the configuration oriented system integration test. System testing is based on process
descriptions and flows, emphasizing pre-driven process links and integration points.
7.2.5 WHITE BOX TESTING
White Box Testing is a testing in which in which the software tester has knowledge of
the inner workings, structure and language of the software, or at least its purpose. It is
purpose. It is used to test areas that cannot be reached from a black box level.
7.2.6 BLACK BOX TESTING
Black Box Testing is testing the software without any knowledge of the inner workings,
structure or language of the module being tested. Black box tests, as most other kinds of tests,
must be written from a definitive source document, such as specification or requirements
document, such as specification or requirements document. It is a testing in which the
software under test is treated, as a black box .you cannot “see” into it. The test provides inputs
and responds to outputs without considering how the software works.
7.2.7 ACCEPTANCE TESTING
User Acceptance Testing is a critical phase of any project and requires significant
participation by the end user. It also ensures that the system meets the functional
requirements.

7.3 TEST RESULTS
Test Test Condition System Expected

Id Behaviour Result Result
1 Straight Face, Non Non Non
Good Light, drowsy drowsy drowsy
Eyes open
2 Tilted Face, Drowsy Drowsy text Drowsy
Good light, alert , text alert,
Eyes closed Alert mesaage , Alert
Alert sound mesaage,
Alert
sound
3 Straight Face, Drowsy Drowsy text Drowsy

Bad Light, alert , text alert ,
Eyes Closed Alert mesaage,, Alert
Alert sound mesaage ,
Alert
sound
Good Light, alert , text alert ,
Eyes Closed Alert mesaage , Alert
Alert sound mesaage
,Alert
sound
Good Light, alert , text alert,

Eyes Closed, Alert mesaage, Alert

With Glasses Alert sound mesaage,
Alert
sound
Bad light , alert , text alert ,
Eyes Closed, Alert mesaage , Alert
With Glasses Alert sound mesaage ,
Alert
sound
7 Straight Face, Non Non Non
Good Light, drowsy drowsy drowsy
Eyes open,
With Glasses
7.3.1 TEST CASES

8. OUTPUT SCREENSHOTS
8.1 Videofeed
FIG 8.1.1: VIDEO FEED AND DRAWING CIRCLE AROUND EYES
8.2 Videodrowsy text
FIG 8.1.2 DROWSY SCREEN ALERT

8.3 MESSAGE ALERT
FIG 8.1.3 ALERT MESSAGE FROM PHONE

8 CONCLUSION AND FURTHER WORK
8.1 CONCLUSION
A novel approach to the face detection problem is given, based on analogic

cellular neural network algorithms. It describes a completely new method for
the finding and helping the normalization of faces, which is a critical step of
this complex task but hardly ever discussed in the literature. Though still in an
experimental stage, we found that the suggested CNN algorithm finds and
normalizes human faces effectively while its time requirement is a fraction of
the previously used methods.
The algorithm starts with the detection of heads on color pictures using
deviations in color and structure of the human face and that of the background.
Reference points are needed for normalization of the face. The eyes were
found the most reliable for this purpose. By normalizing the distance and
position of the reference points, all faces could be transformed into the same
size and position.
Another CNN algorithm finds the eyes on any grayscale image by searching
some characteristic features of the eyes and eye sockets. The investigated
features are useful for other recognition and detection problems too, not only
in face detection. The parts of the algorithm can be used in various tasks as a
subroutine. Tests made on standard database show that the algorithm works
very fast and it is reliable.
The time requirement of the algorithm is about 3 ms which is acceptable for
real-time applications (e.g. security identification, image query, etc.).
Thus we would have successfully designed and developed partial
implementation of the Driver Drowsiness Detector using Python and OpenCv
along with the a cam to detect the face.

8.2 FUTURE WORK
The main goal of this project is to detect the states of the drivers’ eyes to
determine they are in the state of drowsy or not. The system must meet certain
requirement which is detecting drowsiness and alert the driver as well
accurately.
Improvement on the algorithms to detect eyes and mouth need to be done for
future implementation. Luminance changes have to be encounter to ensure the
detection of the gradient of eyes is sufficient to improve the detection results.
The quality of the video or images used in detecting drowsiness affects the
result of the detection. Therefore, a good quality and high frame rate of images
(number of pixel) is one of the factors to get better detection. Better techniques
can be used to compare which are more reliable in detecting drowsiness.
Thus, by making this project successful, the numbers of road accident can be
reduce when this project is implemented in the vehicle to detect the drowsiness
of the driver

10. REFERENCES
10.1 BOOKS
[1.] L.O. Chua and L. Yang, “Cellular neural networks: Theory,”IEEE Trans. on
Circuits and Systems, Vol. 35, pp. 1257–1290,1988.
[2.] L.O. Chua and T. Roska, “The CNN paradigm,” IEEE Trans. onCircuits and
Systems I, Vol. 40, pp. 147–156, 1993.
[3.] F. Werblin, T. Roska and L.O. Chua, “The analogic cellular neural network
as a bionic eye,” Journal of Circuit Theory andApplications, Vol. 23, pp.
[4.]T. Roska, “CNN chip set architectures and the visual mouse,”IEEE Int.
Workshop CNNA ’96, Sevilla, pp. 487–492,1996.
[5.] L.O. Chua and T. Roska, “Cellular neural networks: Foundations and
primer,”lecture notes for curse EE129 at U.C.Berkeley, 1997.
[6.] T. Roska and L.O. Chua, “The CNN universal machine: An

analogic array computer,” IEEE Trans. on Circuits and Systems
II, Vol. 40, pp. 163–173, 1993.
[7.] S.Y. Kung, “Neural networks for intelligent multimedia processing,” Proc.
IEEE, Vol. 86, No. 6, pp. 1244–1272,1998.
[8.] S.-H. Lin, S.Y. Kung, and L.-J. Lin, “Face recognition/detection by
probabilistic decision-based neural network,” IEEETrans. on Neural Networks,
Vol. 8, No. 1, pp. 114–132,1997.

10.2 PAGES
[1.] Alert System for Driver Drowsiness using Real Time detection by Dr. Sudhakar
Tripathi Head of Dapartement of Information Technology Rajkiya Engineering College
Ambedkarnagar,U.P., India
[2.] Drowsiness Detection System Using OpenCV by Aayushi Ahlawat, Simran Kaur,
Sejal Rana ,Department of Computer Science and Engineering, Maharaja Surajmal
Institute of Technology, GGSIPU, New Delhi, Indi
[3.] Face and Eye Detection by CNN Algorithms by DAVID BALYA AND TAMAS ROSKA ´
Analogical and Neural Computing Laboratory, Computer and Automation Research Institute
[4.] Driver Drowsiness Detection with Face Recognition System by Rushikesh Rajendra
Kamble , Department of Computer Engineering in Degree, Sinhgad Institute of
Technology and Science, Pune, Maharashtra, India
10.3 WEBSITES
[1.]https://www.codeproject.com/Articles/26897/TrackEye-Real-Time- Tracking-Of-
Human-EyesUsing-a
[2] https://realpython.com/face-recognition-with-python/
[3.]https://www.pyimagesearch.com/2017/04/24/eye-blink-detection-opencv-python-
dlib/
[4.] https://www.codeproject.com/Articles/26897/TrackEye-Real-Time- Tracking-Of-

Human-EyesUsing-a
[5.] https://github.com/tahaemara/sleep-detection/blob/master/README.md

[6.] https://ieeexplore.ieee.org/document/7396336?reload=true
[7.] https://github.com/mans-men/eye-blink-detection-demo
[8.] https://docs.opencv.org/3.4/d7/d8b/tutorial_py_face_detection.html
[9] https://www.learnopencv.com/training-better-haar-lbp-cascade-eye- detector-

opencv/
[10] https://blog.goodaudience.com/real-time-face-and-eyes-detection-with-opencv-
54d9ccfee6a8
[11] https://www.seeingmachines.com/technology/
[12] https://eyegaze.com/tag/lc-technologies

Drowsiness Detection Using Opencv Final

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

Drowsiness Detection Using Opencv Final

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Drowsiness Detection Using Opencv Final

Uploaded by

Copyright:

Available Formats

DROWSINESS DETECTION USING OPENCV

COMPUTER SCIENCE & ENGINEERING

P.DURGA PRASADU 176R1A0525

K.SIVA GANGA KRISHNA 176R1A0517

Under the esteemed guidance of

Mr. N. Rakesh Kumar , M.Tech.,

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

GIET COLLEGE OF ENGINEERING

Internal Guide Head of the Department

P.DURGA PRASADU 176R1A0525

K.SIVA GANGA KRISHNA 176R1A0517

We hereby declare that the project entitled “DROWSINESS DETECTION USING

P.DURGA PRASADU 176R1A0525

K.SIVA GANGA KRISHNA 176R1A0517

S.NO CONTENTS PAGE NO

1.2 PROBLEM STATEMENT 3

1.3 PROJECT SCOPE 4

1.4 PROJECT OBJECTIVES 4

2.2 EXISTING SYSTEM 7

2.3 FEASIBILITY STUDY 7

2.3.1 ECONOMIC FEASIBILITY 8

2.3.2 TECHNICAL FEASIBILITY 8

2.3.3 SOCIAL FEASIBILITY 8

3. SOFTWARE REQUIREMENT & ANALYSIS 9

3.2 NON-FUNCTIONAL REQUIREMENTS 9

3.4 SOFTWARE REQUIREMENTS 10

3.5 TECHNOLOGIES USED 11

3.5.3 CONVOLUTIONAL NEURAL NETWORKS(CNN) 16

3.5.4 SPYDER IDE 17

3.5.7 HAAR CASCADE CLASSIFIER IN OPENCV 20

3.5.8 EYE ASPECT RATIO 21

3.5.9 68 FACE LANDMARK DETECTOR 23

3.5.10 LIBRARIES IN PYTHON 24

3.6 DEVELOPMENT ENVIRONMENT 27

3.6.1 PYTHON ENVIRONMENT SETUP 27

3.6.2 SPYDER IDE INSTALLATION 31

4.2 CASCADE CLASSIFIER 36

4.3 UML DIAGRAMS 38

4.3.1 USE CASE DIAGRAM 38

4.3.2 CLASS DIAGRAM 40

4.3.3 SEQUENCE DIAGRAM 42

4.3.4 ACTIVITY DIAGRAM 43

4.4 CONTROL FLOW DIAGRAM 45

5.2 SYSTEM MODULES 47

6.2 SOURCE CODE 53

7. TESTING AND VALIDATIONS 59

7.2.1 UNIT TESTING 59

7.2.2 INTEGRATION TESTING 59

7.2.3 FUNCTIONAL TESTING 60

7.2.4 SYSTEM TESTING 60

7.2.5 WHITE BOX TESTING 61

7.2.6 BLACK BOX TESTING 61

7.2.7 ACCEPTANCE TESTING 61

7.3 TEST RESULTS 62

9. CONCLUSION AND FUTURE WORK 66

S.NO FIG NO FIG NAME PAGE NO

1. 1.1.1 THE STATISTIC OF ROAD ACCIDENTS OF 1

4. 3.5.2.1 OPENCV LIBRARY 12