SUMMER TRAINING REPORT

ON

“EMOSIC(EMOTION BASED MUSIC

PLAYER)”

Submitted to
RAJASTHAN TECHNICAL UNIVERSITY

In Partial Fulfilment of the Requirement for the Award of

BACHELOR’S DEGREE IN
INFORMATION TECHNOLOGY AND ENGINEERING

BY

Sneha Mathur 17ESKIT078

UNDER THE GUIDANCE OF

Ms. Nidhi Shrivastava

DEPARTMENT OF INFORMATION TECHNOLOG AND ENGINEERING

SWAMI KESHVANAND INSTITUTE OF TECHNOLOGY ,
JAIPUR

2019-2020
Swami Keshvanand Institute of Technology, Jaipur
Department of Information Technology

CERTIFICATE
Department of Information Technology, SKIT College, Jaipur 3
 Acknowledgement

It is my pleasure to be indebted to various people, who directly or indirectly con-

tributed in the development of this work and who influenced my thinking, behavior,
and acts during the course of study.

I express my sincere gratitude to Mr. Anil Choudhry, HOD, IT for providing

me an opportunity to undergo summer training at Adhoc Networks. I am thankful
to Mr Ashoutosh Singh, CTO Adhoc Networks , for his support, cooperation, and
motivation provided to me during the training for constant inspiration, presence and
blessings. I also extend my sincere appreciation to Mrs. Nidhi Shrivastav who
provided her valuable suggestions and precious time in accomplishing my report.

At last I must express my sincere heartfelt gratitude to all the staff members of
Information Technology Department who helped me directly or indirectly during
this course of work.

Sneha Mathur
17ESKIT078
Contents

1 Introduction 2
1.1 OVERVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 MOTIVATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 OBJECTIVES OF TRAINING . . . . . . . . . . . . . . . . . . . . 2

2 Introduction to Project / Modules 4

2.1 OVERVIEW OF PROJECT . . . . . . . . . . . . . . . . . . . . . . 4
2.2 DESIGN OF PROJECT . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 IMPLEMENTATION OF PROJECT . . . . . . . . . . . . . . . . . 6
2.4 SNAPSHOTS OF PROJECT . . . . . . . . . . . . . . . . . . . . . 8

3 Description of Modules 11
3.1 FISHER FACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 OPEN CV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 HAAR CASCADE . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4 NUMPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.5 Webbrowser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 Results 16
4.1 RESULTS ACHIEVED . . . . . . . . . . . . . . . . . . . . . . . . 16
4.2 SCREENSHOTS OF RESULT ACHIEVED . . . . . . . . . . . . . 16

5 Conclusion 19
5.1 TAKEAWAYS OF TRAINING . . . . . . . . . . . . . . . . . . . . 19
5.2 FUTURE SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . 19
List of Figures

2.1 E-R Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2 Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4 Dataset Creation Prog. 1 . . . . . . . . . . . . . . . . . . . . . . 8
2.5 Dataset Creation Prog. 2 . . . . . . . . . . . . . . . . . . . . . . 8
2.6 XML File Creation Prog . . . . . . . . . . . . . . . . . . . . . . 9
2.7 Emosic Prog. 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.8 Emosic Prog. 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.1 Project Snap 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Project Snap 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Project Snap 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 Project Snap 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Chapter 1

Introduction

1.1 OVERVIEW

Project EMOSIC (an emotion based music player) is a novel approach that helps
the user to automatically play songs based on the emotions of the user. It recognizes
the facial emotions of the user and plays the songs according to their emotion. The
emotions are recognized using a machine learning method concept. The human face
is an important organ of an individual‘s body and it especially plays an important
role in extraction of an individual‘s behavior and emotional state. The webcam
captures the image of the user. It then extract the facial features of the user from the
captured image. Facial expression categorized into 4: happy, sad, angry and neutral.
According to the emotion, the music will be played from the created playlist.

1.2 MOTIVATION

Learning machine learning was purely from an interest in Artificial intelligence

for me. For some reason, just hearing the word robots or anything about AI just
fascinated me but it was just a moment of joy at that time.I believe that data is power
and Machine Learning is something that can unlock this immense potential of data.

1.3 OBJECTIVES OF TRAINING

Project EMOSIC (an emotion based music player) is a novel approach that helps
the user to automatically play songs based on the emotions of the user. It recognizes
the facial emotions of the user and plays the songs according to their emotion. The
emotions are recognized using a machine learning method concept. The human face
is an important organ of an individual‘s body and it especially plays an important
role in extraction of an individual‘s behavior and emotional state. The webcam

Department of Information Technology, SKIT, Jaipur 1

captures the image of the user. It then extract the facial features of the user from the
captured image. Facial expression categorized into 4: happy, sad, angry and neutral.
According to the emotion, the music will be played from the created playlist.

Department of Information Technology, SKIT College, Jaipur 2

Chapter 2

Introduction to Project / Modules

2.1 OVERVIEW OF PROJECT

The most common set of requirements defined by any operating system or soft-
ware application is the physical computer resources, also known as hardware. The
hardware requirements required for this project are:
- Intel i3
- 4GB RAM
- Webcam
- Speaker

Technologies Used are

- Viola Jones Algorithm
- Cloud Platform- Google Colaboratory
- Python 3.6

Department of Information Technology, SKIT, Jaipur 3

2.2 DESIGN OF PROJECT

Figure 2.1: E-R Diagram

Figure 2.2: Block Diagram

Figure 2.3: Flow Diagram

Department of Information Technology, SKIT College, Jaipur 4

2.3 IMPLEMENTATION OF PROJECT

The system uses the following technology to develop the Emotion based Mu-
sic Player i.e., EMOSIC. A Machine learning algorithm- Haar Cascade which is
used to detect the facial expressions, Image Processing, Computer Vision and for
Programming- Python 3.6. Modules used for EMOSIC are Open CV, Webbrowser,
Operating System(OS) and Numpy.

GATHERING OF DATA

When you finally know that exactly what data you want and the regarding equip-
ment are with you, then it takes you to the very first real part of Image Processing
i.e., “Gathering Data”. This part is very crucial as the quality and quantity of the
data you have gathered will directly determine/decide that how good will be the pre-
dictivity of the model will turn out. The data that is collected is then tabulated and
called as “Training Data”.

PREPARATION OF DATA

After the training data is gathered, you will proceed to the next step: Data prepa-
ration, where the data is loaded into a particular place and then it will be prepared
to be used in training. Here, the data is first put all together and then the order is
randomized as the order of data should not affect what is learned. This is also a
good and enough time to do any visualizations of the data, as that will help you to
see if there are any relevant relations between the different variables used, how you
can take their advantage and as well as show you if there is a presence of any data
imbalance .

Department of Information Technology, SKIT College, Jaipur 5

 PREPARATION OF DATA SET

Training After the before steps are completed, you then move onto training where
the data is used to incrementally improve the model’s ability to predict. The training
process involves using the dataset to create an xml file.

EVALUATION

Once training is complete, you now check if it is good enough using this step.
This is where that dataset you set aside earlier comes into play. Evaluation allows
the testing of the model against data that has never been seen and used for training
and is meant to be representative of how the model might perform when in the real
world.

PREDICTION

Machine learning is basically using data to answer questions. So this is the final
step where you get to answer few questions. This is the point where the value of
machine learning is realized. Here you can Finally use your model to predict the
outcome of what you want. The above-mentioned steps take you from where you
create a model to where you Predict its output and thus acts as a learning path.

Department of Information Technology, SKIT College, Jaipur 6

2.4 SNAPSHOTS OF PROJECT

DATASET CREATION

Figure 2.4: Dataset Creation Prog. 1

Figure 2.5: Dataset Creation Prog. 2

Department of Information Technology, SKIT College, Jaipur 7

 XML FILE CREATION

Figure 2.6: XML File Creation Prog

Department of Information Technology, SKIT College, Jaipur 8

 EMOSIC

Figure 2.7: Emosic Prog. 1

Figure 2.8: Emosic Prog. 2

Department of Information Technology, SKIT College, Jaipur 9

Chapter 3

Description of Modules

3.1 FISHER FACE

Fisherface is one of the popular algorithms used in face recognition, and is widely
believed to be superior to other techniques, such as eigenface because of the effort
to maximize the separation between classes in the training process.

The purpose of this research is to establish a program of face recognition appli-

cation using fisherface method by utilizing GUI applications and databases that are
used in the form of a Papuan facial image.

Image recognition using fisherface method is based on the reduction of face space
dimension using Principal Component Analysis (PCA) method, then apply Fisher’s
Linear Discriminant (FDL) method or also known as Linear Discriminant Analy-
sis (LDA) method to obtain feature of image characteristic. The algorithm used in
the process for image recognition is fisherfaces algorithm while for identification or
matching face image using minimum euclidean.

Department of Information Technology, SKIT, Jaipur 10

3.2 OPEN CV

OpenCV (Open Source Computer Vision) is a library of programming functions

mainly aimed at real-time computer vision. In simple language it is library used for
Image Processing. It is mainly used to do all the operation related to Images. What
it can do :

1. Read and Write Images.

2. Detection of faces and its features.

3. Detection of shapes like Circle,rectangle etc in a image. E.g Detection of coin in

images.

4. Text recognition in images. e.g Reading Number Plates

5. Modifying image quality and colors e.g Instagram, CamScanner.

6. Developing Augmented reality apps.

and many more.....

OpenCV is capable of image analysis and processing. This means it is great at

taking frames out of video or taking in two frames from a stereoscopic camera and
run algorithms to extract information. For example using OpenCV would give you
the mathematical tools required to capture images and track a particular object at it
moves around. This is not provided directly but the mathematical tools required to
process the images to extract such information is available.

You can see from the above example that although it can do other things such as
stretch an image or change color, it’s purpose is not that to serve as an image pro-
cessing engine similar to Photoshop or such. It is intended to be very fast almost real
time if the hardware supports it and perform all sorts of functions such as Fourier
transforms very fast and then allow you to either glean information or transform the
stream of images as you like.

Department of Information Technology, SKIT College, Jaipur 11

3.3 HAAR CASCADE

Haar Cascade is a machine learning algorithm used to detect the objects in an

image or video and based on the concept of features proposed by Paul Viola and
Michael Jones in their paper ”Rapid Object Detection using a Boosted Cascade
of Simple Features” in 2001. It is a machine learning based approach where a cas-
cade function is trained with a lot of positive and negative images. It is then used to
identify the objects in other images. It is well known for being able to detect faces
and any part of body in an image, but can be trained to identify almost any object.

Lets take face detection as an example. Initially, the algorithm needs a lot of positive
images of faces and negative images without faces to train the classifier. Then we
need to extract features from it.

First step is to collect the Haar Features. A Haar feature considers adjacent
rectangular regions at a specific location in a detection window, sums up the pixel
intensities in each region and calculates the difference between these sums.

Department of Information Technology, SKIT College, Jaipur 12

3.4 NUMPY

NumPy is the fundamental package for scientific computing with Python. It con-
tains among other things:

• a powerful N-dimensional array object

• sophisticated (broadcasting) functions

• tools for integrating C/C++ and Fortran code

• useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-
dimensional container of generic data. Arbitrary data-types can be defined. This
allows NumPy to seamlessly and speedily integrate with a wide variety of databases.

Numpy is a general-purpose array-processing package. It provides a high-performance

multidimensional array object, and tools for working with these arrays. It is the
fundamental package for scientific computing with Python. Besides its obvious sci-
entific uses, Numpy can also be used as an efficient multi-dimensional container
of generic data. Arrays in Numpy Array in Numpy is a table of elements (usually
numbers), all of the same type, indexed by a tuple of positive integers. In Numpy,
number of dimensions of the array is called rank of the array.A tuple of integers
giving the size of the array along each dimension is known as shape of the array. An
array class in Numpy is called as ndarray. Elements in Numpy arrays are accessed
by using square brackets and can be initialized by using nested Python Lists.

Department of Information Technology, SKIT College, Jaipur 13

3.5 Webbrowser

The webbrowser module provides a high-level interface to allow displaying Web-

based documents to users. Under most circumstances, simply calling the open()
function from this module will do the right thing. Under Unix, graphical browsers
are preferred under X11, but text-mode browsers will be used if graphical browsers
are not available or an X11 display isn’t available. If text-mode browsers are used,
the calling process will block until the user exits the browser. If the environment
variable BROWSER exists, it is interpreted to override the platform default list of
browsers, as an os.pathsep-separated list of browsers to try in order. When the value
of a list part contains the string For non-Unix platforms, or when a remote browser is
available on Unix, the controlling process will not wait for the user to finish with the
browser, but allow the remote browser to maintain its own windows on the display. If
remote browsers are not available on Unix, the controlling process will launch a new
browser and wait. The script webbrowser can be used as a command-line interface
for the module. It accepts a URL as the argument. It accepts the following optional
parameters: -n opens the URL in a new browser window, if possible; -t opens the
URL in a new browser page (“tab”). The options are, naturally, mutually exclusive.

Department of Information Technology, SKIT College, Jaipur 14

Chapter 4

Results

4.1 RESULTS ACHIEVED

At the end of this summer training program,we developed a working- smart music
player which detects ones emotion before playing a song.

• HAPPY FACE - gives us a list of joyful songs

• SAD FACE - gives us a list of emotional songs

• EXCITED FACE - gives us a list of energetic songs

• NEUTRAL FACE - gives us a list of mixed songs

and so on....
The upcoming set of screenshots show, how the program works and gives us
the desired output.

4.2 SCREENSHOTS OF RESULT ACHIEVED

1. This screenshot depicts a sad frowning face.

As soon as this face is detected by the program, a list of songs are selected automat-
ically.

Department of Information Technology, SKIT, Jaipur 15

 Figure 4.1: Project Snap 1

2. This screenshot depicts the command for running this program section.
As soon as this program code is run, face detection starts.

Figure 4.2: Project Snap 2

Department of Information Technology, SKIT College, Jaipur 16

 3. This screenshot depicts a sad happy face.
As soon as this face is detected by the program, a list of songs are selected automatically.

Figure 4.3: Project Snap 3

3. This screenshot depicts all the faces which can be detected by the program.

Figure 4.4: Project Snap 4

Department of Information Technology, SKIT College, Jaipur 17

Chapter 5

Conclusion

5.1 TAKEAWAYS OF TRAINING

The Emotion Based Music Player aims at providing the users with a cheaper,
additional hardware free and accurate emotion based music system. The Emotion
Based Music System will be of great advantage to users looking for music based
on their mood and emotional behavior. It will help reduce the searching time for
music thereby reducing the unnecessary computational time and thereby increasing
the overall accuracy and efficiency of the system. Also with its additional features
mentioned above, it will be a complete system for music lovers and listeners.

5.2 FUTURE SCOPE

The application can be improved by modifying and adding few functionality.

-Making the application run without needing an internet connection.
-Include additional features which helps to categorize user based on many other
factors like location and play songs accordingly.
- Mechanism that would be helpful in music therapy treatment and provide the music
therapist the help needed to treat the patients
-Avoid in future the unpredictable results produced in extreme bad light conditions
and very poor camera resolution.

Department of Information Technology, SKIT, Jaipur 18