MAJOR PROJECT REPORT

on

“MRIBased Brain Tumor

Image Segmentation Model”

Bachelor in Technology
in
Computer Science and Engineering

By:-
SAMARTH GARG & KAJAL SINGH
Enrollment No:- A50105221018 & A50105221146

Under the guidance of

Dr. Ekta Soni

Department of Computer Science and Engineering

Amity School of Engineering and Technology
AMITY UNIVERSITY HARYANA, GURUGRAM

Page | 1
 Department of Computer Science and Engineering
Amity School of Engineering and Technology

DECLARATION

We, Samarth Garg & Kajal Singh , student of B.Tech (Computer Science and Engineering)
hereby declare that project entitled “MRI Based Brain Tumour Image Segmentation Model”
which is submitted by us to department of Computer Science & Engineering, Amity School of
Engineering and Technology, Amity University Haryana, in partial fulfilment of the
requirement for that award of the degree of Bachelors in Technology in Computer Science
and Engineering, has not been previously formed the basis for the award of any degree,
diploma or the other similar title or recognition.

Date:

Page | 2
 Department of Computer Science and Engineering
Amity School of Engineering and Technology

CERTIFICATE

This is to certify that work in the project report entitled “MRI Based Brain Tumour Image
Segmentation Model” by SAMARTH GARG (A50105221018) & KAJAL SINGH
(A50105221146) is a Bonafede record of project work carried out by them under my
supervision and guidance in partial fulfilment of the requirement for the award of the degree
of ‘Bachelor in Technology’ VII semester in the Department of Computer Science and
Engineering, Amity School of Engineering and Technology, Amity University Haryana.
Neither this project nor any part of it has been submitted for any degree or academic award
elsewhere.

Date:-

Dr. Ekta Soni

Computer Science and Engineering
ASET,AUH

Page | 3
 ACKNOWLEDGEMENT

We are presenting this project report entitled “MRI Based Brain Tumour Image
Segmentation Model”
We would like to acknowledge the open-source community for their valuable contribution. The
collaborative spirit and sharing of knowledge have greatly enriched my project’s functionalities
and features.
We would like to thank my guide Dr. Ekta Soni for their constant support and guidance without
whom the development of this project would not have been a success. In the last we would
also like to thank all my colleagues who rendered their abilities to the completion of this
project. Project on the topic “MRI Based Brain Tumour Image Segmentation Model” which
also helped us in doing a lot of research and gaining precious knowledge.

Finally, we wish to express our appreciation to our parent for their love and support.

Samarth Garg
Enrollment No- A50105221078

Kajal Singh
Enrollment No- A50105221146

Page | 4
 ABSTRACT

The project is a predictive model that estimates the value or price of a property provided certain
characteristics.

The prediction model is based on muti variate regression which represented mathematically is
an equation that gives us the estimated value of the defendant variable that is the price of the
property in our case based on several characteristics that the user provides.

At its core the model or the regression equation (that we’ll see in the model section) aims to
find a relationship between the price and the characteristics or the features such that every
feature is along with its weight or the importance factor (it determines the degree of influence
of the particular value on the final price estimation) when plugged in to the equation evaluates
to a price that should give us our estimated or targeted price.

For this model I enact as a data scientist working for a real estate development company who
has been tasked tasked with building a model that can provide a price estimate based on a
home's characteristics.

I go through various steps that a typical data scientist would take in order to complete the job.

Page | 5
 List of Figures

Fig 1 MRI 2

Fig 2. T1 10

Fig 3. T1c 10

Fig 4. T2 10

Fig 5. FLAIR 11

Fig 6. BraTS structure 11

Fig 7. Converting path into IDs 12

Fig 8. Splitting the data for training, testing and validation 12

Fig 9. Data distribution 12

Fig 10. Tumour regions 12

Fig 11. Defining segmentation classes 13

Fig 11. Introduction to Neural Networks 14

Fig 12. Activation function 15

Fig 13. ReLU 17

Fig 14. SoftMax 18

Fig 15 Callbacks 18

Fig 16 Training Graph 19

Fig 17 Training Graph 19

Fig 18 Loading MRI 20

Fig 19 Training MRI 20

Page | 6
 TABLE OF CONTENT
Title. Page No.

Declaration ii

Certificate iii

Acknowledgement iv

Abstract v

List of Figures vi

1. Introduction 1

1.1 Aim

1.2 Terminology
1.3 Key Components of Data Science
2. Literature Review 4

2.1 Machine Learning Libraries & Tools

2.2 Magnetic Resonance Imaging

2.3 Segmentation Techniques & CNN Models

2.4 – Datasets

2.5 Model Architecture

2.6 Neural Networks

2.7 Research Gaps and solution

3. Technology Used 9

3.1 Python

3.2 System Requirements

3.3 Importing dataset and loading images

4. Knowing the dataset 11

4.1 Data collection

4.2 Database composition

4.3 Extension of Tumour Regions

4.4 Converting the paths of all the directories (having the patients’ data) into IDs.

Page | 7
 4.5 Splitting the data for Training, Testing and Validation.

4.6 Defining Segmentation Classes

4.7 Info about files format

4.8 Goal

4.9 Extension of tumour region

5. Model 18

5.1 Neural Network

5.2 Learning Algorithm

5.3 Introduction to CNN

5.4 Importing Modules and Libraries

5.5 Data Generation

5.6 Activation function

5.7 Callbacks

5.8 Training the model

6. Evaluation 27

6.1 Evaluation metrics

6.2 Loading self trained model

6.3 evaluation on dataset

6.4 Evaluation on test data

7. Conclusion and future scopes 30

7.1 Challenges

7.2 Future prospects

References 22

Page | 8
 CHAPTER 1: INTRODUCTION

Brain tumour segmentation is an essential procedure in the medical imaging field that aims to
identify and localize cancerous tissues in the brain through imaging modalities. MRI (Magnetic
Resonance Imaging) is the primary modality for imaging brain tissues because of its ability to
produce high-contrast images of soft tissues, which is crucial for visualizing tumor structures
and boundaries. Traditional manual segmentation requires a radiologist to outline the tumor,
which is a time-intensive and subjective task, potentially leading to variability in results.

The need for accurate and efficient tumour delineation has driven research into automated
techniques that leverage deep learning. In recent years, deep learning models, particularly
Convolutional Neural Networks (CNNs), have shown substantial improvements in
segmentation tasks by learning complex patterns in imaging data. This project employs CNN-
based models, especially U-Net, for brain tumour segmentation, aiming to enhance the
precision and speed of diagnosis, which ultimately assists in developing appropriate treatment
plans.[1]

1.1 Aim

The primary goal of this project is to develop an automated method for segmenting brain
tumors from MRI scans, using a CNN-based deep learning model. This model will be trained
on labelled MRI datasets to detect and classify tumor regions accurately. The project’s goal is
to create a tool that can assist medical practitioners in diagnosing brain tumors, reduce
diagnostic time, and potentially improve patient outcomes through early detection. In doing so,
this project aims to advance the field of medical image processing by contributing a scalable
solution that can process MRI scans efficiently and provide reliable segmentation results.

1.2 Terminology

Image Processing: Image processing involves the manipulation and analysis of images using
digital computers. This encompasses a broad range of operations that can be applied to images,
such as enhancing image quality, extracting valuable information, and making images more
suitable for further analysis or interpretation. Image processing techniques are employed in
various fields, including medicine, remote sensing, surveillance, and entertainment.

Page | 9
Image Segmentation: Image segmentation is a fundamental task in image processing and
computer vision. An image is divided into many segments based on specific feature. Main
motive of this pro cess is to divide an image into useful regions that can further be examined
for various purposes.

MRI: MRI stands for magnetic resonance imaging. It is a medical imaging technique that uses
powerful magnetic fields and radio waves to create detailed images of the internal struc tures
of the human body. MRI is especially useful in examining tissues(soft) such as the brain,
muscles, spinal cord and internal organs. [2] The MRI technique involves using aligning
hydrogen atoms in the tissue using a powerful magnetic field. Radio waves are used to disrupt
this arrangement, causing the atoms to produce signals that are examined up by the MRI
scanner. These signals are then pro cessed by computers to create a detailed composite image
of the area being examined. [3]

MRI images offer high-resolution, detailed views of anatomical structures, which assist
healthcare professionals in diagnosing a variety of conditions, including tumors, injuries,
infections, and abnormalities.

Fig 1 MRI

1.3 Key Components of Data science

1. Data Collection: Gathering relevant data from various sources, which can include structured
data (like databases and spreadsheets) and unstructured data (like text, images, and videos).

2. Data Cleaning and Pre-processing: Removing errors, handling missing values,

transforming data into a suitable format, and preparing it for analysis.

3. Exploratory Data Analysis (EDA): Understanding the characteristics of the data by

visualizing and summarizing its key features, relationships, and patterns.

Page | 10
4. Statistical Analysis and Machine Learning: Applying statistical methods and machine
learning algorithms to model and make predictions based on the data.

5. Model Evaluation and Validation: Assessing the performance of models to ensure their
accuracy and reliability.

6. Communication and Reporting: Articulating the results of data analyses to stakeholders in

a clear and understandable manner.

7. Domain Expertise: Understanding the specific context or industry in which the data is being
analysed, which helps in formulating relevant questions and interpreting the results
appropriately. Most data science projects include most if not all components.

Page | 11
 CHAPTER 2: LITERATURE REVIEW

The emergence of data science can be traced back to the late 20th century, when the rapid
advancement of computing technology allowed for the storage and processing of vast amounts
of data. This marked a paradigm shift in data handling capabilities. In the early 2000s,
statisticians, computer scientists, and mathematicians began recognizing the need for
interdisciplinary approaches to extract meaningful insights from this burgeoning sea of data.
They sought to develop methodologies and tools that could handle the scale, variety, and
complexity of data generated by modern systems.

Around this time, the term "data science" was coined to describe this emerging field that
combined elements of statistics, machine learning, computer science, and domain expertise.
The goal was to bridge the gap between data and actionable knowledge.

As data science gained traction, it found applications in a wide range of industries. For instance,
internet companies like Google and Facebook utilized data-driven algorithms to optimize user
experiences and monetization strategies. E-commerce platforms like Amazon leveraged
recommendation systems based on collaborative filtering and predictive modelling.

Furthermore, the advent of open-source software frameworks and platforms, such as Hadoop
and Spark, democratized the processing of big data, making it accessible to a broader audience.
This, in turn, accelerated the adoption of data science across various sectors. (Ming,
Practitioner's Guide to Data Science)

2.1 – Machine Learning Libraries & Tools

Various Python libraries and frameworks are utilized to handle data preprocessing, model
training, and evaluation. NumPy and pandas are fundamental for data handling and
preprocessing. OpenCV, a popular computer vision library, is used for image manipulation
tasks such as resizing and augmenting the MRI images. TensorFlow and Keras are the primary
frameworks used for building and training the CNN models. Keras provides a high-level API
that simplifies the model-building process, allowing rapid prototyping of neural network
architectures. TensorFlow, a powerful backend, provides support for operations such as model
training and inference across multiple devices, including CPUs and GPUs. The integration of
Keras with TensorFlow enables efficient model development and deployment. Additionally,
scikit-learn is used for splitting data into training, validation, and testing sets, as well as for
calculating evaluation metrics.

Page | 12
2.2 – Magnetic Resonance Imaging

Magnetic Resonance Imaging (MRI) is a widely used imaging modality in clinical practice due
to its high resolution and contrast capabilities, especially for soft tissues. MRI works by using
powerful magnetic fields and radio waves to generate detailed images of internal structures.
Unlike other imaging techniques, such as CT scans or X-rays, MRI does not use ionizing
radiation, making it safer for repeated imaging sessions. In brain imaging, different MRI
sequences or modalities, like T1-weighted, T1-post contrast, T2-weighted, and FLAIR, are
used to highlight various anatomical features. Each modality serves a unique purpose; T1-
weighted images provide clear anatomical detail, T1-post contrast highlights tumor boundaries,
T2-weighted images reveal edema, and FLAIR sequences are effective in distinguishing tumor
regions from cerebrospinal fluid. The combined use of these modalities provides a
comprehensive view of the tumor and surrounding brain tissue, making MRI indispensable for
brain tumor diagnosis and monitoring.

2.3 Segmentation Techniques & CNN Models

Manual segmentation techniques, while traditionally used, present limitations in terms of

accuracy, reproducibility, and efficiency. Radiologists often have different interpretations of
tumor boundaries, which can result in inter-observer variability. To address this, automated
image segmentation methods have been developed, with CNNs emerging as the most effective.
Convolutional Neural Networks are designed to detect and learn spatial patterns within images,
making them ideal for image segmentation tasks. In recent years, architectures like U-Net and
its variants have been widely adopted due to their ability to retain spatial information while
capturing intricate features. U-Net, in particular, has a symmetrical encoder-decoder structure
that allows the model to compress image information into lower-dimensional representations
and then reconstruct it for precise segmentation. The use of skip connections enables the model
to combine contextual information from higher layers with spatial details from lower layers,
resulting in highly accurate segmentation outputs. Studies show that U-Net performs
exceptionally well in medical image segmentation tasks, especially in handling datasets with
limited availability, making it a suitable choice for brain tumor segmentation in this project.

Page | 13
2.4 – Datasets

The BraTS (Brain Tumor Segmentation) dataset is used for training and evaluating the model.
BraTS is a widely recognized dataset in the field, consisting of MRI images labeled with tumor
regions. The dataset includes different MRI modalities—T1, T1-CE, T2, and FLAIR—
providing multiple views that highlight various tumor characteristics, such as edema, necrotic,
and enhancing tumor areas. This diverse dataset enables the model to learn from a wide range
of tumor presentations, improving its generalizability to real-world data.

2.5 Model Architecture

The U-Net model was chosen for its robust encoder-decoder architecture, tailored for semantic
segmentation tasks. The U-Net model is characterized by an encoder path that reduces image
dimensions while extracting feature maps, followed by a decoder path that restores these
dimensions for detailed segmentation. Each level in the encoder path captures increasingly
abstract features, while the decoder path reconstructs these into a segmentation mask that aligns
with the input image’s dimensions. Skip connections between corresponding layers in the
encoder and decoder paths improve the model’s ability to retain context, ensuring the
segmentation of even fine details in the MRI images. This architecture has proven effective in
medical image segmentation, where accuracy in differentiating boundaries between tumor and
non-tumor regions is crucial.[8]

2.6 Neural Networks

A neural network is a computational model inspired by the neural structures and functions of
the human brain. It consists of interconnected systems, or “neurons, that work together to
recognize patterns and solve complex problems. . This is why deep learning is sometimes called
deep neural learning or deep neural networks (DNN). Input nodes act as layers through which
data is entered. The number of output processes and nodes required for each output is different.
For example, an output of “yes” or “no” should have only two nodes, while an output with
more information would need to have many nodes. Layers are the layers that process data and
send it to other layers in the neural network.

Page | 14
2.7 Research Gaps and solutions

In the initial stages of the project on Brain Tumor Segmentation Using MRI, several gaps and
challenges were identified that affected the model's performance and overall project outcomes:

1. Limited Dataset and Data Imbalance Gap: Initially, the dataset used had limited
diversity in tumor types, sizes, and MRI modalities, leading to poor generalization of
the segmentation model.

Solution: we expanded the dataset by incorporating publicly available datasets like

BraTS (Brain Tumor Segmentation Challenge) datasets, which provided multi-modal
MRI scans (T1, T2, FLAIR, etc.). Additionally, I performed data augmentation
techniques such as rotation, flipping, scaling, and elastic transformations to
synthetically balance the data.[4,5]

Tools/Techniques Used: Python libraries (Albumentations, OpenCV, NumPy)

Impact: This significantly improved the model's ability to generalize across different
MRI types and tumor shapes, leading to better segmentation accuracy.

2. Inadequate Model architecture: The initial model architecture was a basic

Convolutional Neural Network (CNN) with limited depth, resulting in poor
segmentation precision, especially around the tumor boundaries.
Solution: I switched to a U-Net based architecture, known for its strength in biomedical
image segmentation.
Tools/Techniques Used: TensorFlow and Keras libraries, U-Net model
Impact: The encoder-decoder structure of U-Net improved boundary detection and
preserved spatial information, leading to a noticeable increase in Dice Coefficient and
IoU (Intersection over Union) metrics.

3. Overfitting of the Model Gap: During training, the model showed excellent
performance on training data but poor performance on validation data, indicating
overfitting.
Solution: I implemented techniques such as Dropout layers, L2 regularization, and
early stopping. Additionally, cross-validation was introduced to ensure model
robustness.
Tools/Techniques Used: Keras callbacks (EarlyStopping), regularization techniques

Page | 15
 Impact: These changes stabilized the training process and improved the model's ability
to generalize to unseen data.

4. Poor Preprocessing and Noise in Image: MRI images contained artifacts and intensity
variations, which confused the model during segmentation.
Solution: I applied preprocessing techniques such as skull stripping, intensity
normalization, and denoising filters before feeding images into the network.

Tools/Techniques Used: SimpleITK, scikit-image, and custom Python scripts

Impact: Cleaner inputs improved model focus on relevant tumor regions, enhancing
overall segmentation performance.

5. Suboptimal Evaluation Metrics : Initially, the model evaluation was limited to simple
pixel accuracy, which did not reflect the true segmentation quality.
Solution: I adopted more comprehensive evaluation metrics like Dice Coefficient,
Jaccard Index (IoU), Precision, Recall, and Hausdorff Distance.
Tools/Techniques Used: Python's scikit-learn metrics module
Impact: Using appropriate metrics provided deeper insights into model performance,
guiding more effective model improvements.[6,7]

Page | 16
 CHAPTER 3: TECHNOLOGY USED

In response to the gaps and challenges identified during the initial phases of the project, it
became essential to adopt specific technologies, tools, and frameworks that could effectively
overcome the limitations and enhance the model’s performance.
This chapter outlines the key technologies and resources utilized — including programming
languages, libraries, and machine learning frameworks — that were critical in addressing the
research gaps and achieving more accurate brain tumour segmentation results.

3.1 Python

Python is a high-level, versatile programming language that has gained immense popularity in
recent years. Guido van Rossum released the language in the late 1980s, and since then, it has
become a cornerstone in various fields of computer science and technology. Known for its
simplicity, readability, and powerful libraries, Python has become the go-to choose for
developers, researchers, and data scientists around the world.

One of Python’s standout features is its simplicity and clean syntax. The language prioritizes
readability, making it easy for developers to write and understand code. This simplicity
accelerates the development process and reduces the potential for errors, particularly in
complex projects. Python’s dynamic typing system allows for flexibility in variable usage,
enabling developers to create dynamic and adaptable applications.

Python’s versatility is evident in its broad range of applications. In web development,

frameworks like Django and Flask empower developers to create robust and scalable web
applications. In the realm of data science and analytics, Python reigns supreme, with libraries
like Pandas, NumPy, and Matplotlib providing powerful tools for data manipulation, analysis,
and visualization.

3.2 SYSTEM REQUIREMENTS

3.2.1 Hardware requirement.

3.2.2 For windows:

• 64-bit Microsoft® Windows® 8/10/11

• x86_64 CPU architecture; 2nd generation Intel Core or newer, or AMD CPU with
support for a Windows Hypervisor

Page | 17
 • 8 GB RAM or more
• 8 GB of available disk space minimum (IDE + Android SDK + Android Emulator)

3.2.3 For macOS:

• MacOS® 10.14 (Mojave) or higher

• ARM-based chips, or 2nd generation Intel Core or newer

with support for Hypervisor. Framework

• 8 GB RAM or more
• 8 GB of available disk space minimum (IDE + Android SDK + Android Emulator)

• 1280 x 800 minimum screen resolution

3.2.4 Software Requirements

3.2.5. Text Editor/IDE: Choose a text editor or Integrated Development Environment (IDE)
of your preference. Popular options include Visual Studio Code or Jupyter Notebooks.

3.3 Importing Dataset and Loading Images

NumPy: This is a simple package for Python arithmetic. It supports many variables and
numbers and provides many arithmetic operations for arrays.–

pandas: The library for data manipulation and analysis, provides data structures such as
DataFrame and Series, which are necessary for the management and analysis of appropriate
data.[15 ]

statsmodels: This library provides classes and functions for statistical model estimation and
hypothesis testing. Together, these libraries create a powerful ecosystem that facilitates sci
entific computing and data analysis, enabling researchers and engineers to perform complex
computations and visualizations efficiently

matplotlib: A library for creating static, interactive visualisations in Python. It is widely and
often used for data visualization.

Page | 18
 CHAPTER 4: KNOWING DATASET

A crucial step in overcoming the research gaps involved working with a more diverse and
comprehensive dataset. Understanding the structure, modalities, and labelling of the MRI data
was essential for building a model capable of accurate tumour segmentation.
This chapter provides detailed insights into the dataset used, highlighting how careful selection
and preprocessing of the data played a pivotal role in improving model performance

4.1 Data collection

BraTS Dataset: The BraTS dataset, short for Brain Tumour Segmentation dataset, is widely
recognised in the medical imaging field, particularly for brain tumour segmentation tasks. Here
are some key details about the BraTS dataset.

Overview

1. Objective: The BraTS dataset aims to advance the development and evaluation of algorithms
for segmenting brain tumours from MRI scans, focusing on gliomas, which are a type of brain
tumour with high variability.

2. Challenges: The dataset tackles issues such as the varied appearance of tumours, the presence
of different tumour subregions (like the enhancing tumour, peritumoral edema, and necrotic
core), and differences in imaging protocols.

Context

The BraTS database is also dedicated to evaluating techniques for brain tumour 2.2. DATASET
COMPOSITION 5 segmentation in multimodal MRI scans. BraTS 2020 focuses mainly on
segmenting gliomas that differ in appearance, image and histology. Given the clinical
importance of the project, BraTS 2020 also mentions the prediction of patient overall survival
through machine learning algorithms. Finally, BraTS 2020 aims to evaluate algorithmic
uncertainty in tumour segmentation.[8]

Access and Usage

The dataset is publicly available for research purposes, typically requiring registration and
acceptance of usage terms to access the data. Researchers are encouraged to share their
findings, contributing to collective progress in the field provided that they mention all the
references/citations.[9]

Page | 19
4.2 Database composition

When doctors utilise an MRI machine, they can adjust it to capture various types of images
known as modalities. Here are some types of images (modalities) that an MRI machine can
capture:

1. T1-weighted (T1): This image is akin to taking a standard photograph, showing the basic
structure inside the body. T1 is effective for distinguishing tumors from healthy brain tissue
and highlights non-enhancing, enhancing, and necrotic tumors.

Fig 2. T1

2. T1-Post contrast enhanced (T1c): Similar to the T1-weighted image, but taken after
injecting a special dye (contrast agent) into the body. This dye enhances the visibility of certain
areas, making tumour boundaries clearer.

Fig 3. T1c

3. T2-weighted (T2): This image reveals different details compared to T1, much like viewing
things through a different lens. In T2 images, the edema (fluid) around a brain tumour is visible.

Page | 20
 Fig 4. T2

4. T2-weighted fluid-attenuated inversion recovery (FLAIR): A specialized type of T2

image, FLAIR is excellent for identifying issues like brain injuries or diseases. It is particularly
useful for detecting edema regions distinct from cerebrospinal fluid (the fluid surrounding the
brain and spinal cord).

Fig 5. FLAIR

The objective of brain tumour segmentation is to identify the position and extent (sever ity) of
the tumour areas, including active tumorous tissue (whether vascularised or not), necrotic
tissue, and edema (swelling around the tumour).

4.3 Extension of Tumour Regions

1. Peritumoral edema: This refers to the swelling of brain tissue surrounding the tumour.
Peritumoral edema typically forms in meningiomas (tumours originating from the meninges,
which are the protective layers surrounding the brain and spinal cord) and spreads from the
nearby tumour.

2. Necrotic: Necrosis refers to the death of cells within the tumour.

3. Non-enhancing tumour: This refers to parts of the tumour that do not show enhancement
after administering a contrast agent during imaging studies. Note: Non-enhancing and necrotic
tumours are both considered core tumours and appear hypo-intense in T1-weighted images. 4.
Enhancing tumour Enhancing tumour segments are areas of the tumour that show increased
contrast enhancement after administering a contrast agent.

Page | 21
 Fig 6. BraTS structure

4.4 Converting the paths of all the directories (having the patients’ data) into IDs.

Fig 7. Converting path into IDs

4.5 Splitting the data for Training, Testing and Validation.

Fig 8. Splitting the data for training, testing and validation

Page | 22
 Fig 9. Data distribution

4.6 Defining Segmentation Classes

Defining segmentation classes- Extension of Tumour Regions, using Python Dictionary for
classification or prediction purposes

Fig 10. Defining segmentation classes

4.7 Info about files’ format

• All BraTS multimodal scans are available as NIFTI files (.nii.gz) and include: a) native
(T1), b) postcontrast T1-weighted (T1Gd), c) T2-weighted (T2), and d) T2 fluid-
attenuated reverse transduction recovery (T2-FLAIR) volume. Analyzes were obtained
using different clinical systems and different printers from different organizations listed
as data providers (n=19).
• All image files were classified by one to four observers following the same annotation
rules. This statement is confirmed by experienced neuroradiologists. Descriptions
include GD enhancing tumor (ET- Tag 4), peritumoral edema (ED- Tag 2), and necrotic

Page | 23
 and non-growing tumor core (NCR/NET- Tag 1). The data is first provided, i.e.
integrated into the same anatomical template, interpolated 1mm3 at the same resolution.

4.8 Goal

The objective of brain tumour segmentation is to identify the position and extent (sever ity) of
the tumour areas, including active tumorous tissue (whether vascularised or not), necrotic
tissue, and edema (swelling around the tumour).

4.9 Extension of tumour regions

1. Peritumoral edema

This refers to the swelling of brain tissue surrounding the tumour. Peritumoral edema typically
forms in meningiomas (tumours originating from the meninges, which are the protective layers
surrounding the brain and spinal cord) and spreads from the nearby tumour.

2. Necrotic

Necrosis refers to the death of cells within the tumour.

3. Non-enhancing tumour

This refers to parts of the tumour that do not show enhancement after administer ing a contrast
agent during imaging studies.

Note: Non-enhancing and necrotic tumours are both considered core tumours and appear hypo-
intense in T1-weighted images.

4. Enhancing tumour

Enhancing tumour segments are areas of the tumour that show increased contrast enhancement
after administering a contrast agent.

Note: Enhancing tumour segments usually appear hyper-intense in T1-weighted im ages

Page | 24
Fig 10. Tumour regions

Page | 25
 CHAPTER 5: MODEL

Based on the gaps identified earlier, a more advanced and specialized model was selected and
customized to improve segmentation accuracy and boundary detection.
This chapter discusses the model architecture, its components, and the reasons behind choosing
this specific design to overcome the initial project limitations.[15]

5.1 Neural Network

A neural network is a computational model inspired by the neural structures and functions of
the human brain. It consists of interconnected systems, or “neurons”, that work together to
recognize patterns and solve complex problems. . This is why deep learning is sometimes called
deep neural learning or deep neural networks (DNN). Input nodes act as layers through which
data is entered. The number of output processes and nodes required for each output is different.
For example, an output of “yes” or “no” should have only two nodes, while an output with
more information would need to have many nodes. Layers are the layers that process data and
send it to other layers in the neural network.[16]

Fig 11. Introduction to Neural Networks [13]

5.2 Learning Algorithm

The learning algorithm comprises two main components: Backpropagation and Optimization.

Page | 26
 • Backpropagation is a method used to calculate the slope of a loss function(gradient)
with respect to weight.
• Optimization involves selecting the most appropriate option from available methods.
In our case it should choose the Apt weight and bias for the perceptron. We choose
gradient descent as the optimization method, where the weight and bias are adjusted
proportional to the negative gradient of the weight or unfairness function value. The
learning value α is used as a hyperparameter to adjust the degree of change of weight
and bias.

5.3 Introduction to CNN

Convolutional Neural Networks, also referred to as CNNs or ConvNets, are a type of feed-
forward artificial neural network. Their connectivity structure is inspired by the organization
of the visual cortex in animals, which is the part of the brain that recognizes retinal images.

In essence, the primary role of a ConvNet is to transform images into a form that is easier to
process, while preserving essential features that are crucial for making accurate
predictions.[17]

Fig 12. Activation function

5.4 Importing Modules and Libraries

1. MinMaxScaler from sklearn.preprocessing: MinMaxScaler is a feature scaling

technique provided by the sklearn.preprocessing module. It is used to scale the data to
a specific range, usually between 0 and 1. This is particularly useful in training neural
networks, including Convolutional Neural Networks (CNNs), as it helps in speeding up
the convergence of the model during training by normalizing the data.

Page | 27
 Usage in CNN for Image Segmentation: WhenusingCNNsforimage segmentation, the
pixel values of images can vary significantly. MinMaxScaler is applied to scale these
pixel values to a range of 0 to 1. This ensures that the neural network trains more
efficiently and the gradients do not become too small or too large, which can adversely
affect the learning process
2. Image, ImageOps from PIL - PIL, which stands for Python Imaging Library, is a
library in Python that allows for opening, manipulating, and saving many different
image file formats. Image and ImageOps are modules within PIL. ImageOps module
provides a collection of image processing operations such as grayscale conversion,
flipping, and padding1. These operations are often used in preprocessing steps to
prepare images for segmentation tasks. For instance, con- verting an image to grayscale
can help in reducing the computational complexity, and padding can ensure that the
images are of uniform size before being fed into the CNN.
3. Keras - Keras is a powerful application programming interface (API) that allows the
use of both definitions and models. For example, the Bureau of Meteorology’s software
system stores daily weather data. The Weather App on your phone communicates with
this system via API to provide you with new phone numbers. This open source library
is designed to facilitate rapid experiments with deep neural networks. Keras
demonstrates ease of use. Although this integration can work independently like other
different libraries.[18]
4. Tensorflow - Developed by Google and released in 2015, TensorFlow is an end-to-end
open source deep learning solution. It is known for its comprehensive knowledge and
educational support, flexible development and deployment, multi-level technical
expertise, and support for multiple platforms, including Android. It is suitable for neural
networks and data flow design in a variety of tasks. It provides multiple levels of
abstraction for building and training models. It helps build and distribute machine
learning applications[19].

5.5 Data Generation

Data generation refers to the process of creating batches of data to feed into a neural network
during training. In the context of Convolutional Neural Networks (CNNs) used for image
segmentation, data generation involves loading and preprocessing images, dividing them into
smaller batches, and applying any necessary transformations such as normalization and

Page | 28
augmentation. This ensures that the model receives a steady stream of properly formatted data,
which helps in efficient and effective training.

1. Constants Definition: Volume Slices: Number of slices per volume. Volume-

Start-At: Index of the first slice to include in each volume. Img-Size: Size of the
images after resizing
2. Data Generator Class: This class inherits from keras.utils.Sequence and is
responsible for generating batches of data for training the
CNN.‘keras.utils.Sequence’ is a utility class pro vided by Keras to create custom
data generators for training neural networks. It’s a subclass of Python’s built-in
‘collections.Sequence’, behaving like a list or tuple, supporting indexing and
slicing operations. By subclassing keras.utils.Sequence, you can implement
methods like len, getitem, and onepochend to customize your data generator’s
behavior. This class is particularly useful for handling large datasets that cannot
be loadedentirely into memory, enabling efficient loading and processing of data
in batches during training.

It initializes with parameters like image dimensions, batch size, number of

channels, and whether to shuffle the data.
• The ‘__len__’ method calculates the number of batches per epoch.
• The ‘__getitem__’ method generates one batch of data.
• The ‘on_epoch_end’ method updates the indexes after each epoch.
• The ‘__data_generation’ method generates the actual data for a batch.
3. Data Generation: Within the ‘__data_generation’ method, for each case in the
batch, it loads the FLAIR (Fluid Attenuated Inversion Recovery) and CE
(Contrast Enhanced) MRI scans along with the corresponding segmentation
mask.
It then resizes each slice of the FLAIR and CE scans to the specified image size
and stores them in the input array X.
The segmentation masks are stored in the output array y.[20]
4. Preprocessing: After generating the data, it preprocesses the segmentation
masks: It assigns the value 3 to any pixels with a label of 4 in the segmentation
mask. This is how 4 segment classes gets converted into 3.

Page | 29
 Converts the segmentation mask into ‘one-hot’ encoded format.one-hot is a
func tion provided by TensorFlow and Keras for converting integer labels into
one-hot encoded vectors. One-hot encoding is a common technique used in
multi-class clas sification problems, where each class is represented by a binary
vector; where ”1” represents the class and all other elements are set to ”0”.
In neural networks, one-hot encoding is frequently employed to encode
categorical labels, such as class labels in classification tasks. Resizes the one-
hot encoded masks to match the image size.
Finally, it returns the pre-processed input and output data.
5. Data Generators: Three instances of the Data Generator class are created for
training, validation, and testing datasets, respectively[21]

5.6 Activation function

An Activation Function in neural networks determines whether a neuron should be ac tivated,

meaning it decides whether the neuron’s input to the network is significant for the prediction
task. It applies a mathematical operation to the neuron’s input, which influences the network’s
learning process and performance.

1. ReLU (Rectified Linear Unit):

The ReLU activation function is widely used in many types of neural networks. It’s a simple
function that acts like a switch. If the input is positive, it just gives back that same value. But
if the input is negative, it says” zero” instead . This simple operation has a profound impact on
training neural networks.[22.23] ReLU helps to address the vanishing gradient problem, where
gradients become exceedingly small and slow down learning in deeper networks. By only
allowing positive values to pass through, ReLU introduces non-linearity, which enables the
network to learn complex patterns. Because ReLU is computationally efficient and often results
in faster training and better performance, it has become the default choice for many neural
network architectures.[24]

Page | 30
 Fig 13. ReLU

2. SoftMax:

Towards the end of a Convolutional Neural Network (CNN), there’s a Dense Layer made of
neurons. Once trained, the feature vector from this layer helps classify images into different
groups. Each neuron in the next layer is linked to every input from the current layer. This setup
can lead to overfitting because all the parameters are involved in this fully connected layer. To
tackle this, methods like dropout are used to lessen overfitting. [23,24]

SOFTMAX is like a finishing touch often added to the last layer of a network. It helps classify
things into different groups. It is specifically used for multi-class classification problems. The
SoftMax function converts the raw output scores (logits) from the network into probabilities,
effectively turning the output into a probability distribution over multiple class. This makes it
easier to interpret the network’s predictions, as the class with the highest probability can be
selected as the predicted category

Fig 14. SoftMax

Page | 31
5.7 Callbacks

Callbacks in Keras are functions used to monitor and control the training process of a neural
network. They help in saving the best model, adjusting learning rates, stopping training early
if required, and logging training metrics.

Fig 15 Callbacks

1. csv logger = CSVLogger(‘training.log’, separator=‘,’, append=False): This line creates

a callback that logs training metrics to a CSV file named ‘training.log’. The separator between
values is set to ‘,’ and if the file already exists, it will be overwritten as ‘append’ is set to False.
[25]

2. Checkpoint: This line creates a callback that saves the model’s weights to ‘best model.h5’
when the validation loss decreases. The ‘monitor’ parameter specifies the metric to watch (‘val
loss’ here), ‘verbose’ controls the amount of output, ‘save best only’ ensures only the best model
is saved, and ‘mode’ determines whether to minimize or maximize the monitored metric.

3. reduce_lr [27]: This line creates a callback that reduces the learning rate by 0.2 if the
validation loss doesn’t improve for 2 consecutive epochs. Parameters like ‘monitor’, ‘factor’,
‘patience’, ‘min lr’, and ‘verbose’ are set to control its behaviour.

4. Early Stopping [28,28]: This line creates a callback to stop training if validation loss doesn’t
improve for 5 consecutive epochs. Parameters like ‘monitor’, ‘patience’, ‘verbose’, and
‘restore best weights’ are set to control its behaviour.

5. callbacks = [checkpoint, reduce lr, early stopping, csv compiles all the callbacks into a list
to be used during training

5.8 Training the Model

Page | 32
The ‘model.fit()’ function in TensorFlow Keras is used to train a neural network model. It takes
input data (training_generator) along with optional validation data (valid_generator), the
number of epochs to train for (epochs), and a list of callbacks to be applied during training.[30]

Fig 16 Training Graph

Fig 17 Training Graph

Page | 33
 Fig 18 Loading Images

Fig 19 Segmented Images

Page | 34
 CHAPTER 6: EVALUATION

It focuses on the evaluation metrics and testing strategies used to measure the model’s
effectiveness, precision, and reliability in segmenting brain tumors from MRI images.
The results highlight the impact of the technological and methodological changes made during
the project.

6.1 Evaluation Metrics

Purpose: Metrics are utilized to assess the model’s performance. Unlike the loss func tion, they
do not impact the training process (i.e., they don’t affect backpropagation).

1. ‘accuracy’:

This metric measures the percentage of correct predictions. It is a common metric for
classification tasks.

2. dice_coef :

This custom metric calculates the Dice coefficient for the entire segmentation task, offering a
direct measure of segmentation performance.

3. dice_coef_necrotic, dice_coef_edema, dice_coef_enhancing :

These custom metrics compute the Dice coefficient specifically for the necrotic, edema, and
enhancing regions, respectively. This provides a more detailed perfor mance evaluation for
each class. [31,32,33]

4. The Mean Intersection over Union:

(Mean IoU or mIoU) is a widely used evaluation metric for image segmentation tasks, aimed
at determining how well the predicted segmentation matches the ground truth segmentation.

Page | 35
6.2 Loading self-trained model:

First 2 files have automatically been saved as we called callbacks on our model and Other 2
files are imported from a source [26,34]. They belong to a pre-trained model showing better
performance.

6.3 Evaluation on test data

6.4 Evaluation on test data

Page | 36
Figure 4.3: Hence this is model is a better one with lower value of test loss = 0.40,
Accuracy = 0.99 and Mean IoU = 0.83
Test loss is as high as validation loss. Our model is working well and is not over-fitted. But
since loss function tells us that how much is the difference between prediction and ground truth,
the value should be as low as possible.[35]

Page | 37
 CHAPTER 7: CONCLUSION AND FUTURE SCOPES

The primary objective of this project is to develop a robust brain tumor segmentation model
using MRI scans. This project automatically identifies and outlines tumor regions in MRI
images, which is essential for accurate diagnosis and treatment planning. It uses deep learning
methods, specifically convolutional neural networks (CNNs), to accomplish this.

7.1Challenges

1. Ensuring the model generalizes well across different patients and MRI modalities.
2. Handling imbalances in the dataset where certain tumor types or regions may be
underrepresented.
3. Fine-tuning the model to improve accuracy while avoiding overfitting. In this project
we learnt about the process of automating the brain tumor segmentation using deep
learning.

Here are some critical points of discussion:

1. Performance Metrics: The Dice coefficient and Mean IoU are crucial for assessing the
model’s seg mentation accuracy. While the Dice coefficient provides a direct measure of
overlap between predicted and ground truth masks, Mean IoU offers a more comprehensive
evaluation across different classes of tumor regions.

2. Data Augmentation: Incorporating data augmentation techniques such as rotation, flipping,

and scaling could further enhance the model’s robustness by exposing it to a wider variety of
scenarios.

3. Model Architecture: Experimenting with different CNN architectures, such as U-Net or its
variants, could potentially yield better segmentation results. These architectures are specifically
designed for image segmentation tasks and might offer improved performance.

4. Transfer Learning: Utilizing pre-trained models on large medical imaging datasets and
fine-tuning them on the brain tumor dataset could accelerate the training process and im prove
accuracy, especially when the dataset is limited,[36]

7.2 Future prospects:

• Advanced Model Architectures Exploring advanced model architectures like Attention

U-Net, U-Net++, and transformer-based models can improve segmentation precision.

Page | 38
 • 3D Segmentation
Transitioning to 3D segmentation models can enhance spatial context understanding
and lead to better segmentation results.
• Transfer Learning and Pre-trained Models
Utilizing pre-trained models and transfer learning can boost performance, especially
with limited data.
• Data Augmentation and Synthesis
Advanced data augmentation and synthesis methods can increase dataset diversity, im
proving model robustness. 5.2.0 Integration with Clinical Workflows Developing user-
friendly interfaces and integration with clinical systems is crucial for practical
deployment. [37]
• Multimodal Data Fusion
Fusing data from multiple imaging modalities can provide a comprehensive view of
tumours and improve segmentation accuracy.
• Continuous Learning and Model
Updating Implementing continuous learning frameworks ensures models remain
relevant and accurate over time.
• Post-processing Techniques
Incorporating post-processing methods like CRFs can refine segmentation outputs for
better accuracy.
• Collaboration and Open Research
Encouraging collaboration and open research initiatives fosters innovation and acceler
ates progress in the field. By addressing these future prospects, brain tumor
segmentation can advance towards developing reliable tools for improved patient
care.[38]

Advanced Model Architectures Exploring advanced model architectures like Attention U-Net,
U-Net++, and transformer-based models can improve segmentation precision. 5.2.0 3D
Segmentation Transitioning to 3D segmentation models can enhance spatial context
understanding and lead to better segmentation result.

Page | 39
 REFERNCES

[1] D. Daimary, M. B. Bora, K. Amitabh, D. Kandar, Brain Tumor Segmentation from MRI
Images Hybrid Convolutional Neural Networks, 2020.

[2] B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al. ”The

Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)”, IEEE Transactions on
Medical Imaging, 2015.

[3] S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. Kirby, et al., ”Segmentation

Labels and Radiomic Features for the Pre-operative Scans BIBLIOGRAPHY 74 of the
TCGA-GBM collection”, The Cancer Imaging Archive, 2017.

[4] S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J.S. Kirby, et al., ”Ad- vancing
The Cancer Genome Atlas glioma MRI collections with expert segmenta- tion labels and
radiomic features”, Nature Scientific Data, 4:170117 (2017

[5] S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, et al., ”Identifying the
Best Machine Learning Algorithms for Brain Tumor Segmentation, Progres- sion
Assessment, and Overall Survival Prediction in the BRATS Challenge

[6] M. Kuhn and K. Johnson Applied Predictive Modelling Volume 615 (2013)

[7] A. Chattopadhyay, M. Maitra, MRI-based brain tumour image detection using CNN based
deep learning method, 100060 (2022).

[8] What is data science, amazon web services

https://builtin.com/data-science/data-science-applications-examples

[9] pandas – Python data analysis library, documentation

https://pandas.pydata.org/

[10] scikit-learn – Machine Learning in python. Documentation

https://scikit-learn.org/stable/

[11] NumPy documentation

https://numpy.org/doc/stable/

[12] seaborn: statistical data visualization, documentation https://seaborn.pydata.org/

[13] https://towardsdatascience.com/introduction-to-math-behind-neural-n etworks-

e8b60dbbdeba

[14] https://www.analyticsvidhya.com/blog/2022/03/basics-of-cnn-in-dee p-learning/

Page | 40
[15] https://www.baeldung.com/cs/epoch-neural-networks#:~:text=An%20epoc
h%20means%20training%20the,to%20train%20the%20neural%20network

[16] https://www.simplilearn.com/keras-vs-tensorflow-vs-pytorch-article
#:~:text=TensorFlow%20is%20an%20open%2Dsourced,runs%20on%20top%20of%
20TensorFlow.

[17] https://www.jeremyjordan.me/semantic-segmentation/

[18] https://www.analyticsvidhya.com/blog/2021/05/convolutional-neural-n etworks-cnn/

[19] https://www.analyticsvidhya.com/blog/2022/03/basics-of-cnn-in-dee p-learning/

[20] A. Chattopadhyay, M. Maitra, MRI-based brain tumour image detection using CNN
based deep learning method, 100060 (2022)

[21] Martin Riva, Batch Normalization in Convolutional Neural Networks, Bael-

dung:https://www.baeldung.com/cs/batchnormalizationcnn#:~:text=3.,learning%20rates%2C
%20making%20learning%20easier.,https://blog.paperspace.com/batch-normalization-in-
convolutional-neural-networks/,https://arxiv.org/pdf/1502.03167

[22] https://medium.com/@minions.k/optimization-techniques-popularly-used-in-deep-
learning-3c219ec8e0cc

[23] https://www.analyticsvidhya.com/blog/2022/10/image-segmentation-with-u-net/

[24] https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/

[25] https://naomi-fridman.medium.com/multi-class-image-segmentation-a5cc671e647a

[26] https://www.kaggle.com/code/basmaahmed159/segmentation-code8/noteboo

k#Create-model-||-U-Net:-Convolutional-Networks-for-Biomedical-Image-Segmentation

[27] cnblogs.com https://www.cnblogs.com/hansjorn/p/12611235.html

[28] fastercapital.com https://fastercapital.com/keyword/ct-scans.html

[29] techtarget.com https://www.techtarget.com/searchenterpriseai/definiti

on/deep-learning-deep-neural-network?_ga=2.153235583.61550829.1611884050-
1899821551.1611884050

[30] baeldung.co https://www.baeldung.com/cs/batch-normalization-cnn

Page | 41
[31] analyticsvidhya.com
https://www.analyticsvidhya.com/blog/2022/10/imagesegmentation-with-u-net/

[32] github.com https://github.com/turhancan97/Neural-Network-Training-on-Matlab

[33] springer.com https://link.springer.com/chapter/10.1007/978-1-4614-6849-3_1

[34] med.upenn.edu https://www.med.upenn.edu/cbica/brats2020/data.html

[35] sciencedirect.com
https://www.sciencedirect.com/science/article/pii/S277252862200022X?via%3Dihub

[36] jai.front-sci.com https://jai.front-sci.com/index.php/jai/article/view/1403

[37] altexsoft.medium.com https://altexsoft.medium.com/unstructured-data-examples-tools-

techniques-and-best-practices-c0fefa57f741

[38] A. Chattopadhyay and M. Maitra, MRI-based Brain Tumor Image Detection Using

CNN based Deep Learning Method, Neuroscience Informatics 2 4:100060 (2022);

https://www.researchgate.net/publication/358864457_MRIbased_Brain_Tumor_Image_Detec
tion_Using_CNN_based_Deep_Learning_Method

Page | 42