M.A.M.

COLLEGE OF ENGINEERING AND TECHNOLOGY

Siruganur,Trichy-621105

TOPIC – SMS SPAM DETECTION USING MACHINE LEARNING

Submitted by

812022205014 - C.GOPAL
812022205020 - A.IRFAN AHAMED
812022205027 – M.MADHAN

ANNA UNIVERSITY
CHENNAI

APRIL 2024

CS3491-AIML PROJECT
CONTENTS:

Abstract
Literature survey
Existing system
Proposed system
Methodology
Implementation
Experimental analysis
Result
Feature Enhancement
Reference

ABSTRACT
 The exponential growth of mobile communication has led to a surge in SMS
spam, posing significant challenges to user privacy and security. In response,
 researchers have explored various machine learning techniques for effective
SMS spam detection. This abstract presents a comprehensive literature
review on the state-of-the-art methods in SMS spam detection using machine
learning algorithms. Traditional rule-based approaches are discussed,
highlighting their limitations in handling evolving spam patterns.
Subsequently, supervised and unsupervised learning algorithms, such as
Naive Bayes, Support Vector Machines, and clustering techniques, are
explored for their effectiveness in classifying spam messages. Feature
extraction methods, evaluation metrics, challenges, and future directions are
also discussed. The comparative analysis of existing approaches sheds light
on the performance and limitations of different techniques. This review
underscores the importance of ongoing research in SMS spam detection to
mitigate the impact of spam messages on users and proposes avenues for
future exploration in the field.

LITERATURE SURVEY:

 Introduction to SMS Spam Detection: Begin with an overview of the problem,

highlighting the growing concern of spam messages infiltrating mobile devices. Discuss
the implications of such spam, including privacy invasion and potential financial scams.
 Traditional Approaches: Explore conventional methods used for SMS spam detection,
such as rule-based filtering and keyword matching. Discuss the limitations of these
approaches, including their inability to adapt to evolving spam techniques and their
susceptibility to false positives.
 Machine Learning Techniques: Delve into the application of machine learning in SMS
spam detection, highlighting the advantages of using algorithms such as Naive Bayes,
Support Vector Machines (SVM), and Random Forests. Discuss how these techniques
can analyze large datasets to identify patterns and classify messages more accurately.

EXISTING SYSTEM

 Data Collection: Gather a large dataset of labeled SMS messages, with each
message labeled as either spam or ham (non-spam)
  Data Preprocessing: Clean and preprocess the data by removing punctuation,
stop words, and performing tokenization.
 Model Training: Train a machine learning model using the extracted features.
Popular algorithms for this task include Naive Bayes, Support Vector Machines
(SVM), and Logistic Regression.

PROPOSED SYSTEM

Applying NB algorithm to the dataset using extracted features with different

training set sizes. The performance in learning curve is evaluated by splitting the
dataset into 70% training set and 30% test set. The NB algorithm shows good
overall accuracy.
We notice that the length of the text message (number of characters used) is a
very
good feature for the classification of spams. Sorting features based on their
mutual
information (MI) criteria shows that this feature has the highest MI with target
labels.
Additionally, going through the misclassified samples, we notice that text
messages with length below a certain threshold are usually hams, yet because of
the tokenscorresponding to the alphabetic words or numeric strings in the
message they might be classified as spams

METHODOLOGY
 The methodology for SMS spam detection using machine learning involves several key
steps. Firstly, a dataset of SMS messages labeled as spam or not spam is collected. These
messages undergo preprocessing, which includes removing special characters, numbers,
and punctuation, converting letters to lowercase, tokenizing words, removing stop
words, and performing stemming or lemmatization. Next, features are extracted from
the preprocessed messages, such as word frequency, TF-IDF, and n-grams.
 A suitable machine learning algorithm, such as Naive Bayes, SVM, or Logistic
Regression, is chosen for classification. The dataset is split into training and testing sets,
and the chosen model is trained on the training set using the extracted features. The
model's performance is evaluated using metrics like accuracy, precision, recall, and F1-
score, as well as through the visualization of a confusion matrix. Hyperparameter tuning
may be performed to optimize the model's performance. Once satisfied, the model is
deployed into production for real-time classification of incoming SMS messages.
Continuous monitoring and maintenance are essential to ensure the model's
effectiveness over time, including periodic retraining with new data and updates to
adapt to changes in the SMS spam landscape.

IMPLEMENTATION
1. For implementing SMS spam detection using machine learning, the first step is to gather
a dataset containing labeled SMS messages, distinguishing between spam and legitimate
messages. Following this, the dataset undergoes preprocessing, including text
normalization, tokenization, and feature extraction.
2. Common techniques like TF-IDF and n-grams are employed to extract meaningful
features from the text. Next, a machine learning model is trained on the extracted
features using algorithms such as Support Vector Machines (SVM), Naive Bayes, or
Logistic Regression.
3. The trained model is then evaluated using appropriate metrics such as accuracy,
precision, recall, and F1-score to assess its performance. Hyperparameter tuning can be
performed to optimize the model's parameters for better results.
4. Once the model achieves satisfactory performance, it can be deployed into production
to classify incoming SMS messages as spam or legitimate in real-time.
5. Regular monitoring and maintenance are crucial to ensure the model's continued
effectiveness, including periodic updates and retraining with new data to adapt to
evolving spam patterns. Additionally, user feedback and manual review mechanisms can
be integrated to improve the model's accuracy and address false positives or false
negatives.

6. To implement SMS spam detection using machine learning, we can follow these steps:

7. Data Preprocessing:

Load the dataset containing SMS messages and their labels (spam or ham) Convert the
labels to numerical form (1 for spam, 0 for ham).Clean the message content by
removing URLs, email addresses, phone numbers, and special characters.Convert all text
to lowercase.Remove stopwords and perform stemming or lemmatization Split the
dataset into training and testing sets.

8. Feature Extraction:
Use a technique like TF-IDF (Term Frequency-Inverse Document Frequency) to convert
text data into numerical features.

9. Model Selection and Training:

Choose a machine learning algorithm to classify SMS messages as spam or ham. Some
popular algorithms for text classification include Naive Bayes, Support Vector Machines
(SVM), and Random Forest.Train the selected algorithm on the training dataset.

10. Model Evaluation:

 Evaluate the performance of the trained model using appropriate evaluation metrics
such as accuracy, precision, recall, and F1-score.

11. The first step in implementing an SMS spam detection system using machine learning is
to collect and preprocess the data. This involves collecting a large dataset of SMS
messages, some of which are labeled as spam and some of which are labeled as ham
(i.e., not spam). The dataset should be balanced, meaning that it should contain roughly
equal numbers of spam and ham messages.

12. Once the data has been collected, it needs to be preprocessed. This involves cleaning
the data by removing any irrelevant information, such as special characters or URLs.
The data should also be normalized, which involves converting all of the text to
lowercase and removing stop words (i.e., common words that do not carry much
meaning, such as "the" or "and").

13. After the data has been preprocessed, it needs to be transformed into a format that can
be used by a machine learning algorithm. This is typically done using a technique called
feature extraction, which involves converting the text data into numerical vectors. One
common approach to feature extraction is to use a bag-of-words model, which involves
representing each message as a vector of word counts.

14. Once the data has been transformed into a numerical format, it can be used to train a
machine learning algorithm. There are many different algorithms that can be used for
SMS spam detection, including Naive Bayes, Support Vector Machines (SVMs), and
Decision Trees. The choice of algorithm will depend on the specific characteristics of the
data and the desired performance metrics

15. Once the algorithm has been trained, it can be used to classify new messages as either
spam or ham. This involves feeding the message into the algorithm and having it output
a probability that the message is spam. If the probability is above a certain threshold, the
message is classified as spam; otherwise, it is classified as ham.

16. Finally, the performance of the algorithm should be evaluated using metrics such as
accuracy, precision, recall, and F1 score. These metrics can be used to compare the
performance of different algorithms and to identify areas for improvement.
Experimental analysis

In the experimental analysis of SMS spam detection using machine learning,

the focus is on evaluating the performance of the trained model and analyzing its
effectiveness in classifying SMS messages accurately. This typically involves
several key components:

1. **Dataset Selection**: Describe the dataset used for the experiments, including
the number of SMS messages, the distribution between spam and legitimate
messages, and any preprocessing steps applied to the data.

2. **Experimental Setup**: Outline the experimental setup, including how the

dataset was split into training and testing sets (e.g., cross-validation or train-test
split), any feature extraction techniques used, and the machine learning
algorithms evaluated.

3. **Performance Metrics**: Define the performance metrics used to evaluate the

model's effectiveness, such as accuracy, precision, recall, F1-score, and the area
under the ROC curve (AUC). Explain why these metrics are relevant for
evaluating SMS spam detection.

4. **Results Analysis**: Present the results of the experiments, including the

performance of each machine learning algorithm on the testing dataset. Compare
the performance metrics of different algorithms and discuss any notable
differences or trends observed.

5. **Discussion**: Analyze the experimental results, discussing the strengths and

weaknesses of each machine learning algorithm tested. Identify factors that may
have influenced the performance of the models, such as the quality of the dataset,
the choice of features, and the complexity of the algorithms.

6. **Comparison with Baseline**: Compare the performance of the machine

learning models with a baseline method, such as a simple rule-based classifier or
random guessing. Discuss whether the machine learning models outperform the
baseline and by how much.
 7. **Robustness and Generalization**: Evaluate the robustness and generalization
ability of the trained model by testing it on unseen data or datasets from different
sources. Discuss any challenges or limitations encountered during this evaluation.

8. **Practical Considerations**: Discuss practical considerations for deploying the

SMS spam detection system in real-world scenarios, such as computational
resources required, model latency, and scalability.

9. **Future Work**: Identify potential areas for future research and improvement,
such as exploring ensemble methods, incorporating domain-specific knowledge,
or integrating feedback mechanisms to adapt the model over time.

Overall, the experimental analysis provides valuable insights into the performance
of machine learning-based SMS spam detection systems and informs decisions
for deployment and further research.

SOURCE CODE

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import accuracy_score, confusion_matrix

# Load SMS data

sms_data = pd.read_csv('sms_data.csv')

# Split data into features (SMS text) and labels (spam or not
spam)
X = sms_data['text']
y = sms_data['label']

# Convert text data into numerical vectors using TF-IDF

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(X)

# Split data into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=42)

# Train a Naive Bayes classifier

clf = MultinomialNB()
clf.fit(X_train, y_train)

# Predict on the test set

y_pred = clf.predict(X_test)

# Evaluate the model

accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)
print("Accuracy:", accuracy)
print("Confusion Matrix:\n", conf_matrix)

Result

The results of our experiment demonstrated that our model achieved a certain level of
accuracy in distinguishing between spam and non-spam messages. However, it's
important to note that the effectiveness of the model could vary depending on factors
such as the quality and size of the dataset, the choice of features and algorithms, and
the preprocessing techniques employed.

Feature Enhancement

One potential feature enhancement for SMS spam detection using machine learning
could involve the integration of advanced natural language processing (NLP)
techniques. By leveraging NLP, the system could analyze the content and context of
 messages more deeply, identifying subtle cues and patterns that indicate spam.
Additionally, implementing a dynamic feedback loop could allow the system to
continuously learn and adapt to new spamming techniques and evolving language
usage, thereby improving its accuracy over time. Furthermore, incorporating user
feedback mechanisms, such as allowing users to manually flag messages as spam,
could provide valuable data for refining the model and enhancing its effectiveness.
By combining these approaches, the SMS spam detection system could achieve
higher precision and recall rates, providing users with a more robust defense against
unwanted messages.

CONCLUSION

 In this project, we implemented SMS spam detection using machine learning

techniques. We utilized a dataset containing SMS messages labeled as spam or
non-spam and employed the TF-IDF (Term Frequency-Inverse Document
Frequency) vectorization technique to convert the text data into numerical
features. We trained a Multinomial Naive Bayes classifier on the training data
and evaluated its performance on a separate test set.

 Furthermore, additional steps could be taken to enhance the performance of the

model, such as experimenting with different classifiers, tuning hyperparameters,
and incorporating more advanced feature engineering techniques like word
embeddings or deep learning architectures.

 Overall, SMS spam detection using machine learning shows promise as an

effective approach to automatically identify and filter out unwanted messages,
thereby improving user experience and security in mobile communication
platforms

REFERENCE
C Oswald, Sona Elza Simon, and Arnab Bhattacharya. Spotspam: Intention analysis–
driven sms spam detection using bert
embeddings. ACM Transactions on the Web (TWEB), 16(3):1–27, 2022.
[2] Sridevi Gadde, A Lakshmanarao, and S Satyanarayana. Sms spam detection using
machine learning and deep learning
techniques. In 2021 7th International Conference on Advanced Computing and
Communication Systems (ICACCS), volume
1, pages 358–362. IEEE, 2021.
[3] Online:Statista (https://www.statista.com/statistics/185879/number-of-text-
messages-in-the-united-states-since-2005/ )
[4] Online: Blog (https://huggingface.co/blog/bert-101)
[5] S Nyamathulla, Polavarapu Umesh, Batchu Rudra Naga Satya Venkat, et al. Sms
spam detection with deep learning model.
Journal of Positive School Psychology, pages 7006–7013, 2022.
[6] Gadde, Sridevi, A. Lakshmanarao, and S. Satyanarayana. "SMS spam detection
using machine learning and deep learning
techniques." 2021 7th International Conference on Advanced Computing and
Communication Systems (ICACCS). Vol. 1.
IEEE, 2021.
[7] Abayomi‐Alli, Olusola, Sanjay Misra, and Adebayo Abayomi‐Alli. "A deep learning
method for automatic SMS spam
classification: Performance of learning algorithms on indigenous dataset." Concurrency
and Computation: Practice and
Experience (2022): e6989.
[8] Github https://github.com/AbayomiAlli/SMS-Spam-Dataset