Sentiment analysis or opinion mining meaning is that understanding the emotion of the text
whether it is positive, negative or neutral using text analysis techniques. Sentiment analysis or
opinion mining can be useful for various business, literature survey, movie reviews, governing bodies
as well as for news reporters. It can be used for knowing the review of a newly launched product or if
a new law is passed by the government, to understand the mass opinion by using this. The statistics
obtained from the sentiment analysis can be used by television debaters to back his/her argument.
Moreover opinion mining or sentiment analysis is used for building chat-bots as well.
Sentiment Analysis On Twitter
Micro-blogging has become one of the most popular way to share one’s thought on particular topic
or issue and for most people it has become a daily routine. One of most popular micro-blogging
channel is twitter-a convenient as well as quick way to share short posts, article links, GIF’s, videos
and more. There are 330 million monthly active users and 145 million daily active users on Twitter.
Being such a popular platform sentiment analysis on its data is a satisfactory way to know the
opinion of the general public.
There is no doubt that twitter is booming platform nowadays. This social media platform allows a
maximum of 128 words post. Since the number of users in twitter is increasing, it is a perfect idea to
perform sentiment analysis or opinion mining on its data and know what people think around the
world.
Basically in our model we will classify tweets into three different categories- Negative, Positive,
Nuetral sentiments.We will divide our data into unigram, bigram and trigram. One thing that will
strike you is that what are these? This is explained further in the blog, keep reading.
Next main thing is that one needs data to work with. There are basically two ways user can get data-
1. Online Datasets Available
2. Twitter Search API
Various twitter datasets are available on Kaggle if one is going to use the data-set available already.
But if you want to build a more customizable model then one should use the Twitter API. For getting
access to Twitter API, you should apply for Twitter Developer Account. After getting confirmation
from Twitter you can use the API for fetching tweets. The Twitter Development Account is completely
free and allows to access certain number tweets from past 7 days, obviously unlimited paid access
can also be bought if required. You can run the API on a topic once in week(to avoid duplicate
tweets). Run the API weekly for 4-5 times to get sufficient amount of tweets.
Here the Twitter dataset is ready!
Data Pre - Processing
The tweets downloaded are in json format so it needs to be converted into csv format. Then all the
extra part like the geotags, '@#$%^&*' symbols must be removed from the tweets. This can be done
using Regular Expression(RE). Moreover the Twitter search API is a restless API so duplicate tweets
need to be removed. Afterwards all the stopwords like 'and,or,this,that,etc' must be removed
because this words do not contribute to significance in sentiment analysis of text.
After this we know computer won't understand text directly. All that a computer understands is 0's
and 1's. So it needs to be converted into numerical form. The process of converting text to numerical
form is called vectorisation of tweets. Vectorisation can be done by various
libraries.CountVectorizer(cv) or Term Frequency Inverse Document Frequency(Tf-idf) can be used to
convert text into numerical form.
Applying Machine Learning(ML) Algorithms
Now the vectorised tweets can split into training and testing data.90% of tweets can be used as
training data-set and 10% can be used as testing data-set.
Subsequently a pipeline can be created using various ML algorithm like Linear Regression, KNN,
Support Vector Machine(SVM) etc. Then all the evaluating factors must be calculated. The accuracy,
precision, recall, F1-Score and confusion matrix can be calculated to test the performance of our
model.
Labels: Machine Learning Sentiment AnalysisShare
Comments
Sentiment AnalysisMay 3, 2024 at 12:41 AM
Sentiment analysis, or opinion mining, delves into understanding the emotional tone of text—be it
positive, negative, or neutral—employing text analysis techniques. It proves invaluable across diverse
sectors, from business and literature surveys to governance and media. By analyzing sentiments on
platforms like Twitter, with its vast user base, one gains insights into public opinion.
Moreover, sentiment analysis using product review data enhances decision-making in the business
sphere, offering a comprehensive understanding of consumer feedback and preferences. This
methodological approach, dissected in our blog post, offers a powerful tool for comprehending
societal sentiment, enriching debates, and enhancing decision-making processes.
Blogs:
Sentiment Analysis
NLP-Natural Language Processing
