CHAPTER 2: REGRESSION
1. CHECKING LINEARITY : [pg.no:21-22]
PROGRAM :
from pandas import DataFrame
import matplotlib.pyplot as plt
Stock_Market = {'Year': [2017, 2017, 2017, 2017, 2017, 2017,
2017, 2017, 2017, 2017, 2017, 2017, 2016, 2016, 2016, 2016,
2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016],
'Month': [12, 11, 10, 9, 8, 7, 6, 5, 4, 3,
2, 1, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1],
'Interest_Rate': [2.75, 2.5, 2.5, 2.5, 2.5,
2.5, 2.5, 2.25, 2.25, 2.25, 2, 2, 2, 1.75, 1.75, 1.75, 1.75,
1.75, 1.75, 1.75, 1.75, 1.75, 1.75, 1.75],
'Unemployment_Rate': [5.3, 5.3, 5.3, 5.3,
5.4, 5.6, None, 5.5, None, 5.6, 5.7, 5.9, 6, 5.9, 5.8, 6.1,
6.2, 6.1, 6.1, 6.1, 6.1, 5.9, 6.2, 6.2],
'Stock_Index_Price': [1464, 1394, 1357,
1293, 1256, 1254, 1234, 1195, 1159, 1167, 1130, 1075, 1047,
965, 943, 958, 971, 949, 884, 866, 876, 822, 704, 719]}
df = DataFrame(Stock_Market, columns=['Year', 'Month',
'Interest_Rate', 'Unemployment_Rate', 'Stock_Index_Price'])
plt.scatter(df['Interest_Rate'], df['Stock_Index_Price'],
color='red')
plt.title('Stock Index Price Vs Interest Rate', fontsize=14)
plt.xlabel('Interest Rate', fontsize=14)
plt.ylabel('Stock Index Price', fontsize=14)
plt.grid(True)
plt.show()
plt.scatter(df['Unemployment_Rate'],
df['Stock_Index_Price'], color='green')
plt.title('Stock Index Price Vs Unemployment Rate',
fontsize=14)
plt.xlabel('Unemployment Rate', fontsize=14)
plt.ylabel('Stock Index Price', fontsize=14)
plt.grid(True)
plt.show()
OUTPUT:
2. SIMPLE LINEAR REGRESSION:[pg.no:22-23]
PROGRAM :
from pandas import DataFrame
from sklearn import linear_model
Stock_Market = {'Year': [2017, 2017, 2017, 2017, 2017, 2017,
2017, 2017, 2017, 2017, 2017, 2017, 2016, 2016, 2016, 2016,
2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016],
'Month': [12, 11, 10, 9, 8, 7, 6, 5, 4, 3,
2, 1, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1],
'Interest_Rate': [2.75, 2.5, 2.5, 2.5, 2.5,
2.5, 2.5, 2.25, 2.25, 2.25, 2, 2, 2, 1.75, 1.75, 1.75, 1.75,
1.75, 1.75, 1.75, 1.75, 1.75, 1.75, 1.75],
'Unemployment_Rate': [5.3, 5.3, 5.3, 5.3,
5.4, 5.6, None, 5.5, None, 5.6, 5.7, 5.9, 6, 5.9, 5.8, 6.1,
6.2, 6.1, 6.1, 6.1, 6.1, 5.9, 6.2, 6.2],
'Stock_Index_Price': [1464, 1394, 1357,
1293, 1256, 1254, 1234, 1195, 1159, 1167, 1130, 1075, 1047,
965, 943, 958, 971, 949, 884, 866, 876, 822, 704, 719]}
df = DataFrame(Stock_Market, columns=['Year', 'Month',
'Interest_Rate', 'Unemployment_Rate', 'Stock_Index_Price'])
# Here we have 1 variable for linear regression
X = df[['Interest_Rate']]
Y = df['Stock_Index_Price']
# Model fitting with sklearn linear regression
regr = linear_model.LinearRegression()
regr.fit(X, Y)
# Displaying Intercept and coefficients
print('Intercept:\n', regr.intercept_)
print('\nCoefficients:\n', regr.coef_)
# Prediction with sklearn
new_interest_rate = 2.75
print('Predicted Stock Index Price:\n',
regr.predict([[new_interest_rate]]))
OUTPUT:
INTERPRETATION:
Simple linear regression is of the form y=w0 + wlx. The output shows wo (Intercept)
as —99 . 4 6431881371655 and W1 (Coefficient) as 564 . 2038924 9. According to the
above example, the equation becomes
Stock_Index_Price= wo+W1* Interest_Rate
i.e, Stock_Index_Price= -99.46431881371655 +564.20389249* Interest Rate
Stock_Index_Price = 1452 . 0 9 63 8 554 which is exactly the predicted stock index price.
3. READING FROM A CSV FILE AND PREDICTING A SET OF DEPENDENT VARIABLES :[pg.no:24-25]
PROGRAM:
import pandas as pd
from pandas import DataFrame
from sklearn import linear_model
# Reading the input data from a csv file
df = pd.read_csv("stock.csv")
# Here we have 1 variable for linear regression
X = df[['Interest_Rate']]
Y = df['Stock_Index_Price']
# Model fitting with sklearn linear regression
regr = linear_model.LinearRegression()
regr.fit(X, Y)
# Displaying Intercept and coefficients
print('Intercept:\n', regr.intercept_)
print('Coefficients:\n', regr.coef_)
# Prediction with sklearn for all the interest rates
new_interest_rate = df[['Interest_Rate']]
df1 = DataFrame(regr.predict(new_interest_rate))
print('Predicted Stock Index Price:\n', df1)
Output:
4. MULTIPLE LINEAR REGRESSION :[pg.no:25-27]
PROGRAM:
from pandas import DataFrame
from sklearn import linear_model
import statsmodels.api as sm
Stock_Market = {'Year': [2017, 2017, 2017, 2017, 2017, 2017,
2017, 2017, 2017, 2017, 2017, 2017, 2016, 2016, 2016, 2016,
2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016],
'Month': [12, 11, 10, 9, 8, 7, 6, 5, 4, 3,
2, 1, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1],
'Interest_Rate': [2.75, 2.5, 2.5, 2.5, 2.5,
2.5, 2.5, 2.25, 2.25, 2.25, 2, 2, 2, 1.75, 1.75, 1.75, 1.75,
1.75, 1.75, 1.75, 1.75, 1.75, 1.75, 1.75],
'Unemployment_Rate': [5.3, 5.3, 5.3, 5.3,
5.4, 5.6, 5.5, 5.5, 5.5, 5.6, 5.7, 5.9, 6, 5.9, 5.8, 6.1,
6.2, 6.1, 6.1, 6.1, 6.1, 5.9, 6.2, 6.2],
'Stock_Index_Price': [1464, 1394, 1357,
1293, 1256, 1254, 1234, 1195, 1159, 1167, 1130, 1075, 1047,
965, 943, 958, 971, 949, 884, 866, 876, 822, 704, 719]}
df = DataFrame(Stock_Market, columns=['Year', 'Month',
'Interest_Rate', 'Unemployment_Rate', 'Stock_Index_Price'])
# Here we have 2 variables for multiple regression.
X = df[['Interest_Rate', 'Unemployment_Rate']]
Y = df['Stock_Index_Price']
# Model fitting with sklearn linear regression
regr = linear_model.LinearRegression()
regr.fit(X, Y)
# Displaying Intercept and coefficients
print('Intercept:\n', regr.intercept_)
print('Coefficients:\n', regr.coef_)
# Prediction with sklearn
new_interest_rate = 2.75
new_unemployment_rate = 5.3
print('Stock Index Price:')
print(regr.predict([[new_interest_rate,new_unemployment_rate]]))
# Prediction with statsmodels
X = sm.add_constant(X) # adding a constant
model = sm.OLS(Y, X).fit()
predictions = model.predict(X)
print(model.summary())
Output:
INTREPRETATION OF RESULT:
This output includes the intercept and coeffcients. We can use this information to
build the multiple linear regression equation as follows. Stock_Index_Price = (Intercept) +
(Interest_Rate coef)*X1 +(Unemployment_Rate coef)*X2 Substituting the values of
intercept and coeffcients we get Stock_Index_Price= (1798.4040) +(3455401)*X1+ (-250.1466)*X2
Let. Interest_Rate = 2.75 (i.e., X 1= 2.75) and Unemployment_Rate = 5.3
(i.e., X2= 5.3). Substituting the above data into the regression equation, we will get the exact same
predicted results as displayed. = (1798.4040) + (3455401)*(2.75)+(-250.1466)*(5.3) = 1422.86
The table OLS Regression results displays a comprehensive table with statistical info
generated by statsmodels. Following are some important information from the OLS Regression
Results table.
Adj. R-squared reflects the fit of the model.
R-squared values range from 0 to 1, where a higher value generally indicates a better fit,
assuming certain conditions are met.
const coeffcient is our Y-intercept. It means that if Interest Rate coeffcient is zero, then the
expected output (i.e., the Y) would be equal to the const coeffcient.
Interest Rate coefficient represents the change in the output Y due to a change of one unit in
the interest rate (everything else held constant).
Unemployment Rate coefficient represents the change in the output Y due to a change of one
unit in the interest rate (everything else held constant).
std err reflects the level of accuracy of the coeffcients. The lower it is, the higher is the level of
accuracy.
P >ltl is your p-value. A p-value of less than 0.05 is considered to be statistically
significant. Confidence interval represents the range in which our coefficients are likely to fall
(with a likelihood of95%).
Notice that the coeffcients captured in this table (highlighted) match with the coeffcients generated
by sklearn. We got consistent results by applying both sklearn and statsmodels.
5. LINEAR REGRESSION:[pg.no:29-30]
PROGRAM :
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Importing the dataset
dataset = pd.read_csv('position_salaries.csv')
X = dataset.iloc[:, 1:2].values
y = dataset.iloc[:, 2].values
# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=0)
# Fitting Linear Regression to the dataset
from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)
# Visualizing the Linear Regression results
plt.scatter(X_train, y_train, color='red')
plt.plot(X_train, regressor.predict(X_train), color='blue')
plt.title('Linear Regression')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()
Output:
Explanation:
In this example, we have used 4 libraries namely numpy, pandas, matplotlib and
sklearn. We have imported libraries and got the dataset first. The dataset is a table
which contains all values in our csv file. X, the 2nd column which contains Years of
Experience array and y the last column which contains Salary array. We have split
our dataset to get training set and testing set (both X and y values per each set).
Test_size=0.2:We have split our dataset (10 observations) into 2 parts (training
set, test set) and the ratio of test set compare to dataset is 0.2 (2 observations will
be put into the test set. We can put it 1/7 to get 20% or 0.2, they are the same. We
should not let the test set too big. If it's too big, we will be lacking data to train. Normally, we should
pick around 5% to 30%.
Train_size : If we use the test size already, the rest of data will
automatically be assigned to train_size.
Random_state : This is the seed for the random number generator. We can put
an instance of the RandomState class as well. If we leave it blank or 0, the
RandomState instance used by np.random will be used instead. We have
already the train set, test set, and built the linear regression model. Now, will build
a polynomial regression model and visualize it.
6. POLYNOMINAL REGRESSION :[pg.no:30-31]
PROGRAM :
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Importing the dataset
dataset = pd.read_csv('position_salaries.csv')
X = dataset.iloc[:, 1:2].values
y = dataset.iloc[:, 2].values
# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2, random_state=0)
# Fitting polynomial regression to the dataset
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
poly_reg = PolynomialFeatures(degree=4)
X_poly = poly_reg.fit_transform(X)
lin_reg = LinearRegression()
lin_reg.fit(X_poly, y)
# Visualizing the Polynomial Regression results
def viz_polynomial():
plt.scatter(X, y, color='red')
plt.plot(X, lin_reg.predict(poly_reg.fit_transform(X)),
color='blue')
plt.title('Polynomial Regression')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()
return
viz_polynomial()
OUTPUT:
7. LOGISTIC REGRESSION:[pg.no:32-33]
PROGRAM FOR CONFUSION MATRIX:
import pandas as pd
import seaborn as sn
import matplotlib.pyplot as plt
data = {'y_Predicted': [1, 1, 0, 1,0,1,1,0,1,0,0,0],
'y_Actual': [1, 0, 0, 1, 0, 1, 0, 0, 1,0,1,0]
df = pd.DataFrame(data, columns=['y_Actual', 'y_Predicted'])
# Creating confusion matrix
confusion_matrix = pd.crosstab(df['y_Actual'], df['y_Predicted'], rownames=['Actual'],
colnames=['Predicted'],margins=True)
# Generating heatmap and displaying it
ax = sn.heatmap(confusion_matrix, annot=True)
plt.show()
# Getting the statistics of the confusion matrix
print(confusion_matrix)
Output:
8. PROGRAM:[pg.no:33-35]
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn import metrics
import seaborn as sn
import matplotlib.pyplot as plt
candidates = {
'gmat': [780, 750, 690, 710, 680, 730, 690, 720, 740,
690, 610, 690, 710, 680, 770, 610, 580, 650, 540,
590, 620, 600, 550, 550,570, 670, 660, 580, 650,
660, 640, 620, 660, 660, 680, 650, 670, 580, 590, 690],
'gpa': [4,3.9, 3.3, 3.7, 3.9, 3.7, 2.3, 3.3, 3.3,
1.7, 2.7, 3.7, 3.7,3.3, 3.3, 3, 2.7, 3.7, 2.7, 2.3,
3.3, 2,2.3, 2.7, 3, 3.3, 3.7, 2.3, 3.7,
3.3, 3, 2.7, 4, 3.3, 3.3, 2.3, 2.7, 3.3, 1.7,
3.7],
'work experience': [3, 4, 3, 5, 4, 6, 1, 4, 5, 1,3, 5, 6,
4, 3, 1, 4, 6, 2, 3, 2, 1, 4, 1, 2, 6, 4, 2, 6, 5, 1, 2, 4,
6, 5, 1,2, 1, 4, 5],
'admitted': [1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1,
1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1,
1, 0, 0, 0, 0, 1]
df = pd.DataFrame(candidates, columns=['gmat', 'gpa',
'work experience', 'admitted'])
X = df[['gmat', 'gpa', 'work experience']]
y = df['admitted']
# Splitting the dataset into training and testing
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.25, random_state=0)
# Fitting logistic regression to the dataset
logistic_regression = LogisticRegression()
logistic_regression.fit(X_train, y_train)
y_pred = logistic_regression.predict(X_test)
# Creating confusion matrix
confusion_matrix = pd.crosstab(y_test, y_pred,
rownames=['Actual'], colnames=['Predicted'], margins=True)
# Generating heatmap and displaying it
ax = sn.heatmap(confusion_matrix, annot=True)
plt.show()
print(confusion_matrix)
# Displaying accuracy
print('Accuracy:', metrics.accuracy_score(y_test,y_pred))
Output:
9. PROGRAM:[pg.no:37-38]
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn import metrics
import seaborn as sn
import matplotlib.pyplot as plt
candidates = {
'gmat': [780, 750, 690, 710, 680, 730, 690, 720, 740,
690, 610, 690, 710, 680, 770, 610, 580, 650, 540, 590, 620,
600, 550, 550, 570, 670, 660, 580, 650, 660, 640, 620, 660,
660, 680, 650, 670, 580, 590, 690],
'gpa': [4,3.9, 3.3, 3.7, 3.9, 3.7, 2.3, 3.3, 3.3,
1.7, 2.7, 3.7, 3.7,3.3, 3.3, 3, 2.7, 3.7, 2.7, 2.3,
3.3, 2,2.3, 2.7, 3, 3.3, 3.7, 2.3, 3.7,
3.3, 3, 2.7, 4, 3.3, 3.3, 2.3, 2.7, 3.3, 1.7,
3.7],
'work experience': [3, 4, 3, 5, 4, 6, 1, 4, 5, 1,3, 5, 6,
4, 3, 1, 4, 6, 2, 3, 2, 1, 4, 1, 2, 6, 4, 2, 6, 5, 1, 2, 4,
6, 5, 1,2, 1, 4, 5],
'admitted': [1,1,1,1,1,1,0,1,1,0,0,1,1,1,1,0,0,1,
0,0,0,0,0,0,0,1,1,0,1,1,0,0,1,1,1,0,0,0,0,1]
df = pd.DataFrame(candidates, columns=['gmat', 'gpa',
'work experience', 'admitted'])
X = df[['gmat', 'gpa', 'work experience']]
y = df['admitted']
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.25, random_state=0)
logistic_regression = LogisticRegression()
logistic_regression.fit(X_train, y_train)
new_candidates = {
'gmat': [590, 740, 680, 610, 710],
'gpa': [2,3.7, 3.3, 2.3, 3],
'work experience': [3, 4, 6, 1, 5]
df2 = pd.DataFrame(new_candidates, columns=['gmat', 'gpa',
'work experience'])
y_pred = logistic_regression.predict(df2)
print(df2)
print(y_pred)
Output:
