keyboard_arrow_down Introducing Pandas Objects
import numpy as np
import pandas as pd
keyboard_arrow_down The Pandas Series Object
A Pandas Series is a one-dimensional array of indexed data. It can be created from a list or array as follows:
data = pd.Series([5,14,99,888])
data
0 5
1 14
2 99
3 888
dtype: int64
data[3]
888
data.values
array([ 5, 14, 99, 888])
data.index
RangeIndex(start=0, stop=4, step=1)
Like with a NumPy array, data can be accessed by the associated index via the familiar Python square-bracket notation:
data[1]
14
data[1:3]
1 14
2 99
dtype: int64
keyboard_arrow_down Series as generalized NumPy array
Index is the difference between numpy and pandas
data = pd.Series([0.25, 0.5, 0.75, 1.0],
index=['a', 'b', 'c','d'])
data
a 0.25
b 0.50
c 0.75
d 1.00
dtype: float64
And the item access works as expected:
data['b']
0.5
keyboard_arrow_down Series as specialized dictionary
The Series -as-dictionary analogy can be made even more clear by constructing a Series object directly from a Python dictionary:
student_dict = {'Ram': 123,
'Shyam': 124,
'Arun': 125}
students = pd.Series(student_dict)
students
Ram 123
Shyam 124
Arun 125
dtype: int64
By default, a Series will be created where the index is drawn from the sorted keys. From here, typical dictionary-style item access can be
performed:
#To access rollno of Ram
students['Ram']
123
keyboard_arrow_down The Pandas DataFrame Object
The next fundamental structure in Pandas is the DataFrame . Like the Series object discussed in the previous section, the DataFrame can be
thought of either as a generalization of a NumPy array, or as a specialization of a Python dictionary. We'll now take a look at each of these
perspectives.
# initialize list of lists
data = [['tom', 10], ['nick', 15], ['juli', 14]]
# Create the pandas DataFrame
df = pd.DataFrame(data, columns=['Name', 'Age'])
# print dataframe.
df
Name Age
0 tom 10
1 nick 15
2 juli 14
df.index
RangeIndex(start=0, stop=3, step=1)
Additionally, the DataFrame has a columns attribute, which is an Index object holding the column labels:
df.columns
Index(['Name', 'Age'], dtype='object')
Thus the DataFrame can be thought of as a generalization of a two-dimensional NumPy array, where both the rows and columns have a
generalized index for accessing the data.
df['Name']
0 tom
1 nick
2 juli
Name: Name, dtype: object
keyboard_arrow_down Constructing DataFrame objects
A Pandas DataFrame can be constructed in a variety of ways. Here we'll give several examples.
keyboard_arrow_down From a single Series object
A DataFrame is a collection of Series objects, and a single-column DataFrame can be constructed from a single Series :
pd.DataFrame(students, columns=['rollno'])
rollno
Ram 123
Shyam 124
Arun 125
keyboard_arrow_down From a list of dicts
Any list of dictionaries can be made into a DataFrame .
Even if some keys in the dictionary are missing, Pandas will fill them in with NaN (i.e., "not a number") values:
pd.DataFrame([{'a': 1, 'b': 2}, {'b': 3, 'c': 4}])
a b c
0 1.0 2 NaN
1 NaN 3 4.0
keyboard_arrow_down From a two-dimensional NumPy array
Given a two-dimensional array of data, we can create a DataFrame with any specified column and index names. If omitted, an integer index will
be used for each:
np.random.rand(3, 2)
array([[0.48925761, 0.81202557],
[0.37526746, 0.9834642 ],
[0.10226165, 0.37402615]])
pd.DataFrame(np.random.rand(3, 2),
columns=['foo', 'bar'],
index=['a', 'b', 'c'])
foo bar
a 0.965321 0.512423
b 0.969355 0.437354
c 0.196705 0.719428
keyboard_arrow_down From a NumPy structured array
We covered structured arrays in Structured Data: NumPy's Structured Arrays. A Pandas DataFrame operates much like a structured array, and
can be created directly from one:
A = np.zeros(3, dtype=[('A', 'i8'), ('B', 'f8')])
A
array([(0, 0.), (0, 0.), (0, 0.)], dtype=[('A', '<i8'), ('B', '<f8')])
pd.DataFrame(A)
A B
0 0 0.0
1 0 0.0
2 0 0.0
keyboard_arrow_down Index as ordered set
Pandas objects are designed to facilitate operations such as joins across datasets, which depend on many aspects of set arithmetic. The
Index object follows many of the conventions used by Python's built-in set data structure, so that unions, intersections, differences, and other
combinations can be computed in a familiar way:
indA = pd.Index([1, 3, 5, 7, 9])
indB = pd.Index([2, 3, 5, 7, 11])
indA & indB # intersection- common elements
<ipython-input-71-b0dd807d5915>:1: FutureWarning: Index.__and__ operating as a set operation is deprecated, in the future this will be a
indA & indB # intersection- common elements
Int64Index([3, 5, 7], dtype='int64')
indA | indB # union - all elements
Index([3, 3, 5, 7, 11], dtype='int64')
indA ^ indB # symmetric difference
Index([3, 0, 0, 0, 2], dtype='int64')
DATA INDEXING AND SELECTION
import pandas as pd
data = pd.Series([0.25, 0.5, 0.75, 1.0],
index=['a', 'b', 'c', 'd'])
data
a 0.25
b 0.50
c 0.75
d 1.00
dtype: float64
# masking
data[(data ==0.5) ]
b 0.5
dtype: float64
# fancy indexing
data[['a', 'd']]
a 0.25
d 1.00
dtype: float64
Indexers: loc, iloc, and ix
data = pd.Series(['a', 'b', 'c'], index=[1, 3, 5])
data
1 a
3 b
5 c
dtype: object
# explicit index when indexing - user defined index
data[1]
'a'
# implicit index when slicing
data[1:3]
3 b
5 c
dtype: object
Because of this potential confusion in the case of integer indexes, Pandas provides some special indexer attributes that explicitly expose
certain indexing schemes.
First, the loc attribute allows indexing and slicing that always references the explicit index:
data.loc[1] #Local means explicit
'a'
data.loc[1:3]
1 a
3 b
dtype: object
The iloc attribute allows indexing and slicing that always references the implicit Python-style index:
data.iloc[1:3] #Implicit
3 b
5 c
dtype: object
student= [['Ram', 123,80,85], ['Shyam', 124,70,75],
['Arun', 125,35,60], ['Gopal', 235,95,70]]
data = pd.DataFrame(student,columns=['Name','Rollno',"FDS_Mark","DS_Mark"])
data
Name Rollno FDS_Mark DS_Mark
0 Ram 123 80 85
1 Shyam 124 70 75
2 Arun 125 35 60
3 Gopal 235 95 70
#Select all roll numbers
data['Rollno']
0 123
1 124
2 125
3 235
Name: Rollno, dtype: int64
data['name_dept'] = data['Name'] + "_CSE A"
data
Name Rollno FDS_Mark DS_Mark name_dept
0 Ram 123 80 85 Ram_CSE A
1 Shyam 124 70 75 Shyam_CSE A
2 Arun 125 35 60 Arun_CSE A
3 Gopal 235 95 70 Gopal_CSE A
#Select first two rows
data[:2]
Name Rollno FDS_Mark DS_Mark name_dept
0 Ram 123 80 85 Ram_CSE A
1 Shyam 124 70 75 Shyam_CSE A
#Operating on Pandas Data
#Dividing mark column by 100
data['FDS_Mark']/100
FDS_Mark
0 0.80
1 0.70
2 0.35
3 0.95
dtype: float64
data['DS_Mark']-15
DS_Mark
0 70
1 60
2 45
3 55
dtype: int64
data['Total_mark']=data['FDS_Mark']+data['DS_Mark']
data
Name Rollno FDS_Mark DS_Mark name_dept Total_mark
0 Ram 123 80 85 Ram_CSE A 165
1 Shyam 124 70 75 Shyam_CSE A 145
2 Arun 125 35 60 Arun_CSE A 95
3 Gopal 235 95 70 Gopal_CSE A 165
data['Total_mark'].mean()
142.5
data['Total_mark'].median()
155.0
data['Total_mark'].mode()
0 165
dtype: int64
#Handling Missing Data
isnull(): Generate a boolean mask indicating missing values
notnull(): Opposite of isnull()
dropna(): Return a filtered version of the data
fillna(): Return a copy of the data with missing values filled or imputed
import numpy as np
data = pd.Series([1, np.nan, 'hello', None])
data
0
0 1
1 NaN
2 hello
3 None
dtype: object
data.isnull()
0 False
1 True
2 False
3 True
dtype: bool
data.dropna() # Inplace changes original copy
0 1
2 hello
dtype: object
data
0 1
1 NaN
2 hello
3 None
dtype: object
#Filling Null Values
data.fillna(0)
0 1
1 0
2 hello
3 0
dtype: object
# forward-fill
data.fillna(method='ffill')
0 1
1 1
2 hello
3 hello
dtype: object
Start coding or generate with AI.
