Late Summer 2024

Systems and Computer Engineering

Carleton University

Copyright © 2022 - 2024, Department of Systems and Computer Engineering

Last edited: July 13th, 2024
§ Practical Programming, 3rd ed.
§ Chapter 10, Reading and Writing Files
§ What Kinds of Files are There? (pp. 173 - 174)
§ Opening a File (pp. 175 - 178)
§ Techniques for Reading Files (pp. 179 - 183)
§ Processing Whitespace-Delimited Data (pp. 192 -
195)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 2

Copyright (c) Systems and Computer Engineering Carleton University
§ Reading data from text files
§ Using a case study, introduce iterative, incremental
software development

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 3

Copyright (c) Systems and Computer Engineering Carleton University
§ Know the meaning of these words and phrases
§ Iterative development
§ Incremental development
§ Text file
§ open and close (built-in functions)
§ split and strip methods (type str)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 4

Copyright (c) Systems and Computer Engineering Carleton University
§ Be able to use apply iterative, incremental
development when working on a small-scale software
project
§ Be able to write code that processes data read from
a text file

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 5

Copyright (c) Systems and Computer Engineering Carleton University
§ Throughout the semester, we will work on building a
text-based Pokémon game
§ Here’s how we will tackle the implementation of the
game:
§ Set Up the Environment § User Interface
§ Choose a Pokémon Dataset § User Input
§ Import Required Modules § Game Loop
§ Load Pokémon Data § Win/Lose Conditions
§ Create Game Mechanics § Testing and Refinement
§ Implement Game Logic § Game Completion
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 6
Copyright (c) Systems and Computer Engineering Carleton University
§ Use iterative, incremental development
§ Iterative development
§ Repeatedly cycle through these steps: identify the
requirements for the next iteration, design and
implement the code that achieves those
requirements, test and debug the code, reflect on
progress so far
§ Rationale: it's easier to find bugs if we construct a
program in a step-wise manner

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 7

Copyright (c) Systems and Computer Engineering Carleton University
§ Incremental development
§ Each iteration adds one or more features to the
code developed in previous iterations
§ Rationale: it's easier to determine if we're taking the
wrong approach (incomplete or incorrect
requirements, weak design) if we "grow" a program
through a series of prototypes

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 8

Copyright (c) Systems and Computer Engineering Carleton University
§ Prepare some text files that we'll use to demonstrate
the application
§ pokemon.csv
§ For initial testing, use a text editor prepare a smaller,
simpler text file

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 9

Copyright (c) Systems and Computer Engineering Carleton University
§ 5 rows (to check if the program can read multiple
lines)
§ Different types of Pokémon (to check if the program
sorts the details)
§ Some rows with empty cells (to check if the program
always checks for the same number of columns)
§ Has different data types (Strings, integers, floats)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 10

Copyright (c) Systems and Computer Engineering Carleton University
§ Read a text file, one line at a time
§ Read the first line from the csv file as the head of the
table
§ Split the current line into words/values, convert each
word to lowercase
§ Display the rows

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 11

Copyright (c) Systems and Computer Engineering Carleton University
§ The solution is small enough that it can be
implemented as a single Python module
§ Most of the text processing steps will be handled by
one function
§ If the function becomes too complex, split it into
simpler functions
§ Use Python's container types as much as possible
§ We'll need a short script to call the function

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 12

Copyright (c) Systems and Computer Engineering Carleton University
§ Use the Function Design Recipe that you learned in
ECOR 1041 to define the header and docstring of the
function that processes text files
§ The function will take the name of a file (a string)
and return a sorted list of Pokémon details (list of
dictionaries)
§ Minimal implementation of the function body: just
return an empty list

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 13

Copyright (c) Systems and Computer Engineering Carleton University
def load_pokemon(file_name : str) -> list[dict]:

"""Return a list of dictionaries containing unique pokemon data read from

file_name
Preconditions: file_name has the following columns
['name', 'attack', 'defense', 'hp', 'pokedex_number', 'speed', 'type1',
'type2', 'generation’]

>>> load_pokemon("pokemon_test.csv")
[{'name': 'abomasnow', 'attack': 132, 'defense': 105, 'hp': 90,
'pokedex_number': 460, 'speed': 30, 'type1': 'grass', 'type2': 'ice',
'generation': 4},
{Next item},

...]
"""
return []

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 14

Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to demonstrate that we can read
and print the text from the specified text file, one line
at a time
§ Before we can read information from a file, we need
to open it
In_file = open(file_name, "r")
§ Function open opens the file in mode "r" (read-from-
file) and returns an object that "knows" how to access
the file

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 15

Copyright (c) Systems and Computer Engineering Carleton University
§ Open the file and iterate through the lines
§ With text files, you always have three phases
1. Open the file
2. [Repeatedly] read the line
3. Close the file

Files must reside in the same folder as your

application (as a default, for now)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 16

Copyright (c) Systems and Computer Engineering Carleton University
§ To open a file, the function open() is used
§ Syntax:
§ To open the file for reading, you can just specify the
file’s name
§ in_file = open("input_file.txt")
§ in_file = open("input_file.csv")
§ The function returns a file object
§ In the example above, the object will be bound to in_file

Note: A .csv (comma separated values) file is

a text file with specific formatting for the data
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 17
Copyright (c) Systems and Computer Engineering Carleton University
§ When opening a file, you need to specify the access mode
§ There are several access modes:
§ “r” – open the file as read-only; the file must exist
§ “w” – create a new file for writing; if the file already exists
it will be overwritten and treated as a new blank file
§ “a” – append data to file; a file is created if it doesn’t
exist
§ “x” – create a specific file, returns an error if the file exists
§ In addition you can specify if the file should be handled as
binary or text mode
§ “t” – Text - Default value. Text mode
§ “b” – Binary - Binary mode (e.g. images)
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 18
Copyright (c) Systems and Computer Engineering Carleton University
§ Practical Programming explains several techniques for
reading files
§ The "for line in file" technique is a good choice when
we want to do the same thing with every line in a file
§ This for loop reads the file, one line at a time:
§ line is assigned a string containing the line

for line in in_file:

# process line

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 19

Copyright (c) Systems and Computer Engineering Carleton University
§ The closure of a file is performed using close() function
§ Syntax:
in_file.close()
§ If closing a file is omitted from the code, it will act in a
similar manner to if it was included
§ This is because the program opens the file, reads the
data, but doesn’t close it explicitly, and then probably
implicitly closes the file when it closes
§ Therefore it can be seen as a “nicety”, rather than a
“necessity” – omitting this step will become a problem
as a program grows

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 20

Copyright (c) Systems and Computer Engineering Carleton University
§ This is because if we don’t close the file, a program still
has exclusive access over it until the program closes
§ Thus, the file is locked to that application and cannot
be accessed by another application
§ Implicit closing can also take time, and thus your new
version of your program might not have access to the
file for several seconds as well
§ Note: this is actually the same for any system
resource (memory, network, etc)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 21

Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon
def load_pokemon(file_name : str) -> list[dict]:
"""Return a list of dictionaries containing unique pokemon data read from
file_name
Preconditions: file_name has the following columns
['name', 'attack', 'defense', 'hp', 'pokedex_number', 'speed', 'type1',
'type2', 'generation']
"""
in_file = open(file_name, 'r')
for line in in_file:
print(line)
in_file.close()
We normally close a file
after we've finished
return []
reading it
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 22
Copyright (c) Systems and Computer Engineering Carleton University
>>> load_pokemon("pokemon_test.csv")
name,attack,defense,hp,pokedex_number,speed,type1,type2,gene
ration

Abomasnow,132,105,90,00460,30,grass,ice,4

Blissey,10,10,255,00242,55,normal,,2

Cloyster,95,180,50,00091,70,water,ice,1
…
§ Output is double-spaced because each line read from the file has
a newline character at the end, and print also outputs a newline
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 23
Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to call method split to split each line into a list
of words (strings)
§ By default, split removes leading and trailing whitespace (spaces,
tabs, newlines), but doesn't remove punctuation marks
>>> line = ' Hello, world! '
>>> line.split() returns the list
['Hello,', 'world!’]

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 24

Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to loop over the list of words
§ In the loop, call method strip on each word to remove leading
and trailing punctuation
§ Example:
>>> word = 'Hello,'
>>> word.strip(string.punctuation) returns the string 'Hello'

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 25

Copyright (c) Systems and Computer Engineering Carleton University
§ Clean up the words, removing any punctuation and
converting all to lower case
§ Within a line
§ For each word:
1. Remove
2. Convert to lower case

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 26

Copyright (c) Systems and Computer Engineering Carleton University
def load_pokemon(file_name : str) -> list[dict]:
""" Refer to previous slides
"""
in_file = open(file_name, 'r')
for line in in_file:
line = line.strip()
line = line.lower()
line = line.split(',')

# All three statements can be combined into one:

# line = line.strip().lower().split(',’)
in_file.close()

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 27

Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to separate table header read from the csv file
from the data it self
§ An easy way to do this is to modify the for loop and add a
condition to check if it was the first line read or not
§ If it is the first line read, print out “header”
§ If it is any subsequent line, print out “row:”

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 28

Copyright (c) Systems and Computer Engineering Carleton University
def load_pokemon(file_name : str) -> list[dict]:

""" Refer to previous slides

"""
in_file = open(file_name, 'r’)
first_line = True # Flag to check if it was the first line read or not
for line in in_file:
line = line.strip().lower().split(',’)
if first_line:
first_line = False
table_header = line
print("header:", table_header)
else:
print("Row :", line)
in_file.close()
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 29
Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to store the data (rows) into a
dictionary
§ A dictionary is a Key:Value pair.
§ Each row in the CSV file has the same number of
items.
§ We can use the elements from the table_header list
to identify the keys of the dictionary
§ We can modify the if/else control statements to add
the items to the dictionary

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 30

Copyright (c) Systems and Computer Engineering Carleton University
if first_line:
# print(line) #to show columns
first_line = False
table_header = line
else:
pokemon = {}
pokemon[table_header[0]] = line[0]
pokemon[table_header[1]] = line[1]
pokemon[table_header[2]] = line[2]
pokemon[table_header[3]] = line[3]
pokemon[table_header[4]] = line[4]
pokemon[table_header[5]] = line[5]
pokemon[table_header[6]] = line[6]
pokemon[table_header[7]] = line[7]
pokemon[table_header[8]] = line[8]
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 31
Copyright (c) Systems and Computer Engineering Carleton University
if first_line:
# print(line) #to show columns
first_line = False
table_header = line
else:
pokemon = {}
pokemon[table_header[0]] = line[0]
pokemon[table_header[1]] = int(line[1])
pokemon[table_header[2]] = int(line[2])
pokemon[table_header[3]] = int(line[3])
pokemon[table_header[4]] = int(line[4])
pokemon[table_header[5]] = int(line[5])
pokemon[table_header[6]] = line[6]
pokemon[table_header[7]] = line[7]
pokemon[table_header[8]] = int(line[8])
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 32
Copyright (c) Systems and Computer Engineering Carleton University
§ Edit load_pokemon to store the dictionaries into the
resulting list
§ Return the resulting list

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 33

Copyright (c) Systems and Computer Engineering Carleton University
§ First we need to create an empty list outside the for loop
pokemons_list = [] #Create empty list
§ Then we can append the dictionary to the list. We have
two options:
§ Using append function
pokemons_list.append(pokemon)
§ Using concatenation operator “+”
pokemons_list += [pokemon]
§ Notice the [ ], this will make sure that the dictionary is
added as a list element
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 34
Copyright (c) Systems and Computer Engineering Carleton University
§ The function add_stat_total should add the health
of each Pokémon to it’s dictionary
ℎ𝑒𝑎𝑙𝑡ℎ = 𝑎𝑡𝑡𝑎𝑐𝑘 + 𝑑𝑒𝑓𝑒𝑛𝑠𝑒 + 𝑠𝑝𝑒𝑒𝑑 + ℎ𝑝
§ Adding an item to the dictionary is done by adding a
new key and if it doesn’t exists it will create a
dictionary item
§ The function shouldn’t return anything
§ Remember that when you pass a function as an input
parameter to a function, you are passing an alias to that
function
Dr. Rami Sabouni ECOR1042 - Late Summer 2024 35
Copyright (c) Systems and Computer Engineering Carleton University
def add_stat_total(pokemons : list[dict]) -> None:
""" Refer to provided files for details
"""

for i in range(0, len(pokemons)):

# Summing the pokemon stat
stat_total = pokemons[i]['attack'] +
pokemons[i]['defense'] + pokemons[i]['speed'] +
pokemons[i]['hp']

# Updating the pokemon dictionary

pokemons[i].update({'stat_total': stat_total})
# Alternative method:
# pokemons[i]['stat_total'] = stat_total

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 36

Copyright (c) Systems and Computer Engineering Carleton University
§ It looks like the first part of our game (load Pokémon data) is finished
§ Lets add a short script that calls the function
if __name__ == "__main__":
file_name = 'pokemon_test.csv'
# Load the values from the csv file We'll discuss this when
pokemons = load_pokemon(file_name) we look at modules

# print each dictionary in a new line

for pokemon in pokemons:
print(pokemon)

# Add the stats column to each dictionary

add_stat_total(pokemons)

# print each dictionary in a new line

for pokemon in pokemons:
print(pokemon)

Dr. Rami Sabouni ECOR1042 - Late Summer 2024 37

Copyright (c) Systems and Computer Engineering Carleton University
38

Dr. Rami Sabouni ECOR1042 - Late Summer 2024