Assignment 2 – Homework exercise

Write your solution in Overleaf and save as a pdf file. Submit your pdf file.
Write your code in only a notebook Google Colab with attached your result and submit
your Goolge Colab link.
Name of your pdf file: MSSV_Fullname_assg2.pdf
Name of your Colab file: MSSV_Fullname_assg2.ipynb
Any copy the source code of other students will be banned.
Deadline: 22h 23/8/2021
You should follow as below:

Part 1: In Mathematics for Machine learning Books:

Solve on Overleaf and implement on Google Colab
2.4b,d
2.5b
2.11
3.3
4.2
4.4
4.7d
4.9
Part 2:
The core library for numerical computation in Python is called NumPy. NumPy provides useful abstractions
to create and manipulate multidimensional arrays (e.g., vectors, matrices, tensors), and functions that are thin
layers over high-performance C/C++/Fortran BLAS and LAPACK libraries. Conventionally you would start
your numeric Python code with an import numpy as np. The most basic object is the NumPy array, which is a
multi-dimensional array usually made up of type double, i.e., 64-bit floating point numbers. A vector is usually
a one-dimensional array and a matrix is a two-dimensional array:
e x amp l e _ v e c t o r = np . a rr a y ( [ 5 0 . 0 , 1 . 4 , 0 . 4 , 0 . 2 3 , 1 . 0 4 ] )
e x amp l e _ma tri x = np . a rr a y ( [ [ 3 . 4 , 1 . 1 0 , 0 . 8 ] ,
[ 4 . 2 , 1 0 0 . 0 , −1 . 4 ] ,
[1.1,0.44,9.4]])
Higher-dimensional arrays (tensors) are possible, but come up less often than vectors and matrices. The tuple
made up of the size each dimension is called the shape and can be accessed as an attribute, so:
p r i n t ( e x amp l e _ v e c t o r . shape , e x amp l e _ma tri x . shape )
# Output : ( 5 ,) ( 3 , 3 )
Create a Colab notebook and do the following:
(A) Use arange to create a variable named foo that stores an array of numbers from 0 to 29, inclusive. Print foo
and its shape.
(B) Use the reshape function to change foo to a validly-shaped two-dimensional matrix and store it in a
new variable called bar. Print bar and its shape.
(C) Create a third variable, baz that reshapes it into a valid three-dimensional shape. Print baz and its shape.
(D) There are several different ways to index into NumPy arrays. Use two-dimensional array indexing to set
the first value in the second row of bar to -1. Now look at foo and baz. Did they change? Explain what’s going
on. (Hint: does reshape return a view or a copy?)
(E) Another thing that comes up a lot with array shapes is thinking about how to aggregate over specific
dimensions. Figure out how the NumPy sum function works (and the axis argument in particular) and
do the following:
(i) Sum baz over its second dimension and print the result.
(ii) Sum baz over its third dimension and print the result.
(iii) Sum baz over both its first and third dimensions and print the result.
(F) Along with shaping and indexing, we also do a lot of slicing which is where you index with ranges to get
subvectors and sometimes submatrices. Write code to do the following:
(i) Slice out the second row of bar and print it.
(ii) Slice out the last column of bar using the -1 notation and print it.
(iii) Slice out the top right 2 × 2 submatrix of bar and print it
Part 3:
One feature of NumPy that is powerful but tricky is the ability to perform broadcasting, which really just refers
to repeatedly performing an operation over one or more dimensions.
(A) The most basic kind of broadcast is with a scalar, in which you can perform a binary operation (e.g., add,
multiply, ...) on an array and a scalar, the effect is to perform that operation with the scalar for every element
of the array. To try this out, create a vector 1, 2, . . . , 10 by adding 1 to the result of the arange function.
(B) Now, create a 10 × 10 matrix A in which Aij = i + j. You’ll be able to do this using the vector you just
created, and adding it to a reshaped version of itself.
(C) A very common use of broadcasting is to standardize data, i.e., to make it have zero mean and unit
variance.
First, create a fake “data set” with 50 examples, each with five dimensions.
import numpy.random as npr
d a t a = np . exp ( npr . r andn ( 5 0 , 5 ) )
You don’t worry too much about what this code is doing at this stage of the course, but for completeness: it
imports the NumPy random number generation library, then generates a 50 × 5 matrix of standard normal
random variates and exponentiates them. The effect of this is to have a pretend data set of 50 independent
and identically-distributed vectors from a log-normal distribution.
(D) Now, compute the mean and standard deviation of each column. This should result in two vectors of length
5. You’ll need to think a little bit about how to use the axis argument to mean and std. Store these
vectors into variables and print both of them.
(E) Now standardize the data matrix by 1) subtracting the mean off of each column, and 2) dividing each
column by its standard deviation. Do this via broadcasting, and store the result in a matrix called
normalized. To verify that you successfully did it, compute the mean and standard deviation of
the columns of normalized and print them out
Part 4:
Fairly often in machine learning and other disciplines that use linear algebra to formalize computational
problems, you see explicit matrix inverses like A -1b in papers and books. Of course, you learn in linear algebra
classes how to perform such an inverse, and in the NumPy linear algebra library you’ll see a function that will
compute the inverse. It seems sensible right? And it’s right there on the page: Invert the matrix A and then
multiply by the vector b. How hard can it be?
Well... don’t do that. When you see A -1 b you should read that as “the x that is the result of solving the linear
system Ax = b”. You should use something like the solve function in the NumPy linear algebra package. If you
have to solve multiple such systems with the same matrix, you might think about performing an LU
decomposition.
The issue is one of numerical stability, which boils down to the fact that 1) floating point numbers on
computers don’t have infinite precision, and 2) some quantities (e.g., infinite sums or numerical integrals)
require us to truncate and approximate if we want the computation to complete. These issues are deep and this
course won’t delve into them significantly. However, the “explicit inverse” issue is worth looking at just to see
an example.
(A) A Vandermonde matrix is a matrix generated from a vector in which each column of the matrix is an
integer power starting from zero. So, if I have a column vector [x1, x2, . . . , xN ]^T , then the associated
(square) Vandermonde matrix would be

Use what you learned about broadcasting in the previous problem to write a function that will produce a
Vandermonde matrix for a vector [1, 2, . . . , N]^T for any N. Do it without using a loop. Here’s a stub to get
you started:

def vandermonde (N) :

vec = np . a r a n g e (N) +1
# F i n is h me .
Use your function for N = 12, store it in variable named vander, and print the result.
(B) Now, let’s make a pretend linear system problem with this matrix. Create a vector of all ones, of length 12
and call it x. Perform a matrix-vector multiplication of vander with the vector you just created and store
that in a new vector and call it b. Print the vector b.
(C) First, solve the linear system the naïve way, pretending like you don’t know x. Import numpy.linalg,
invert V and multiply it by b. Print out your result. What should you get for your answer? If the answer is
different than what you expected, write a sentence about that difference.
(D) Now, solve the same linear system using solve. Print out the result. Does it seem more or less in line
with what you’d expect
Part 5:
One important Pythonic skill is the ability to load, manipulate, and store external data in files. There are
several different ways to do this, but one of the most powerful and convenient tools is the pickle library.
(Pickling being a way to safely prepare food for long-term storage.) The first step towards pickling data in
Python is to understand how to open files. Tutorials and documentation abound online, but for pickling, the
most straightforward thing to do is to use the with open pattern. For reading, this typically looks like:
i m p o rt p i c k l e a s p kl
wit h open ( ’ i n _ f i l e . p kl ’ , ’ r b ’ ) a s f h :
 u n p i c k l e d _ d a t a = p kl . l o a d ( f h )
What’s happening here is that we’re opening a file (that here conventionally has the extension .pkl) in “read
binary” mode as indicated by the ‘rb’. The variable fh is a handle to the opened file that here we just hand to
the pickle library to process. Writing pickled data is very similar:
wit h open ( ’ o u t _ f i l e . p kl ’ , ’wb ’ ) a s f h :
p kl . dump ( d a t a _ t o _ p i c k l e , f h )
We’re opening a file in “write binary” mode and then handing the dump function a Python variable and the
filehandle we just opened. The pickle library knows how to handle many different kinds of Python data
structures; one powerful thing to do is to give it a dict with a bunch of different things you’d like to save.
(A) Use the attached coords.pkl file I sent. Create a Colab notebook. Upload the coords.pkl file and write code
to open and unpickle it. After you load it, you’ll have a NumPy array that is a 296 × 2 matrix. As the name
indicates, these are coordinates in 2d. The first column are x and the second column are y. Plot these data using
Matplotlib. This will look best if it is large with equal aspect ratio; the code below might be helpful for that.
fi g , ax = p l t . s u b p l o t s ( f i g s i z e = ( 1 5 , 1 5 ) )
#...plotting...
ax . s e t _ a s p e c t ( 1 )
(B) In graphics and computer vision, affine transformations are a powerful way to deform objects and images.
We’ll look at a simple version of this here, in which we’ll use 2 × 2 matrix to transform the two-dimensional
coordinates you loaded above. For example, these linear maps rotate, scale, and shear:

Use subplot to make three subplots, rendering the coordinate data after applying each of these three
transformations for parameter values of your choosing. Make sure to set the subtitles appropriately.
(C) Do these transformation commute? Create a rotation matrix and a shear matrix for some non-trivial
parameters. Create a figure with two subplots, in which you compose these transformations: one with
rotate-then-shear and another that is shear-then-rotate. Make sure to set the subtitles appropriately.