Note
Click here to download the full example code
Warm-up: numpy¶
Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024
A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.
This implementation uses numpy to manually compute the forward pass, loss, and backward pass.
A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.
99 967.9031466580327
199 685.5070882237483
299 486.3500547708435
399 345.8661544348595
499 246.74996163437044
599 176.80690768257315
699 127.44163338262639
799 92.59419697507451
899 67.99119944857532
999 50.61843376754606
1099 38.34941784876846
1199 29.683658370787292
1299 23.562181913825583
1399 19.23749063123848
1499 16.18186370074691
1599 14.022683259863054
1699 12.49681088830256
1799 11.418396726609682
1899 10.6561621441435
1999 10.117365170276921
Result: y = -0.037920750619778204 + 0.8527630571462719 x + 0.006541960485925736 x^2 + -0.0927645836312748 x^3
import numpy as np
import math
# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)
# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
# y = a + b x + c x^2 + d x^3
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = np.square(y_pred - y).sum()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')
Total running time of the script: ( 0 minutes 0.236 seconds)
