Exercise: Sketching the Optimal Separating Hy-

perplane
We are given the following observations:

Obs. X1 X2 Y
1 3 4 Red
2 2 2 Red
3 4 4 Red
4 1 4 Red
5 2 1 Blue
6 4 3 Blue
7 1 3 Blue

Sketch the optimal separating hyperplane, and provide the equation for this
hyperplane (of the form β0 + β1 X1 + β2 X2 = 0).

Solution
Sketch of the Optimal Separating Hyperplane
[Include your sketch here]

Equation of the Optimal Separating Hyperplane

1 Calculations
1. **Calculate X T X**:
 
3 4
2 2
    
3 2 1 2 4 1
1   = 50
4 37
XT X =
4 2 4 1 3 2
3  1
 37 50
4 3
1 3
2. **Calculate (X T X)−1 **:
Using the inverse of a 2x2 matrix:
 −1
T −1 1 50 37
(X X) =
det(X T X) 37 50
First, let’s find the determinant of X T X:

det(X T X) = (50)(50) − (37)(37) = 2500 − 1369 = 1131

1
 So,
 −1
1 50 37
(X T X)−1 =
1131 37 50
To find the inverse of the 2x2 matrix, we can use the formula:
 −1  
1 50 37 1 50 −37
inverse = =
det(X T X) 37 50 1131 −37 50

3. **Calculate X T b**:
 
1 − β0
  1 − β0 
 

3 2 1 2 4 1  1 − β0 
XT b =


4 2 4 1 3 3 −1 − β0 

−1 − β0 
−1 − β0

 
3(1 − β0 ) + 2(1 − β0 ) + 1(1 − β0 ) + 2(−1 − β0 ) + 4(−1 − β0 ) + 1(−1 − β0 )
=
4(1 − β0 ) + 2(1 − β0 ) + 4(1 − β0 ) + 1(−1 − β0 ) + 3(−1 − β0 ) + 3(−1 − β0 )
 
−12 − 9β0
=
−12 − 9β0
4. **Compute β**:

β = (X T X)−1 X T b

50 −37
  
1131 1131 −12 − 9β0
= −37 50
1131 1131 −12 − 9β0
50 −37
 
1131 (−12 − 9β0 ) + 1131 (−12 − 9β0 )
= −37 50
1131 (−12 − 9β0 ) + 1131 (−12 − 9β0 )
 −762 612

1131 − 1131 β0
= −612 762
1131 − 1131 β0
 −762−612β0 
1131
= −612−762β 0
1131

5. **Solve for β0 **: Let’s use Observation 1 (3, 4):

β 0 = 1 − β1 × 3 − β2 × 4
Substituting the values of β1 and β2 :
   
−762 − 612β0 −612 − 762β0
β0 = 1 − ×3− ×4
1131 1131

2
Now, we can solve this equation to find β0 . Let’s simplify it:
−762 − 612β0 −612 − 762β0
β0 = 1 − −
377 282.75
Multiplying through by 1131 (the common denominator):

1131 1131
1131β0 = 1131 − (−762 − 612β0 ) × − (−612 − 762β0 ) ×
377 282.75

1131β0 = 1131 − (−2026.50 − 1617.75β0 ) − (−1965.38 − 2480.25β0 )

1131β0 = 1131 + 2026.50 + 1617.75β0 − 1965.38 − 2480.25β0

1131β0 + 1617.75β0 + 2480.25β0 = 1131 + 2026.50 − 1965.38

(1131 + 1617.75 + 2480.25)β0 = 1131 + 2026.50 − 1965.38

(5229.00)β0 = 192.12

192.12
β0 =
5229.00

β0 ≈ 0.03672