Module 6 & 7: Nonlinear Regression & Process Optimization
Nonlinear Regression: a form of regression analysis in which observational data are modeled by a
function which is a nonlinear combination of the model parameters and depends on one or more
independent variables.
Department of Chemical Engineering
Non-linear Regression
xi - Independent variables
ai - Parameters in the non-linear function
yi - Given y values (or dependent variables)
ycalc - Calculated y values
Department of Chemical Engineering
Nonlinear Regression: Motivating Example
Department of Chemical Engineering
Nonlinear Regression: Motivating Example
Estimate k1 and k2.
Department of Chemical Engineering
Nonlinear Regression in MATLAB
X = lsqcurvefit(fun,x0,xdata,ydata)
Input data: xdata,
Observed output: ydata
It starts at x0 and finds coefficients X to best fit the nonlinear function denoted by ‘fun’
(f(xdata) = ydata).
Department of Chemical Engineering
Nonlinear Regression in MATLAB
Example: Consider that the input xdata and observed response data ydata are as given
below.
xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3]
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5]
Find parameters s(1) and s(2) to fit a model of the form y = s(1)exp(s(2)x). Fit the model
using the starting point x0 = [100,-1].
xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3];
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5];
plot(xdata, ydata, ‘ko’)
Department of Chemical Engineering
Nonlinear Regression in MATLAB
Example: Consider that the input xdata and observed response data ydata are as given
below.
xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3]
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5]
Find parameters s(1) and s(2) to fit a model of the form y = s(1)exp(s(2)x). Fit the model
using the starting point x0 = [100,-1].
fun = @(s,xdata)s(1)*exp(s(2)*xdata);
s0 = [100,-1];
S = lsqcurvefit(fun,s0,xdata,ydata)
S = 498.8309 -0.1013
Department of Chemical Engineering
Introduction to Process Optimization
Department of Chemical Engineering
Components of Optimization Problem
Department of Chemical Engineering
Some Popular Optimization Problems
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB
Optimization solvers
[X, fval] = fminbnd (myfun, 𝐱𝐋𝐁 , 𝐱 𝐔𝐁 ) Bounded Min. problem
[X, fval] = fminsearch (myfun, 𝒙𝟎 ) Min. problem
[X, fval] = fminunc (myfun, 𝒙𝟎 ) Min. problem
myfun – function to be minimized in terms of x
𝐱 𝐋𝐁 - Lower bound on x
𝐱 𝐔𝐁 - Upper bound on x
𝒙𝟎 - Initial guess for x
X – Optimal values of x and fval – Corresponding objective function.
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB
Example: 𝐦𝐢𝐧 𝒇 𝒙
𝒙
2 2
=100(𝑥2 −𝑥1 ) + (1−𝑥1 )2
fun = @(x)100*(x(2) - x(1)^2)^2 + (1 - x(1))^2;
x0 = [-1.2,1];
[x,fval] = fminsearch(fun,x0)
x = 1.0000 1.0000 and fval = 8.1777e-10
[x,fval] = fminunc(fun,x0)
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB
[X, fval] = optimizer (@myfun, x0 )
Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5
𝑥
function f = myfun(x)
f = 12*x^5 - 45*x^4 + 40*x^3 + 5;
end
[X, fval] = fminsearch (@myfun, 1.5)
[X, fval] = fminunc (@myfun, 1.5)
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB
[X, fval] = optimizer (@myfun, xLB , xUB )
Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5
𝑥
such that x ∈ (0.5, 2.5)
Optimizer Decision
Variables
myfun
Objective
function
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB
[X, fval] = optimizer (@myfun, xLB , xUB )
Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5
𝑥
such that x ∈ (0.5, 2.5)
function f = myfun(x)
f = 12*x^5 - 45*x^4 + 40*x^3 + 5;
end
[X, fval] = fminbnd (@ myfun, 0.5, 2.5)
Department of Chemical Engineering
Constrained Function Minimization in MATLAB
Example
Department of Chemical Engineering
Constrained Function Minimization in MATLAB
[X,fval] = fmincon(@FUN,X0,A,B,Aeq,Beq,LB,UB,@NONLCON)
function f = FUN(x)
f = 9.82*x(1)*x(2)+2*x(1);
end
Department of Chemical Engineering
Constrained Function Minimization in MATLAB
function [c, ceq] = NONLCON(x)
% Nonlinear inequality constraints
c(1) = 2500/(pi*x(1)*x(2))-500;
c(2) = 2500/(pi*x(1)*x(2))-
(pi^2*(x(1)^2+x(2)^2))/0.5882;
% Nonlinear equality constraints
ceq = [];
end
Department of Chemical Engineering
Constrained Function Minimization in MATLAB
A = [-1 0; 1 0; 0 -1; 0 1];
B = [-2; 14; -0.2; 0.8];
Aeq = []; LB = [];
Beq = []; UB = [];
[X,fval] = fmincon(@FUN,[0.2,0.2],A,B,Aeq,Beq,LB,UB,@NONLCON)
Department of Chemical Engineering
Constrained Function Minimization in MATLAB
X = fmincon(@FUN,X0,A,B,Aeq,Beq,LB,UB,@NONLCON)
Try this
Min x
2
1
2
x 2 11 x1 x 2 7
2
2
x1, x 2
s.t.
x12 x 22 25
5 x1 , x 2 5
Department of Chemical Engineering
Function Minimization in MATLAB
Department of Chemical Engineering
Function Minimization in MATLAB
Optimization Problem Formulation
Use any initial guess for both k1 and k 2 . Plot concentration vs time with the optimal values of k1 and k 2 .
function f = optim_param(k,t,data)
for i = 1:10
a(i) = exp(-(k(1) + k(2)) * t(i));
b(i) = (k(1) / (k(1) + k(2))) * (1 - exp(-(k(1) + k(2)) * t(i)));
c(i) = (k(2) / (k(1) + k(2))) * (1 - exp(-(k(1) + k(2)) * t(i)));
fcn(i) = (data(i,1) - a(i))^2 + (data(i,2) - b(i))^2 + (data(i,3) - c(i))^2;
end
f = sum(fcn);
end
Department of Chemical Engineering
Function Minimization in MATLAB
clc
function f = clear all
optim_param(k,t,data) data = load('rxn_data.txt');
for i = 1:10 n = 10;
a(i) = exp(-(k(1) + k(2)) * t = 0.1:0.1:1;
t(i)); fun = @(k)optim_param(k,t,data);
b(i) = (k(1) / (k(1) + k(2))) * x0 = [0.2,0.1];
(1 - exp(-(k(1) + k(2)) * t(i))); [X,fval] = fminunc(fun,x0);
c(i) = (k(2) / (k(1) + k(2))) *
(1 - exp(-(k(1) + k(2)) * t(i))); % For plotting
fcn(i) = (data(i,1) - a(i))^2 +
(data(i,2) - b(i))^2 + (data(i,3) - for i = 1:10
c(i))^2; a(i) = exp(-(X(1) + X(2)) * t(i));
end b(i) = (X(1) / (X(1) + X(2))) * (1 - exp(-(X(1) + X(2)) * t(i)));
f = sum(fcn); c(i) = (X(2) / (X(1) + X(2))) * (1 - exp(-(X(1) + X(2)) * t(i)));
end end
plot(t,a,'-*',t,b,'-o',t,c,'-d')
Department of Chemical Engineering
Function Minimization in MATLAB
Department of Chemical Engineering
Data Visualization in MATLAB boxplot(MPG,Origin)
xlabel('Country of Origin’)
Box plot ylabel('Miles per Gallon (MPG)')
Create a box plot of the miles per
gallon (MPG) data in MATLAB -
carsmall.
load carsmall
boxplot(MPG)
xlabel('All Vehicles')
ylabel('Miles per Gallon (MPG)')
For grouped data - Create a box plot
of MPG from the sample data, grouped
by the vehicles' country of origin.
Data Visualization in MATLAB
Scatter Plot
Create 1000 normally
distributed random points
each for x and y and plot
them.
x = randn(1000,1);
y = randn(1000,1);
scatter(x,y,'filled')
plot(x,y,'bo')
Data Visualization in MATLAB
Line Plot
Create x as a vector of linearly
spaced values between 0 and
2π.
Use an increment of π/100
between the values.
y = sin (x). Create a line plot of
the x vs y data.
x = 0:pi/100:2*pi;
y = sin(x);
plot(x,y)
Data Visualization in MATLAB
Heatmap
Create a matrix of data.
Then create a heatmap of the
matrix values.
cdata = [45 60 32; 43 54 76; 32 94 68; 23 95 58];
heatmap(cdata)
Data Visualization in MATLAB
3-D Data Line Plot
Define t as a vector of
values between 0 and 10π.
Define st and ct as vectors of
sine and cosine values of t.
Then plot st, ct, and t.
t = 0:pi/50:10*pi;
st = sin(t);
ct = cos(t);
plot3(st,ct,t)
Data Visualization in MATLAB plot3(x,y,z,'bo')
3-D Data Scatter Plot
Use sphere of radius 16 to
define vectors x, y, and z.
Create a 3-D scatter plot of x, y
and z.
[X,Y,Z] = sphere(16);
x = [0.5*X(:); 0.75*X(:); X(:)];
y = [0.5*Y(:); 0.75*Y(:); Y(:)];
z = [0.5*Z(:); 0.75*Z(:); Z(:)];
scatter3(x,y,z)
Data Visualization in MATLAB
High Dimensional Data plot (> 3-D) Read a 4-D data from MATLAB.
Show the Pair-
wise plots of
the data
load fisheriris
plotmatrix(meas)
Data Visualization in MATLAB
High Dimensional Data plot (> 3-D)
Dimension Reduction Techniques – Reduce High Dimensional Data
to 2 or 3 dimensions such that the data can be visualized.
Commonly used –
i. Principal Component Analysis (PCA)
ii. t-Stochastic Distributed Neighbors (t-SNE)
Data Visualization in MATLAB
Principal Component Analysis (PCA) – High Dimensional Data Plot
Transforms a large set of variables into load fisheriris
a smaller one that still contains most of [coeff,score,~,~,estimated] = pca(meas);
the information in the large set. scatter(score(:,1),score(:,2),'filled')
coeff Principal component coefficients
score Principal component scores representations of the data
in the principal component space
estimated the percentage of the total variance explained by each
principal component.
Data Visualization in MATLAB
PCA – 4-D Data Plot
Transforms a large set of variables
into a smaller one that still contains
most of the information in the large
set.
load fisheriris
[coeff,score,~,~,estimated] = pca(meas);
scatter(score(:,1),score(:,2),'filled')
Data Visualization in MATLAB
t-Stochastic Distributed Neighbors (t-SNE) - High Dimensional Data Plot
Transforms a large set of variables into
a smaller one that still contains most of
the information in the large set.
load fisheriris
rng(8)
Y = tsne(meas);
scatter(Y(:,1),Y(:,2),'filled')
Y two-dimensional embedding of the
high-dimensional data in ‘meas’.
rng for ensuring reproducibility.
Data Visualization in MATLAB
t-Stochastic Distributed Neighbors (t-SNE) - High Dimensional Data Plot
Y = tsne(meas,'NumDimensions',3);
scatter3(Y(:,1),Y(:,2),Y(:,3),'filled')
Y three-dimensional embedding
of the high-dimensional data ‘meas’.
