Course #
Course Name
Lecture - 3
Instructor: First and Last names
Reference: Book Chapters 3 and 4
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB:
for Engineering Students and Professionals
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 2
for Engineering Students and Professionals
In practice, doing something as well as possible within practical
constraints is very satisfactory.
Optimization provides us with the means to make things
happen/work in the best possible practical way.
Brute-force or manual optimization is done by trial and error, and
using past experience, which for most practical problems:
Will lead to highly sub-optimal solutions, since only few trials can be performed
in a limited time, and/or
Is too time consuming to be practically feasible.
This is where quantitative optimization comes in:
Uses mathematical strategies to provide an efficient and systematic way to
optimize.
Using the capabilities of modern day computing, these mathematical strategies
become all the more powerful in implementation.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 3
for Engineering Students and Professionals
Box A shows two inputs – 1. the dream design and 2. the initial design.
Box B shows the analysis phase.
Box D is where the design is improved in a very systematic way.
Box E shows manual optimization by a human being (trial & error/intuitive)
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 4
for Engineering Students and Professionals
Box A shows two inputs – 1. the dream design and 2. the initial design.
Box B shows the analysis phase.
Box D is where the design is improved in a very systematic way.
Box E shows manual optimization by a human being (trial & error/intuitive)
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 5
for Engineering Students and Professionals
Might take forever Not an organized
without reaching approach
desired performance
Traditional Non-Opt:
Do trial and error.
Point A Point B
(Initial bad design) (Great optimal design)
Optimization
Yield optimal designs in
a reasonable timeframe A systematic approach
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 6
for Engineering Students and Professionals
Undergraduates: To acquire the ability to optimize designs
yourself, and feel comfortable and confident with the results
As a student, in the classroom setting (e.g., Capstone Design project);
As an engineer (post graduation), in the industry setting.
Graduate Students: To be able to use optimization to find better
ways to proceed with your experiments, system modeling, or
designs, in the course of your ongoing and future research.
Industry Personnel: To acquire the ability to leverage the
immense potential of optimization in different real-life projects.
Learn software tools to readily apply optimization in your projects;
Acquire knowledge to be able to critically verify the optimal designs
Acquire knowledge to be able to identify the challenges in optimizing a
system, and to know where to look for the solutions.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 7
for Engineering Students and Professionals
Analysis and Optimization are two core components of a
systems design process.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 8
for Engineering Students and Professionals
Engineering Analysis can be defined as:
The application of scientific principles and processes to reveal the
properties and the state of a system, and also understand the
underlying physics driving the system behavior.
Analysis generally demands disciplinary knowledge pertinent to
the system or mechanism being analyzed.
Practical systems involve multiple disciplines, e.g., designing an
aircraft requires structural, aerodynamic, and control analyses.
If disciplinary understanding has reached certain level of
maturity, mathematical analysis tools might be readily available.
On the other hand, in the case of mechanisms or phenomena
that are not yet well understood, in-depth and fundamental
analysis might be required thereby demanding the involvement
of a disciplinary expert.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 9
for Engineering Students and Professionals
Analysis need not be a purely mathematical or theoretical process,
especially in designing new systems.
Analysis could involve “experiments–testing–mathematical inference”
as an iterative process.
Such an approach is necessary due to any of the following reasons:
1. The underlying physics is not well understood; the fundamental
disciplinary principles or theory do not directly apply;
2. There are geometrical complexities and inherent uncertainties; or
3. There is a lack of knowledge of the material properties (e.g.,
thermodynamic or structural properties).
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 10
for Engineering Students and Professionals
Design in general terms can be defined as the creation of a plan
and/or strategy for constructing a physical system or process.
Based on the “object of design”, engineering design could be
classified into:
1. product design
2. systems design
3. industrial design, and
4. process design
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 11
for Engineering Students and Professionals
Practical engineering design can be perceived
as a multi-stage process.
Overlap or iteration among the different stages
is often prevalent and necessary.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 12
for Engineering Students and Professionals
Mathematical optimization is the process of maximizing and/or
minimizing one or more objectives without violating specified
design constraints, by regulating a set of variable parameters
that influence the objectives and the design constraints.
The three types of quantities in optimization:
1. Objectives: The quantities that you would like to improve – e.g., fuel
efficiency of an aircraft (to be maximized), or manufacturing cost of
the aircraft (to be minimized).
2. Constraints: The quantities or criteria that your design needs to
satisfy – e.g., the cargo/payload capacity of an aircraft.
3. Variables: The quantities that you can directly change to improve
the design, where the values of the objectives and constraints are
regulated by these quantities – e.g., the dimensions and the
material of the aircraft wing.
Objective functions and constraint functions are often together
called criteria functions, since one can be converted into another.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 13
for Engineering Students and Professionals
There does not exist a unique structure to how they are related − the
structure generally depends on the available human, computational,
and physical resources and on the choices of decision-makers.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 14
for Engineering Students and Professionals
Design in general is the enveloping process that includes analysis
and optimization as sub-processes.
Primary steps in design include (i) defining the design objectives,
constraints, and variables, (ii) performing or using analysis, and
(iii) performing optimization, (iv) verifying the optimum design.
Analysis provides you with the knowledge to relate the variable
values to the criteria functions of interest (through models).
Optimization, which is the main driver for improving the design,
uses the knowledge (model) obtained through analyses.
Analysis also provides the opportunity to investigate the
performance of the final optimum design.
On the other hand, optimization could provide food for further
analysis – e.g., which region of the design space is to be analyzed
in more detail.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 15
for Engineering Students and Professionals
Engineering design could involve qualitative elements that go
beyond quantitative “analysis” and “optimization”, while often
impacting these two quantitative activities.
Creativity- and aesthetics-driven decisions: Although qualitative
in nature, these decisions have important quantitative
implications for the later stages of design where analysis and
optimization are involved (e.g., impact on material options).
Market-driven decisions: Design specification can also be driven
by an understanding of the market, especially in the case
industrial and product design. Although, quantitative market
analysis might be available in certain cases, such availability is
not necessarily generic (e.g., imagine the first Iphone).
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 16
for Engineering Students and Professionals
In the context of design, analysis and optimization are related to
each other through modeling.
Modeling: A process by which an engineer or a scientist
translates the actual physical system/phenomena under study
into a set of mathematical equations or operations.
Mathematically, modeling can be represented as: 𝑷𝑷 = 𝒇𝒇 𝑿𝑿 ,
where, P the quantity of interest, which is expressed as a
function of a vector of design variables, X.
Mathematical models may not be a single analytical function.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 17
for Engineering Students and Professionals
Physics-based Analytical Models: These are theoretical models derived
from the physics of the system (e.g., analytical solution of the
conduction heat transfer differential equation).
Generally fast in execution.
Often inadequate for complex systems (e.g., involving complex geometry).
Simulation-based Models: These models generally leverage a
discretized representation of the system, in translating the system
behavior into a set of algebraic equations. These set of equations are
solved using numerical techniques, by harnessing the power of
computers. Examples include CFD.
Often time-intensive (could take hours/weeks/months).
Generally adequate in representing system complexities.
Surrogate Models: Surrogate models are purely mathematical or
statistical models, with certain generic functional forms and
coefficients that can be tuned using a set of input-output data that is
generated from physical or simulation-based experiments.
Fast in execution.
Can represent complex systems, generally at the cost of accuracy or fidelity.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 18
for Engineering Students and Professionals
The success of optimization depends both on:
the capabilities of the optimization method/algorithm;
the effectiveness of the optimization formulation.
Modeling the optimization or problem formulation essentially
involves developing a clear definition of the design variables,
design objectives, and design constraints.
Problem formulation is also strongly correlated with the choice
of optimization algorithms – e.g.
During problem formulation, One can convert equality constraints into
inequality constraints using a tolerance value, in order to leverage powerful
algorithms that perform well in the absence of equality constraints.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 19
for Engineering Students and Professionals
It is important to ensure that optimization problem formulation
is coherent with the system behavior model.
For example, the input output exchange (between analysis and
optimization) demanded by the optimization formulation should
be satisfied by the capabilities of the models used thereof.
The choice of analysis models also affects the choice of
optimization algorithm, and vice versa. For example:
If you choose an algorithm that generally requires a large number of system
evaluations, then a fast model of the system behavior is needed.
If the system behavior model is inherently highly nonlinear, you will need to
formulate the optimization problem such that a nonlinear optimization
algorithm can be used to solve the problem.
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 20
for Engineering Students and Professionals
What’s Next?
Lecture 4 - Introduction to Quantitative Optimization
Lecture 5 - Multiobjective Optimization
Lecture 6 - Numerical Essentials
Copyright ©2014 by A. Messac Optimization in Practice with MATLAB: 21
for Engineering Students and Professionals
