Optimization theory

Notation 5.1   In the following section, the function $f$ is defined as:
$f\colon X\subseteq \mathbb{R}^p \rightarrow Y\subseteq \mathbb{R}$. $X$ is called the decisions space, and Y is called the criteria space.

Problem 5.2 (Unconstrained optimization)   Given the function $f$ that depends on one or more variable $x\in X$, the problem of optimize $f$, in this context, is equal to find:

$$\min_{\mathbf{x}\in X} f(\mathbf{x})$$

this is also known as an unconstrained optimization, since there are not any constraints on the values the function $f$ may assumes.

The unconstrained optimization is seldom applied in the field of digital circuits, so the constrained optimization is defined as:

Problem 5.3 (Constrained optimization)   Find

$$\min_{x\in X} f(\mathbf{x}) \qquad\textrm{subject to}\quad g_j(\mathbf{x}) \leq h_j, \mathstrut j=1, 2, \dotsc, m$$

where the $n$ equations $g_i(\mathbf{x}) \leq h_i$ constitute the set of constraints of the optimization.

The function $f$ is also called the objective of the optimization, or the cost function of the problem.

The above problems are classical optimization problems, or mono-objective problems. The multi-objective unconstrained optimization is defined as the problem to optimize a vectorial function, so that the objective-function is a vector of objective-functions.

Notation 5.4   In the following (multi-objective optimization), the function $f$ is defined as:
$f\colon X\subseteq \mathbb{R}^p \rightarrow Y\subseteq \mathbb{R}^n$, or $f=(\mathstrut f_1,f_2,\dots, n) \vert f_i\colon X\subseteq \mathbb{R}^p \rightarrow Y\subseteq \mathbb{R}$,

Problem 5.5 (Unconstrained multi-objective optimization)   Find

$$\min_{\mathbf{x}\in X} f_i(\mathbf{x}), \mathstrut i=1, 2, \dotsc, n$$

where there are $n$ objective functions.

Finally, the multi-objective constrained optimization is defined as:

Problem 5.6 (Constrained multi-objective optimization)   Find

$$\min_{\mathbf{x}\in X} f_i(\mathbf{x}), \mathstrut i=1, 2, \dotsc... ...\textrm{subject to}\quad g_i(\mathbf{x}) \leq h_i, \mathstrut i=1, 2, \dotsc, m$$

where there are $n$ objective functions and $m$ constraints.

The multi-objective optimization is a very complex problem, since the problem of finding the minimum of two or more functions is apparently only trivial: the set of independent variables $\mathbf{x}_{min}$ that minimizes, let's say, the function $f_1$, it is not supposed to minimizes (and generally it does not) the other functions. So there should be a way to combine the information of minimum among all the functions. The intuitive way of linear combination is somewhat problematic:

$$f_{tot}(\mathbf{x}) = \sum_{i=1}^{n} \alpha_i f_i(\mathbf{x}), \ \alpha_i\in\mathbb{R}$$

because the functions $f_i$ cannot be commensurable among them. For example, if there is one function $f_j$ that is $f_j >> f_i, \forall i\neq j$, then this function dominate the total objective, giving false results for the optimization problem. This problem is illustrated in §4.1.2.