The conjugate direction method

Let $\mathbf{u}, \mathbf{v} \in X\subseteq\mathbb{R}^n$. They are said mutually orthogonal if $\mathbf{u}^T\mathbf{v}=0$. Similarly they are said mutually conjugate with respect to a matrix $\mathbf{A}$ if $\mathbf{u}^T\mathbf{Av}=0$.

Property 5.32   A set of of mutually conjugate vectors in $X\subseteq\mathbb{R}^n$ constitutes a basis for $X$.

The importance of a set of mutually conjugate vectors is stated from the following theorem:

Theorem 5.33   Every descent method of optimization using mutually conjugate directions is quadratically convergent.

The concept of conjugate directions is important, since, in an intuitively manner, a minimization attained along one of this directions does not perturb the the minimization along the other direction.