- An inner product (28.133) is a special function that induces a rich geometry on a vector space (28.1)-(28.2), including length (28.148), distance (28.153) and angle (28.161).
- The inner product leads to the notion of orthogonality (28.167), which in turn leads to orthgonal projection or best prediction (28.190).
In this section we review basic notions of geometry that hold for inner product spaces [W].
To be an inner product, (28.133) must display the following properties for any vectors and any scalar :
A commonly used inner product on the vector space is the dot product [W], which is defined as
where recall vectors in are commonly denoted using the bold notation .
We can generalize the dot product to the Mahalanobis inner product by introducing a full-rank (28.75) scaling matrix so that
for . In fact, on finite-dimensional inner product spaces, every inner product can be expressed in terms of a Mahalanobis inner product (28.140) on the vector coordinates, see Section 28.6 for details.
Using the formula for the determinant of a diagonal matrix (28.109), we see that
which has inverse
Note also that while we have restricted our attention to vector spaces over the real numbers for simplicity, the concepts in this section can be easily extended to vector spaces over arbitrary fields [W], for example the field of complex numbers. This means that the scalars are elements from the chosen field, rather than the real numbers.
In an inner product space , any linear operator (28.51)-(28.52) can be expressed uniquely using the inner product . First note that the symmetry (28.134) and linearity of the inner product (28.135)-(28.136) imply that for any fixed vector , the inner product is a linear function of , i.e.
for . This suggests how we may use the inner product to represent a linear operator that maps to .
for all .
The identification between inner products and linear operators which map to the real line (28.147) generalizes to the Riesz representation theorem (30.59), which is discussed in more detail in Section 30.3.3. This result lies at the foundation of linear pricing theory (21a.26), which is covered in depth in Chapter 21.
In our simple example of a real vector space with the dot product (28.138), this length corresponds to the standard Euclidean norm
Similarly, for (28.28) we obtain
for , where is the standard Euclidean norm (28.149) and the full-rank scaling matrix. On finite-dimensional inner product spaces, any norm induced by an inner product (28.148) can be expressed as the Mahalanobis norm (28.152) of the vector coordinates, see Section 28.6 for details.
Similarly, for (28.28) we obtain
for . In the example of a real vector space with the dot product, this distance (28.153) corresponds to the standard Euclidean distance
for , where is the standard Euclidean norm (28.149) and the full-rank scaling matrix. Note that the Mahalanobis distance (28.156) reduces to the standard Euclidean distance (28.154) when is the identity matrix. The Mahalanobis distance (28.156) allows us to generalize the absolute z-score (36.41) in the multivariate framework of an -dimensional random variable to the multivariate absolute z-score (36.42), as discussed in Section 36.2.
Example 28.34. Polarization identity.
Continuing from Example 28.32, we can verify the polarization identity (28.157) for the pair of vectors (28.27)-(28.28). Indeed, from our previous calculations of the norms (28.150) and (28.151) and the distance (28.155)
which agrees with our previous calculation of the inner product (28.139).
where equality holds if and only if the vectors are linear dependent.
Example 28.35. Cauchy-Schwarz inequality.
Continuing from Example 28.34, we can verify the Cauchy-Schwarz inequality (28.159) for the pair of vectors (28.27)-(28.28). Indeed, from our previous calculations of the norms (28.150) and (28.151) and the inner product (28.139), we see that
as the Cauchy-Schwarz inequality (28.159) states.
This is well-defined since the Cauchy-Schwartz inequality (28.159) guarantees that .
Example 28.36. Angle.
Continuing from Example 28.35, we can calculate the standard Euclidean angle between the pair of vectors (28.27)-(28.28). Using the previously calculated inner product (28.139) and lengths (28.150) and (28.151), the angle between and is calculated as (28.161)
Care must be taken when determining the orthogonality of vectors using their coordinates, for details see Section 28.6.1.