- An inner product (15.134) is a special function that induces a rich geometry on a vector space (15.1)-(15.2), including length (15.147), distance (15.154) and angle (15.163).
- The inner product leads to the notion of orthogonality (15.169), which in turn leads to orthgonal projection or best prediction (15.192).
In this section we review basic notions of geometry that hold for inner product spaces [W].
An inner product space is a vector space (15.1)-(15.2) with an associated inner product . An inner product is any function that takes as input any two elements in a vector space and outputs a real number, i.e.
To be an inner product, (15.134) must display the following properties for any vectors and any scalar :
A commonly used inner product on the vector space is the dot product [W], which is defined as
where recall vectors in are commonly denoted using the bold notation . We want to stress that the dot product (15.139) is one specific inner product on , but there are in fact infinitely many inner products which can be defined on , and more generally in any vector space, see Section 17.3.5 for an example of an inner product in . The only requirement for an inner product is that it needs to satisfy (15.135)-(15.136)-(15.137)-(15.138).
for , where is a symmetric (15.121), positive definite matrix (15.125) and is a full-rank (15.73) matrix such that , for more details see Section 15.6.2. In fact, on finite-dimensional inner product spaces, every inner product can be expressed in terms of a Mahalanobis inner product (15.141) on the vector coordinates, see Section 15.6 for details.
which has inverse
The Mahalanobis inner product of and with scaling matrix , (15.141) is calculated as
Note also that while we have restricted our attention to vector spaces over the real numbers for simplicity, the concepts in this section can be easily extended to vector spaces over arbitrary fields [W], for example the field of complex numbers. This means that the scalars are elements from the chosen field, rather than the real numbers.
In an inner product space , any linear operator (15.49)-(15.50) can be expressed uniquely using the inner product . First note that the symmetry (15.135) and linearity of the inner product (15.136)-(15.137) imply that for any fixed vector , the inner product is a linear function of , i.e.
for . This suggests how we may use the inner product to represent a linear operator that maps to .
for all .
The identification between inner products and linear operators which map to the real line (15.146) generalizes to the Riesz representation theorem (17.84), which is discussed in more detail in Section 17.3.3. This result lies at the foundation of linear pricing theory (0b.25), which is covered in depth in Chapters 0b-0c.
In our simple example of a real vector space with the dot product (15.139), this length corresponds to the standard Euclidean norm
Similarly, for (15.26) we obtain
for , where is a symmetric (15.121), positive definite matrix (15.125), is the standard Euclidean norm (15.148) and a full-rank matrix such that , for more details see Section 15.6.2. On finite-dimensional inner product spaces, any norm induced by an inner product (15.147) can be expressed as the Mahalanobis norm (15.151) of the vector coordinates, see Section 15.6 for details.
Similarly, for (15.26) we obtain
for . In the example of a real vector space with the dot product, this distance (15.154) corresponds to the standard Euclidean distance
for , where is the standard Euclidean norm (15.148), is a symmetric (15.121), positive definite matrix (15.125) and is a matrix such that , for more details see Section 15.6.2. Note that the Mahalanobis distance (15.157) reduces to the standard Euclidean distance (15.155) when is the identity matrix. The Mahalanobis distance (15.157) allows us to generalize the absolute z-score (22.41) in the multivariate framework of an -dimensional random variable to the multivariate absolute z-score (22.42), as discussed in Section 22.2.
In this context the Mahalanobis distance (15.157) takes into account the correlations between the random variables, as opposed to the standard Euclidean distance (15.155) which implicitly assumes they are uncorrelated.
Example 15.35. Polarization identity.
Continuing from Example 15.34, we can verify the polarization identity (15.159) for the dot product (15.139) using the pair of vectors (15.25)-(15.26). Indeed, from our previous calculations of the norms (15.149) and (15.150) and the distance (15.156)
which agrees with our previous calculation of the inner product (15.140).
where equality holds if and only if the vectors are linear dependent.
Example 15.36. Cauchy-Schwarz inequality.
Continuing from Example 15.35, we can verify the Cauchy-Schwarz inequality (15.161) for the dot product (15.139) using the pair of vectors (15.25)-(15.26). Indeed, from our previous calculations of the norms (15.149) and (15.150) and the inner product (15.140), we see that
as the Cauchy-Schwarz inequality (15.161) states.
This is well-defined since the Cauchy-Schwartz inequality (15.161) guarantees that .
Example 15.37. Angle.
Continuing from Example 15.36, we can calculate the standard Euclidean angle between the pair of vectors (15.25)-(15.26). Using the previously calculated inner product (15.140) and lengths (15.149) and (15.150), the angle between and is calculated as (15.163)
Care must be taken when determining the orthogonality of vectors using their coordinates, for details see Section 15.6.1.