Consider a perturbation vector of size defined by the standard Euclidean inner product (14.122)
Consider a perturbation vector of size defined by the inner product (17.82)
Example 17.42. Sample mean
In Example 44.27 we consider the sample mean (44.151). We demonstrate that it is in fact a functional (17.275) of the historical distribution with flexible probabilities (44.155).
The Gateaux derivative is the functional analysis generalization of the directional derivative.
In particular, we denote the Gateaux derivative in the direction of the Dirac delta (17.96) as follows
Example 17.43. Influence function of the sample mean
In Example 44.28 we compute the influence function (44.156) of the sample mean (44.151), which is the Gateaux derivative of the functional (44.155).
The Fréchet derivative is the functional analysis generalization of the gradient.
for all . Hence, we can interpret the gradient (17.282) as a map from the vector space into itself
for all vectors , where is the standard Euclidean dot product (14.122)
Note that this is in the form of the Riesz representation (14.159), where the directional derivative is linear in for fixed .
The Fréchet derivative is the unique linear functional that recovers the Gateaux derivative (17.280) of the function in a generic direction
where is the inner product (17.82)
This is analogous to the fact that the gradient is the unique vector that recovers the directional derivative (17.285) in the finite dimensional case. Notice that the Riesz representation theorem (17.100) then tells us that is the Riesz representation of , viewed as a linear functional of .
Unlike in the finite dimensional case, there may be occurrences where the Gateaux derivatives (17.281) are well defined and yet the Fréchet derivative (17.292) is not well-defined [W], though such occurrences are beyond the scope of the present discussion.
When both derivatives are defined, there is a simple connection between the two: by considering in the inner product representation of the Fréchet derivative (17.289), and using the sifting property (17.95) of the Dirac delta we obtain that the Fréchet derivative (17.289) at any point is the Gateaux derivative in the direction of the Dirac delta (17.281) at that point
for all points , similar to how the entry of the gradient is the partial derivative (17.283) in the finite dimensional case.
Similar to (17.287), for a given functional , the Fréchet derivative provides us with the first order Taylor expansion of the functional
where is the norm induced by the inner product (17.290).
The second order Fréchet derivative is the functional analysis generalization of the Hessian.
In multivariate calculus, it is convenient to arrange the second-order derivatives in matrix form in the Hessian (15.50). We can interpret the Hessian as a map from the vector space into a matrix, which means a linear application from the vector space in itself
The left hand side in (17.295) are the terms of the first order Taylor expansion (17.292). The calculation of the inner product on the right hand side follows from the linear action (17.294) and the dot product (14.122)
In the context of functional analysis, the second order Fréchet derivative [ W] is a map from the function space (17.83) into a kernel, which means a linear operator from the function space into itself
The left hand side in (17.299) are the terms of the first order Taylor expansion (17.292). The calculation of the inner product on the right hand side follows from the linear action (17.298) and the inner product (17.82)