Notation

We spend a tremendous amount of time on notation and conventions, to make them both intuitive for readers from different disciplines and consistent across a wide set of topics. In the short Section 1 here below we point out a few key tenets that appear throughout all the present work. For the full list of notations, keep reading further below.

Key notation tenets

Random/non-random

Lower case for all non-random quantities (scalars or vectors/matrices)

$$x,y,\alpha ,\beta ,\mu ,{\sigma }^2,\dots$$

(1)

Upper case for all random variables (scalars or vectors/matrices)

$$X,Y,A,B,M,{\Sigma }^2,\dots$$

(2)

Examples:

- Random variable with normal distribution

$$X\sim N(\mu ,{\sigma }^2)$$

(3)

- Realization of a random vector

$$x\stackrel{\text{realization}}{\Leftarrow }X$$

(4)

Exceptions:

- Although we consistently use lower case $ϵ$ for a non-random realized residual, we use the font $\epsilon$, as opposed to the upper case $E$, for the residual as a random variable

$$ϵ\stackrel{\text{realization}}{\Leftarrow }\epsilon$$

(5)

- Consistently with most literature, we prefer to use $F$for the cdf (2b.7), even when this is not a random variable.

Scalar/vector/matrix

Non-bold for all scalar (random or non-random)

$$x,X,\alpha ,\beta ,\dots$$

(6)

Bold for all vectors and matrices (random or non-random)

$$x,X,\alpha ,\beta ,\dots$$

(7)

Examples:

- Matrix of coefficients

$$\beta \equiv {\left\{{\beta }_{n,k}\right\}}_{n=1,\dots ,\overline{n}}^{k=1,\dots ,\overline{k}}$$

(8)

- Normal random vector

$$X\equiv \left(\begin{array}{c}{X}_1\\ ⋮\\ X_{\overline{n}}\end{array}\right)\sim N(\mu ,{\sigma }^2)$$

(9)

where $\mu$ is a vector and ${\sigma }^2$ is a matrix, see also (12) below.

Symmetric, positive (semi)definite matrix

Squared bold for all symmetric, positive (semi)definite matrices (33.140)

$$s^2,{\sigma }^2,{\Sigma }^2,\dots$$

(10)

where $s,\sigma ,\Sigma ,\dots$are the respective symmetric matrix roots (33.450).

Examples:

- Covariance matrix

$${\sigma }^2\equiv {\left\{{\left[{\sigma }^2\right]}_{n,m}\right\}}_{n,m=1}^{\overline{n}}$$

(11)

of a normally distributed multivariate random variable

$$X\sim N(\mu ,{\sigma }^2)$$

(12)

- Random matrix

$${\Sigma }^2\equiv {\left\{{\left[{\Sigma }^2\right]}_{n,m}\right\}}_{n,m=1}^{\overline{n}}$$

(13)

with Wishart distribution

$${\Sigma }^2\sim Wishart(\nu ,{\sigma }^2)$$

(14)

Dummy counters

Same base letter for a dummy counter $n$ and end point $\overline{n}$

$$n=1,\dots ,\overline{n}$$

(15)

and similar for all other letters/counters $i,j,k,l,t,\dots$

Examples:

- Summation

$${\sum }_{n=1}^{\overline{n}}{c}_n$$

(16)

- Indices of a rectangular matrix (8) or square matrix (11).

Collection of variables

Calligraphic subscript for collections of (multivariate) variables

$$X_J\equiv {\left\{X_j\right\}}_{j\in J}$$

(17)

where $J\equiv \left\{j_1,j_2,\dots \right\}$ is a continuum or discrete set of indices. The counterpart of the above (17) for realized variables is

$$x_J\equiv {\left\{x_j\right\}}_{j\in J}$$

(18)

Examples:

- Entries of a vector $x\equiv {\left(x_1,\dots ,x_{\overline{n}}\right)}^{\text{'}}$

$$x_N\equiv {\left\{x_n\right\}}_{n\in N}$$

(19)

where $T\equiv N\subseteq \left\{1,\dots ,\overline{n}\right\}$;

- (Multivariate) stochastic processes monitored at selected times

$$X_T\equiv {\left\{X_t\right\}}_{t\in T}$$

(20)

where the monitoring times can be discrete $J\equiv T\subseteq Z$, or a continuum $J\equiv T\subseteq R$;

- (Multivariate) random fields monitored at selected locations

$$X_U\equiv {\left\{X_t\right\}}_{t\in U}$$

(21)

where $J\equiv U\subseteq R^k$.

Mean-covariance/probabilistic environment

Mean-covariance		Probabilistic
$X{\sim }_{Cv}\{\mu ,{\sigma }^2\}$	$X^{\text{'}}$s mean-covariance class (2a.27) is $\{\mu ,{\sigma }^2\}$	$X\sim \{\begin{array}{c}{f}_X\text{/}{p}_X\\ F_X\\ {\phi }_X\end{array}$	$X$’s distribution is $\{\begin{array}{c}\text{pdf (}\text{2b.21}\text{) / pmf (}\text{2b.32}\text{)}\\ \text{cdf (}\text{2b.7}\text{)}\\ \text{characteristic function (}\text{2b.41}\text{)}\end{array}$
$X\stackrel{Cv}{=}Y$	$X,Y$ have the same mean-covariance class	$X\stackrel{d}{=}Y$	$X,Y$ have the same distribution
$X\perp {\perp }_{Cv}Y$	$X,Y$ are uncorrelated (3a.6)	$X\perp \perp Y$	$X,Y$ are independent (3b.6)
$X\parallel z$	$X$ projected on $Z$ evaluated at $z$ (3a.39)	$X|z$	$X$ conditioned on $Z$ evaluated at $z$ (3b.11)
$X\perp {\perp }_{Cv}Y\parallel Z$	$X,Y$ are partially uncorrelated with respect to $Z$ (3a.79)	$X\perp \perp Y|Z$	$X,Y$ are conditionally independent given $Z$ (3b.41)

Data/information

We use the letter $i_t$ (lower case) for all data, namely observable information variables that are realized at a given time $t$

$$i_t\equiv {\left\{x_s\right\}}_{s\in T_t}\text{,}$$

(22)

rather than the more cumbersome notation $x_{T_t}$ (18), where the set of observations points up to time $t$ may vary from case to case

$$T_t\equiv \{\begin{array}{cc}\left\{t_1,t_2,\dots ,t_{\overline{m}}\right\}\le t& \text{finite observations, (un)evenly spaced}\\ \left\{-\infty ,\dots t-1,t\right\}& \text{infinite discrete observations, evenly spaced, includes current}\\ [t_0,t]& \text{continuous observations, finite interval}\\ (-\infty ,t]& \text{continuous observations, infinite interval}\\ \text{...}& \text{...}\end{array}$$

(23)

We use the letter $I_t$ (upper case) for the random information variables counterparts of the data variables (22), namely observable information variables that are not yet realized

$$I_t\equiv {\left\{X_s\right\}}_{s\in T_t}\text{,}$$

(24)

rather than the more cumbersome notation $X_{T_t}$ (17).

Examples:

- Equally (unit) spaced data samples of a stochastic process from inception to the latest available realization

$$i_t\equiv \{x_{t_{start}},x_{t_{start}+1},\dots ,x_{t_{end}}\},t-1<t_{end}\le t$$

(25)