Gaussian Processes
Consider the linear model:
latex(\Huge $y(\mathbf{x}) = \mathbf{w}^{\prime}\phi(\mathbf{x})$)
and the prior distribution:
\mathbf{0},\alpha^{-1}\mathbf{I})$)
now latex($y(\mathbf{x})$) is a random function or process.
In practice, we evaluate this function at discrete points latex($\mathbf{x}_1,\mathbf{x}_2,\ldots,\mathbf{x}_N$). Hence, we want the joint distribution of latex($\mathbf{Y} = \{y(\mathbf{x}_1),y(\mathbf{x}_2),\ldots,y(\mathbf{x}_N$)\}). latex($\mathbf{Y}$) is given by:
latex(\Huge $\mathbf{Y} = \mathbf{\Phi}\mathbf{w}$)
or
That is, latex($\mathbf{Y}$) is a linear combination of normally distributed random variables, as therefore is itself a (multivariate) normally distributed random variable with mean and variance:
![\Huge
\begin{eqnarray}
\mathbb{E}[\mathbf{Y}] &=& \mathbf{\Phi}\mathbb{E}[\mathbf{w}] = \mathbf{0} \nonumber \\
\mathrm{cov}[\mathbf{Y}] &=& \mathbb{E}[\mathbf{Y}\mathbf{Y}^{\prime}] = \mathbf{\Phi}\mathbb{E}[\mathbf{w}\mathbf{w}^{\prime}]\mathbf{\Phi}^{\prime} = \alpha^{-1}\mathbf{\Phi}\mathbf{\Phi}^{\prime} = \mathbf{K} \nonumber \\
K_{nm} &=& \frac{1}{\alpha}\phi(\mathbf{x}_n)^{\prime}\phi(\mathbf{x}_m) \nonumber
\end{eqnarray}](/wiki/Classes/BMTRY790/KernelMethods/040_Gaussian_Processes?action=AttachFile&do=get&target=latex_424b0f1a8737763532046cf057ee9646ec033132_p1.png "\Huge
\begin{eqnarray}
\mathbb{E}[\mathbf{Y}] &=& \mathbf{\Phi}\mathbb{E}[\mathbf{w}] = \mathbf{0} \nonumber \\
\mathrm{cov}[\mathbf{Y}] &=& \mathbb{E}[\mathbf{Y}\mathbf{Y}^{\prime}] = \mathbf{\Phi}\mathbb{E}[\mathbf{w}\mathbf{w}^{\prime}]\mathbf{\Phi}^{\prime} = \alpha^{-1}\mathbf{\Phi}\mathbf{\Phi}^{\prime} = \mathbf{K} \nonumber \\
K_{nm} &=& \frac{1}{\alpha}\phi(\mathbf{x}_n)^{\prime}\phi(\mathbf{x}_m) \nonumber
\end{eqnarray}")