Enterprise Risk Management Formula Book
5. Statistical Methods
[this page  pdf  back links]
5.1 Sample moments
A random sample of observations has (equally
weighted) sample moments as follows:
‘Population’ moments (e.g. population variance,
population skewness,
population excess
kurtosis) are calculated as if the distribution from which the data was
being drawn was discrete and the probabilities of occurrence exactly matched
the observed frequency of occurrence.
The least squares estimator for parameters of a distribution
are the values of the parameters that minimise the square of the residuals, so
the least squares estimator for the mean, , is the value
that minimises
Nonequally weighted moments give different weights to
different observations (the weights not dependent on the ordering of the
observations), e.g. the sample nonequally weighted mean (using
weights ) is:
5.2 Parametric inference (with an underlying
following the normal distribution)
One sample:
For a single (equally weighted) sample of size ,
, where then the
following statistics are distributed according to the Student’s t
distribution and the chisquared distribution:
Two samples:
For two independent samples of sizes and
, and , where and then the
following statistic is
distributed according to the F distribution:
If then:
where is the pooled
sample variance.
5.3 Maximum likelihood estimators
If is the maximum likelihood estimator
of a parameter based on a
sample then
where is the
likelihood for the sample, i.e. and hence
is
asymptotically normally distributed with mean and
variance equal to the CramérRao lower bound
Likelihood ratio test:
where is the
maximum loglikelihood for the model under (with free
parameters) and is the
maximum loglikelihood for the model under (with free
parameters). Nonequally weighted estimators can be identified by weighting the
terms
appropriately.
5.4 Methodofmoments estimators
Method of moments estimators are the parameter values
(for the parameters
specifying a given distributional family) that result in replication of the
first moments of
the observed data. For the normal distribution these involve and either (the sample
variance, if a small sample size adjustment is included) or (the
‘population’ variance, if the small sample size adjustment is ignored and we
select the estimators to fit and . In the generalised
method of moments approach we select parameters that ‘best’ fit the
selected moments (given some criterion for ‘best’), rather than selecting
parameters that perfectly fit the selected moments.
5.5 Goodness of fit
Goodness of fit describes how well a statistical model fits
a set of observations. Examples include the following, where is the ’th
order statistic, is the
supremum (i.e. largest value) of the set , is
the cumulative distribution function of the distribution we are fitting and is the
empirical distribution function:
(a) KolmogorovSmirnov
test: . Under the
null hypothesis (that the sample comes from the hypothesized distribution), as then
tends to a
limiting distribution (the Kolmogorov distribution).
(b) CramérvonMises test:
(c) AndersonDarling test:
where
If data is bucketed into ranges then we may also use
(Pearson’s) chisquared goodness of fit test using the following test
statistic, where is the sample
size and is the
observed count, is the
expected count and and are the lower
and upper limits for the ’th bin. The test
statistic follows approximately a chisquared distribution with degrees
of freedom, i.e. where is
the number of nonempty cells and is the number
of estimated parameters plus 1:
We may also test whether the skew or kurtosis or the two
combined (the JarqueBera test) appear materially different from what would be
implied by the relevant distributional family. If the null hypothesis is that
the data comes from a normal distribution then, for large ,
, and .
The Akaike
Information Criterion (AIC) (and other similar ways of choosing between
different types of model that tradeoff goodness of fit with model complexity,
such as the Bayes Information Criterion, BIC) involves selecting the
model with the highest information criterion of the form where there
are unknown
parameters and we are using a data series of length for
fitting purposes. For the AIC and for the
BIC .
5.6 Linear regression
In the univariate case suppose where , then
(equally weighted) estimates of and are:
where
Also
The individual expected responses are and satisfy
the following ‘sum of squares’ relationship:
The variance of the predicted mean response is:
The variance of a predicted individual response is the
variance of the predicted mean response plus an additional .
For generalised least squares, if we have different
series each with observations
we are fitting then the
vector of least squares estimators, is given by where is a matrix
with elements and is an dimensional
vector with elements .
5.7 Correlations
The observed (sample) correlation coefficient
(i.e. Pearson correlation coefficient) between two series of equal
lengths indexed in the same manner and is (where , and are as given
in the section on linear regression):
If the underlying correlation coefficient, ,
is zero and the data comes from a bivariate normal distribution then:
For arbitrary ()
the Fisher z transform
is where:
If the data comes from a bivariate normal distribution then is
distributed approximately as follows:
Two nonparametric measures of correlation are:

Spearman’s
rank correlation coefficient, where and are the ranks
within and of
and respectively:

Kendall’s
tau, where computation is taken over all and
with and
(for the moment ignoring ties) a concordant pair is a case where and a
discordant pair is a case where :
There are various possible ways of handling ties in these
two nonparametric measures of correlation (ties should not in practice arise
if the random variables really are continuous).
5.8 Analysis of variance
Given a single factor normal model
where with .
Variance estimate:
Under the null hypothesis given above
where:
5.9 Bayesian priors and posteriors
Posterior and prior distributions are related as follows:
i.e.
For example, if is a random
sample of size from a where is known and
the prior distribution for is then the
posterior distribution for is:
where is
‘credibility weighted’ as follows:
and
NAVIGATION LINKS
Contents  Prev  Next