Portfolio Backtesting
2. Backtesting of risk models
[this page | pdf | references | back links]
Return to Abstract
and Contents
Next page
2.1 One key reason why investors care about risk
measurement (both the computation of Value-at-Risk type risk statistics
and also the use of stress tests if derived in a similar manner), is
that it provides a guide, albeit imperfect, regarding the potential range of
future outcomes that the investor’s portfolio might experience if invested in a
particular way. This means that the risk models underlying the computations
involved are amenable to verification by comparing predictions with actual
future outcomes.
2.2 There are two ways of thinking about risk model
backtesting:
(a) It can be thought of as a quick and ‘cheap’ way to carry out such a comparison without
actually having to wait for the future to arrive. It involves identifying how
well a risk model would have worked in the past had it been applied to
the positions then present.
(b) It can also be thought
of as a core step in the calibration of a (time series based) risk model to
(past) market behaviour. To calibrate such a risk model to observed market
behaviour, we parameterise the risk model in a suitable fashion and we choose
which parameters to adopt by finding the model variant that best fits the data.
2.3 Backtesting also has a prominent (if sometimes
just implicit) role in regulatory frameworks. Regulatory frameworks have
increasingly incentivised firms to use their own risk models when determining
their own regulatory capital requirements. Such models typically need to be
approved by regulators before they can be used in such a manner. Given the
complexity of the types of firms most likely to go down this route, it is not
surprising that regulators are less than sanguine about their own ability to
mitigate the possibility that firms might adopt overly optimistic assumptions
in risk modelling. Hence, these regulatory frameworks also often include
elements that penalise firms in capital terms if their risk models too often
seem to underestimate actual magnitudes of outcomes. This makes it natural for
firms to want to understand how well their risk models might have worked in the
past (and for regulators to want to be provided with such information before
approving a firm’s model).
2.4 For firms opting to use industry-wide
regulator-specified capital computations, backtesting might appear somewhat
less important. However, this is because it has been (or ought to have been)
carried out by the regulator itself when specifying the computation in
question.
2.5 More generally, as risk measurement and
management have acquired greater importance in business management it is
natural for greater scrutiny to be placed on the validity of risk measures.
Backtesting provides one way of ‘quality assuring’ such statistics.
NAVIGATION LINKS
Contents | Prev | Next