Portfolio Backtesting

2. Backtesting of risk models

[this page | pdf | references | back links]

Return to Abstract and Contents

Next page


2.1          One key reason why investors care about risk measurement (both the computation of Value-at-Risk type risk statistics and also the use of stress tests if derived in a similar manner), is that it provides a guide, albeit imperfect, regarding the potential range of future outcomes that the investor’s portfolio might experience if invested in a particular way. This means that the risk models underlying the computations involved are amenable to verification by comparing predictions with actual future outcomes.


2.2          There are two ways of thinking about risk model backtesting:


(a)    It can be thought of as a quick and ‘cheap’ way to carry out such a comparison without actually having to wait for the future to arrive. It involves identifying how well a risk model would have worked in the past had it been applied to the positions then present.


(b)   It can also be thought of as a core step in the calibration of a (time series based) risk model to (past) market behaviour. To calibrate such a risk model to observed market behaviour, we parameterise the risk model in a suitable fashion and we choose which parameters to adopt by finding the model variant that best fits the data.


2.3          Backtesting also has a prominent (if sometimes just implicit) role in regulatory frameworks. Regulatory frameworks have increasingly incentivised firms to use their own risk models when determining their own regulatory capital requirements. Such models typically need to be approved by regulators before they can be used in such a manner. Given the complexity of the types of firms most likely to go down this route, it is not surprising that regulators are less than sanguine about their own ability to mitigate the possibility that firms might adopt overly optimistic assumptions in risk modelling. Hence, these regulatory frameworks also often include elements that penalise firms in capital terms if their risk models too often seem to underestimate actual magnitudes of outcomes. This makes it natural for firms to want to understand how well their risk models might have worked in the past (and for regulators to want to be provided with such information before approving a firm’s model).


2.4          For firms opting to use industry-wide regulator-specified capital computations, backtesting might appear somewhat less important. However, this is because it has been (or ought to have been) carried out by the regulator itself when specifying the computation in question.


2.5          More generally, as risk measurement and management have acquired greater importance in business management it is natural for greater scrutiny to be placed on the validity of risk measures. Backtesting provides one way of ‘quality assuring’ such statistics.


Contents | Prev | Next

Desktop view | Switch to Mobile