Basel II endorses stress testing as one of the main risk assessment tools. While the definition of stress testing is open to interpretation, it is mostly taken as the change in the value of a portfolio resulting from external market conditions. In practice, the process is made up of four distinct steps:

1. External stress events (such as bird flu or default of a major financial institution) are ‘invented’.
2. These events are ‘translated’ and their economic impact on market factor (interest rate, GDP, …) is modeled
3. Running simulation on an organization’s portfolio based on the change in the market factors
4. Producing statistics to compare the result to the modeled portfolio in comparison with the same portfolio under ‘standard’ conditions

While the latest credit crises have demonstrated the necessity of stress testing, a complementary, yet just as important application of the same methodology has been mostly neglected: testing the stability of our measurements.

In most financial institutions, once a model has been verified and approved, it becomes part of the workflow and is rarely questioned or tested again. However, as all models have their limitations, understanding the limitations of a model is just as important as knowing its output.

One of the limitations of any risk model is its sensitivity to input parameters. In some cases models can become too insensitive, when big changes in market factors will only produce minute changes in the output. In other occasions, models may demonstrate the opposite behavior and become too sensitive, producing a ‘butterfly effect, that is, minute changes in the input parameters produces unjustifiably large changes in the output. Such behaviors could happen with any model, and don’t reflect on the quality of the model in general, but rather indicate that we are trying to use the model under conditions to which it is not calibrated.

Regularly using stress testing like functionality with marginal-change scenarios, rather than the ‘typical’ stress scenarios, will highlight the occasions when we cannot blindly rely on our model– a crucial piece of information when using any model for making decisions.

RiskMetrics Group will be holding a webcast, Iterative Basel II Implementation for European Banks: Credit simulation in low data coverage environments, on Wednesday, Feb. 20 at 14:00 GMT/15:00 CET. The webcast will cover: low data environments, the three pillars of Basel II, different approaches to missing data, the iterative approach to implementing Basel II, and the weakness of the linear approach to Basel II in a low data environment.

To register for the webcast, please visit here.