Shipping Fair Value...

...seems like the shipping industry is as about as confused as the finance industry about establishing "fair value" for assets according to this article in the FT.

Pricing Model Validation: Mitigating Model Risk

I managed to catch some of the day yesterday at the "Pricing Model Validation: Mitigating Model Risk" conference. I thought it would be worthwhile going along since firstly the past 12-18 months have made model risk very topical (take a look at previous posts from Riskminds, the Modeller's Manifesto and Wilmott/Rowe).

Secondly more of our clients are looking at managing and centralising pricing models/curve calculators in addition to just managing the underlying data (see this Insight Investment client case study for a recent public example). I am calling this "Analytics Management" which is the business-focussed technology stack that combines pricing models/calculators/analytics with all of the "Data Management" underneath. But enough of my thinly-veiled positioning statements...and on with some of the (hopefully) useful content from the conference outlined below - maybe scan the headings in bold below for those talks of interest but I would particularly recommend the ones by Tanguy Dehapiot and Yuyal Millo...

Model Risk 2009 defining and forecasting. First speaker was Professor Phillip Sibbertsen of the University of Hannover on defining and measuring model risk. Phillip started by saying that "Model Risk" was a new category of risk within the confines of "Operational Risk", and that operational risk as defined by the regulators does not yet currently include the "model risk" of market risk and credit risk, nor the "model risk" of the operational risk model itself. (I am sure I could write that up better!...). Phillip put forward that model risk is not formally a "risk" since it has no probability distribution and that he suggested it should be thought of as "model uncertainty". He also clarified that model risk applies both at the large, portfolio scale (e.g. choice of VAR model etc) and at the smaller, instrument level scale (i.e. pricing of derivatives).

Additionally in terms of measuring model risk then he excluded human failure from model risk measurement since in his view this was difficult to quantify - this approach did not meet with the approval of some of the audience were questioning how this could be excluded from a practical point of view. Phillip's colleague, Corinna Luedtke, then presented some work they had done on calibrating different GARCH models to observed data and showing how even a poor model could produce reasonable forecasts of risk if the time period was short. The work was interesting but again the audience highlighted that the human choice (failure?) in choosing the set of models to try was part of "model risk" and should not be excluded from the definition of model risk.

Is a model accurate? Testing the implementation of a model. Second speaker was David Chevance, Head of Equity & FX Model Validation at Dresdner Kleinwort. David outlined the different sorts of model risk: mathematical errors, missing risk factors, divergence from industry practice, model inconsistencies and implementation risk. He then outlined the sources of these risks: bugs, approximations, numerical precision, numerical boundaries and limitations on numerical methods (e.g. Sobol numbers in high dimension monte-carlo simulations).

David said a key area to start with in validating a model implementation was the front-office documentation of the product, its inputs and payoffs, its pricing model but also details of calibration methods used/needed etc. He made the point here that the documentation can sometimes specify just the deal, but sometimes can express the pricing methodology and pricing parameters. The emphasis was on completeness, accuracy and making use of all of the information available in the documentation. Obviously the ability to review the code used to implement the model was also necessary.

He discussed the trade-offs between a simple validation approach in terms of speed and efficiency of resources against the more time-consuming, resource hungry but more accurate approach of full replication of the model. He also suggested that in choosing a method of validation it was important to balance resource demands against what is actually being validated: payoffs from a single trade, a type of pricing model or a family of financial products. Desired accuracy of the validation was also important, given the trade-off between accuracy and effort, and the fact that small bugs are much more common than large.He finally discussed model version control, the necessary discipline of documenting changes and regression tests for new models, and the regular cycle of model review. Overall it was an interesting talk with a good practical focus.

Practical aspects of valuation model control process. One of the most entertaining and interesting speakers of the day was Tanguy Dehapiot, Head of Validation and Valuation, Group Risk Management at BNP Paribas. He started by referring to a few documents "Supervisory guidance for assessing banks’ financial instrument fair value practices", April 2009 (BCBS 153) which was then implemented within “Enhancement to the Basel II framework” (BCBS 157). The first part of his presentation was around these documents and what the regulators expect to be in place, so I guess the best approach is to read them (the BCBS 153 document content is only 12 pages long, quite short for a regulator!)

Tanguy pointed out that in his view "Mark to Market" and "Mark to Model" are often misleading as both are often required. He prefers the term "Valuation Methodology". He proposed four valuation modes: Direct Price Quotation, Use of Similar Instruments, Risk Replication, Expected Uncertain Cashflows (NPV) and categorised a useful hierarchy/matrix of which financial products fit into which valuation mode and for what purposes. Within model risk, he split off judgemental errors (choice of model etc) as part of market risk and credit risk and operational errors (model implementation and coding) as more definable and avoidable parts of operational risk.

He had some interesting slants on data, saying that he had been surprised that even getting all of the static data necessary to price simpler instruments like bonds had proven difficult. He outlined how model parameters are often stored across a variety of systems (curve definitions in one place, pricing methodology somewhere else) implying to me that this is sometimes difficult to pull together and needs some centralisation to improve transparency around this.

His opinion on market parameters (both observed prices and derived data such as implied volatility surfaces) were often stored in a larger central database but warned that this market parameter database needs to be reviewed as part of the model validation process since some of its data is derived (i.e. calculated, maybe using a model!) and as such should not be taken as perfect for all time and for all purposes. He said that it was important to categorise the origin of data and suggested the following types:

• Quoted on an active exchange
• Actual private transaction in an active market
• Tradable broker quotes
• Consensus prices from market makers
• Non-binding indicative prices from market makers
• Counterparty valuation, collateral valuation
• Actual transactions in inactive market

Tanguy proposed that there should a valuation matrix for each instrument, where there might a different valuation methodology used for end of day valuation verses intraday, for risk or for trading, for pricing individually or within a portfolio reval. I guess here the rational is appropriateness, efficiency and transparency about what needs to used when. He also added that he disliked the term "Model Validation" since it seemed to imply that a model was "valid" and preferred "Model Approval" to cover the decision to use a model and "Model Review" to cover model analysis. He said he found managing the "stock" of existing models (and keeping up with when to review them) more difficult than managing the "flow" of new models and products.

Overall Tanguy was a very interesting and funny speaker with lots of practical insights and a fair amount of opinion thrown in, which is always good in my view.

The usefulness of inaccurate models: Financial risk management "in the wild". This talk was given by Dr Yuval Millo of the London School of Economics and he focussed on the evolution of the use of the Black Scholes Merton (B-S-M) model at the CBOE and how the model came to be the means by which the whole options market "communicated". Yuyal is a social scientist and prefaced his talk by stating that "Social Sciences are good at predicting the past"

First thing I didn't know (amongst the many things I do not know...) is that the B-S model was not published until a couple of weeks after the CBOE started trading stock options in April1973. Yuyal said that initially the B-S-M derived prices were not accurate at all (around 25% off the market price on CBOE) and that the model was based on assumptions that plainly were not the case on the exchange (only calls available, no short selling, no continuous trading). The model was used by local Chicago trading firms and the story goes that Fischer Black sold large paper "sheets" of option pricing matrices to these traders (there being no calculators/PCs/mobiles around at the time).

As the markets developed, larger East Coast banks entered the market with stocks being held and traded in New York and options being traded in Chicago, so trading became geographically dispersed. This started the need for "early morning meetings" to discuss the market and the B-S-M model and its parameters became the "lingua franca" or means of communication of options market participants.

He described the first years of the Options Clearing Corporation (OCC) which was set up to ensure that the financial obligations of options and buyers were met. Around 1979-80 the OCC worked overnight to calculate margin requirements, based on the (now?) arcane idea that different margin amounts should be associated with different option strategies (straddles, butterflies etc) and the job of the OCC was to take a portfolio of Option and optimise which combination of strategies would minimise the margin required for the whole portfolio. He said that there were disputes between traders and the OCC around margin levels and difficulties for the SEC with updating their Net Capital Rules as each new option strategy was created. Eventually, the OCC adopted the B-S-M model and implied volatility as the means of calculating margin against market value which enabled them to move away from the operational difficulty of strategy optimisation.

So the B-S-M became the way in which traders communicated about the market but also the model became vital operationally within clearing for the market. By 1987 B-S-M had become the de-facto standard for the market, with the model driving the market in turn driving use of the model. During the Oct '87 crash the model proved to be very innaccurate but the use of the model did not diminish - maybe pschologically the market participants needed a model (even a wrong model) to make communication easier.

I found this talk very interesting and members of the audience asked whether any similar analysis was going to be done on the Gaussian Copula model used to price CDOs. Yuyal said that one of his colleagues was undertaking this research currently. Given that he seemed to be very positive about the use of the B-S-M model within options markets I asked whether he had any opinions on Taleb's criticism of fiancial engineers and modelling. Yuyal said that he and Nassim were friends and agreed to disagree on certain topics...

Stress testing modelling parameters. Next up was Peirpaolo Montana, Head of Model Validation at West LB. Having joined the finance industry out of a career in mathematics and then at a regulator, Pierpaulo began by saying that back in the heady days of 2004 the banks thought that their own risk management systems and practices were well ahead of the regulators. He said that in light of the crisis this proved not to be the case but he now feels that this is now more evenly balanced (not sure I would agree, still lots of catchin to do for some institutions I would suggest).

He said that whilst regulators require the validation of risk models and pricing models, and that stress testing of a portfolio is required, that the stress testing of a pricing model is not a requirement and has received much less attention and in his view was not done to much degree before 2007. His point here was that pricing models should work under stress too, otherwise they are a weak foundation for building other risk measures such as stressed VAR.

Whilst focussing on pricing models, he mentioned that risk models also need to be carefully chosen and appropriate to the institution and the types of trading activities it undertakes. As an example he put forward that a simple VAR calculator might be appropriate for a long only equity fund but completely innappropriate for a relative value portfolio.

He said that stress testing had recently received much more attention as a risk management tool and cited the BIS document "Revisions to the Basel II market risk framework" where stressed VAR is introduced as part of the regulatory capital charge calculation. He also mentioned that in order to avoid "standard model" treatment of complex securitised products an institution must be able to demonstrate that its VAR model can cope with these products under times of market stress.

Pierpaulo then described the stress testing of base correlation in CDO pricing, and how even moving the base correlation from its usual level of 70% to 99% would not have predicted the valuations observed in the recent crisis. In this way he says that stress testing of models can detect implementation problems and some model weaknesses, but it cannot assist in coping with structural breaks in the market. He also discussed how the B-S-M model is used everywhere (even places it should not really be valid for) since it is a robust model based on the no-arbitrage hypothesis - in contrast the CDO base correlation and other models are not so robust since they are not arbitrage free.

(end of post!)
 

 

Regulatory moves and moods

Seems that the latest EU and Basel Committee proposals on banking regulation cannot make everyone happy (now there's a surprise...). Whilst many seem very happy at the incremental nature of the proposals to increase capital requirements for securitisations and proprietary trading, some of those in the Glass-Stiegal/banking utility camp are less than impressed. I am with the incremental camp myself, but have to acknowledge that the sceptics are not short of ammunition when saying that we are heading back to the future...meanwhile over in hedge fund land, London is currently in a very bad mood with the EU...

Lessons for Risk Management - Wilmott and Rowe

Great event organised by PRMIA and IAFE last night at Goldman's London offices with a long title:

 "A Little Thought Goes A Long Way and Lessons for Risk Management from the Current Crisis".

The event was moderated by Giovanni Bellossi of FGS Capital, and featured speaking slots by Paul Wilmott and David Rowe of Sungard. Here are my notes on the evening, please forgive any innaccuracies, and please persevere through some of the techy quant stuff, as their general points are well worth understanding.

• Giovanni quoted from Nassim Taleb about how VAR is invalid and that mainstream financial mathematics should be banned (or words to that effect, see earlier post on Taleb)

• He added that whilst what Taleb says cannot be ignored, he said that despite the current crisis and its causes that we should not "throw the baby out with the bathwater" and added that Taleb "...is not only able to recognise a cow but also knows how to milk one."

• Giovanni said that financial mathematics has much to offer and that whilst VAR is simply a number, one of its great benefits has to make one measure of risk simple and compelling enough to get traders and risk managers talking.

Paul Wilmott then took the floor and put forward his thoughts:

On Taleb and the Black-Scholes Model

• Paul mentioned that he and Taleb were great friends, and whilst he agreed with much of what Taleb says he has areas of disagreement, particularly over the use of the Gaussian distribution in finance and its implications for "fat tail" events
• Paul Googled "Taleb" and found more entries for Taleb than for Stephen Hawkin which shows how much attention had come his way due to the "Black Swan" debate
• He thinks that he and Taleb are the "Marmite of finance" (for those of you not in the UK who do not know Marmite, it is a sandwich spread that you either love or hate, never anything inbetween)
• He suggested that every quant needs a much more fundamental and practically grounded understanding of financial mathematics.
• Paul refered to some work (mentioned by Giovanni) that Peter Carr of Bloomberg had done on discrete daily hedging that showed that this option replication technique could remove up to 85% of the risk and that all quants should know about this 15% error term when trying to calculate an option price to the Nth decimal place.
• He described how in the past he had set up a volatility arbitrage hedge fund, wanting to improve upon the flawed assumption of the Black-Scholes (B-S) model that volatility is constant and to build the world's best volatility model for option pricing.
• Paul said that he did build the world's best volatility model (?!), but soon found it took too long to calculate, so he reverted back to B-S and has become an unfashionable fan of the model and its assumptions.
• He added that many of the variants on B-S to overcome its limitations have made the model worse and harder to calibrate.
• In some part due to Taleb's opinions on fat tails of distributions, B-S and other models are now very unpopular but Paul claims that not many people have actually bothered to robustly test the B-S model or take a practical, evidence based approach such as that adopted by Peter Carr.

• Paul then showed some example charts and said that with a limited number of opportunities for regular time-period hedging it was not valid to use risk-neutral pricing whereas if the same number of hedges could be used optimally (implying at irregular time periods) then risk-neutral was valid and hedging could be more effective. He emphasised that this was the kind of practical stuff that a quant should know and that quants show know less about esoteric complex financial mathematics.

Correlation

• Paul said that of all of the issues that need addressing in mathematical finance, the one that he has very few answers on is correlation.
• He showed that even basic questions about correlation are poorly understood, even by quants - a question he asks some quants was that if two asset prices both start out at 100, and they have a correlation (of returns) of 1 (perfect correlation) what is the price of the second asset after a year if the first moves to 200. The answer is not 200, and he showed how assets could diverge in overall direction but still have a correlation of 1 or rise together with a perfect negative correlation of -1.
• Paul illustrated how correlation was a very blunt measure that is mis-used by people to summarise the highly complex and historically unstable relationships between assets driven for example by industry sector success (leading to +ve correlation) or competitive success (leading to -ve correlation)
• As a result, he said that financial products whose value depends on correlation should not be transacted in any great size and moved on to the example of CDOs, where a CDO with 1,000 underlying mortgages has been modelled with 1/2 million correlations all assumed to be 0.6. Why this assumption should be made was his main point.

Sensitivity to Parameters

• His main point here was that a constant should not be varied, otherwise it is not a "constant", in particular focussing on volatility used in the B-S model and the calculation of Vega as prices are moving.
• Paul added that sensitivity measures may apply locally and is such may look comparible from one situation to another, but quants need to understand how outputs respond over a wider range of inputs, and not to be inhibited by accepted practices and beliefs.

Complexity

• Models need to be robust and transparent, and that quants should aim for the mathematical sweet spot.
• Paul put forward the following analogy that at least when driving an old car over a long distance, you knew that the car was likely to break down at least once, but you also knew that it was likely that you could fix it. Contrast this with driving a modern sports supercar and finding that it has (unexpectedly?) broken down - you don't know how to fix it, you do not complete your journey and it costs you an ordinate amount of money to put things right...

Self-Referential Feedback

• Paul described here how the hedging of derivatives contracts in the underlying markets can cause price movements in underlying markets that cause derivatives contracts to re-price that cause more hedging in the underlying markets...
• He was critical of credit derivative pricing as being too complex and too "mathsy" (...but had to admit that he had also endorsed some of this work at the time)

Calibration

• Paul said that model parameter calibration is the devil's work...
• He refered us to inverse problems in mathematics as a background to this issue in mathematical finance.
• He emphasised how markets and price behaviour is fickle and driven by human opinions and behaviours
• He said that on-going and regular re-calibration of a model is very, very likely to mean that the model is wrong (he had a particular example of calibrating a particular model he hates where vol is a function of underlying price and time.

David Rowe, Sungard's specialist spokesman on risk management, then took over from Paul and set out his five topics for discussion:

• Statistical Entropy - fundamentally that information can only be extracted from data, with the emphasis on extraction of information (from that already in the data) rather than creation of new information.
• Structural Imagination - that we need to be aware of how the market assumptions we make are themselves a model and that we need to spend more time on thinking about what could happen outside our current understanding or market experience.
• Self-Referential Feedback - the feedback loops in pricing, risk management and economics
• Complexity and Dark Risk - when you add (untested) complexity of a model to limited data sets you get a recipe for disaster.
• Alternate Means of Valuation - when the primary means of valuing a security is not available (illiquid markets anyone?) then what is the secondary means of calculation value.

Some further notes from David's talk:

• AAA rating should imply a failing once every 10,000 years, with some super senior CDO tranches being rated as better than AAA - David pointed out that even as recently as the early 1990s there were problems in the US housing market that indicated that AAA did not mean what it was taken to mean.
• On structural imagination, David said that quants and risk managers must look for unrepresented variables in a model and track them early to monitor their effects
• On feedback he cited an example where increased returns drove product innovation which drove up (CDO) volumes, which caused underwriting standards to fall, that allowed further complexity, that then led to unreliable risk estimation which then led to more product innovation... and so on.
• He suggested that quants adopt the "second means of valuation" mantra in a similar way to credit specialists always having the mantra when assessing credit of "what is the second means of repayment" (e.g. a lien on a house) when the primary means (mortgage payments) goes away.
• David showed a nice classification from an IASB paper on classifying financial instruments:

Level 1: fair values measured using quoted prices in active markets for the same instrument.

Level 2: fair values measured using quoted prices in active markets for similar instruments or using other valuation techniques for which all significant inputs are based on observable market data

Level 3: fair values measured using valuation techniques for which any significant input is not based on observable market data

David additional proposed the interesting level of "Level ?" for some products, and said that obviously more attention needs to spent on Level 2 and 3 instruments under conditions of reduced (non-existant?) market liquidity.

Summary Session:

Paul and David then answered some questions from the audience:

• Paul said that some risk managers lacked the imagination necessary for good risk management, being confined in standard procedures, beliefs and ways of doing things. He wants risk managers who are good at thinking laterally.
• Paul said that risk management was often an afterthought, not part of the trading process.
• David said that VAR has proven useful despite its weaknesses, in his opinion preventing failures from non-extreme events regardless of the recent extremes
• David said that in answer to Taleb's criticism of using history in modelling, it quite frankly is all we have to go on. He quoted Mark Twain in that:

"History does not repeat itself but it does rhyme"

The talks were interesting, and even on points that have been discussed elsewhere both speakers had some interesting slants and good analogies. But maybe I am biassed, as the wine afterwards wasn't bad either!...

Over The Counter Arguments

George Soros has waded back into the current saga concerning OTC derivatives in his article last week in the FT. The main part of the article focusses on financial markets reform, but ends with a vehement attack on derivatives, building upon some of his earlier ideas (see post) and seemingly going much further:

"Finally, I have strong views on the regulation of derivatives. The prevailing opinion is that they ought to be traded on regulated exchanges. That is not enough. The issuance and trading of derivatives ought to be as strictly regulated as stocks. Regulators ought to insist that derivatives be homogenous, standardised and transparent."

He ends by saying that "CDS are instruments of destruction that ought to be outlawed.". To the extent that Mr Soros attracts press/political attention is probably something the OTC markets should worry about, although it would seem his views are already consistent with many involved in influencing the US financial markets policy - take for instance the submission by Christopher Whalen to the US Senate on OTC Derivatives:

"Simply stated, the supra-normal returns paid to the dealers in the closed OTC derivatives market are effectively a tax on other market participants, especially investors who trade on open, public exchanges and markets."

Fortunately however there are also some more balanced views around - I found the following post on the "(in)efficient frontiers" blog, which references the earlier Senate submission by Richard Bookstaber on OTCs. Mr Bookstaber starts by saying that derivatives can improve financial markets, allowing investors to shape returns, exactly meet contingencies and package risk. Mr Bookstaber also puts forward a very clear summary how participants have also over recent years use derivatives to game the system to achieve tax avoidance, investment mandate avoidance, speculation and to hide risk-taking.

So back to the Soros article, there was a letter in response a few days later from a partner at the legal firm Ashurst's, saying that unfortunately risk does not confirm to a standard. In this I agree, standardising contracts can lead to increased complexity - there was a recent example given by a swaps dealer at JPMorgan who said that a corporate with particular cashflows to be hedged does want to be dealing with the basis risk and admin of using standardised contracts - the corporate treasurer wants something that matches the exposure they have and takes it away, end of story. Again this is an example of derivatives "risk" not being just about the product type, but also about which institution is holding the contract and what they are using it for (see earlier post).

Not sure however how much the Ashurst's partner who wrote the response letter is worried about lucrative legal fees for OTC derivative contracts dying off if Soros-like standardisation occurs - it is a world of vested interests at the moment, never more vested than in a crisis...

 

Liquidity Derivatives - the next OTC?

Given the drive the FSA is making in forcing financial institutions to implement "Liquidity Risk Management" (see background on JWG-IT site) are we going to see renewed interest in the creation of "Liquidity Derivatives" to hedge liquidity risk? I found the following post on the subject applied to hedge funds but not much information else where, although Tony Jackson did an interesting article on liquidity in the FT last week, indicating that liquidity derivatives have been tried before with little success.

I was thinking of the advent of credit derivatives being driven in no small part by Basel II regulation on capital charges for credit risk. Maybe given the current battle going on around OTC regulation (see FT feature today) there are institutions working on liquidity derivatives but nobody in the finance industry wants to admit that they are already creating the next "innovative" OTC to nullify regulatory charges?

Mr Geithner better watch out, innovation will always beat "rules" in my view...

What Investment Banking is all about?

I admire the boldness and openness of Richard Jory, Editor of Structured Products Magazine, for asking the serious and under-answered question whether it is a problem or not that the likes of the equity derivatives division of BNP Paribas lose 1.5 billion Euro in one quarter after making around 15 billion over the past five years? Certainly there is an automatic rush to condemn such losses given their size and the current context of the global financial crisis.

The first sentence of Richard's article should be framed as an excellent and apt piece of prose - I am not sure he is behind the times or predicting a future return to investment banking normality:

"Losing a lot of money in one year, or one quarter after making a whole truckload of money in the preceding five years is, frankly, what investment banking is all about."

Not sure he will find many regulators backing his view, but let's see...

Challenging Fair Value

Concise letter on the continuing debate on fair value accouting to the FT from Hugh Shields, Chief Economics Advisor to the Institute of Chartered Accountants of Scotland.

It seems that most commentators come down positively on the side of fair value accounting from what I have read, with the two main points of:

• Don't blame the messenger
• Pro-cyclical behaviour is driven by the regulatory calculation based on fair value accounting, not by fair value accounting in itself

A recent paper "The Fair Value Controversy: Ignoring the Real Issue" and survey "Reactions to an EDHEC Study on the Fair Value Controversy" by EDHEC seem to support this view, with only 25% of respondents believing that any amendments are necessary, and 75% believing that changes will only lead to more problems.

Unfortunately, it would seem that the SEC in the US has produced 250 pages of suggested tweaks to fair value accounting (see Lex article). Maybe the desire for preventative rules and the political need to be seen to be "doing something" are too strong for regulators to resist...

Originative sin

Very balanced article in the FT by John Plender on innovation in financial markets, quoting Merton Miller as saying in 1986 that "the major impules to successful financial innovations have come from regulations and taxes". Apart from a good narrative on the current crisis, Plender ends by suggesting that as both taxes and regulation are likely to increase in the near future, it may be unwise to call the death of financial innovation just yet.

Independent No More?...

John Gapper article again from the FT - this time announcing the death of the independent investment bank (click here). Thought it was worth noting this one as a benchmark for where people's thoughts are now...

Never ending liquidity for FX?

FX volumes grow from $99,000bn in 2007 from $71,000bn in 2006 - growth driven by automated trading, cheaper execution leading to more and more participants. How fluid can the market become? Article link:

http://www.ft.com/cms/s/0/10fc4512-2c4f-11dd-9861-000077b07658.html?nclick_check=1

Investment banks after the crisis...

FT questioning what the investment banking industry will look like once the credit crisis has passed, in particular whether big name "stand alone" investment banks can continue to prosper when faced with competition from broader based institutions with a strong retail base:

http://www.ft.com/cms/s/0/80b3cbac-24ff-11dd-a14a-000077b07658.html