Wondering about the various mathematical formulas that were used by various Wall Street firms in quantifying risk? Here are several pieces that discuss the role quants played in modeling risk and uncertainty:
- My Manhattan Project by Michael Osinksi (NY Mag)
- Recipe for Disaster by Felix Simon (Wired)
- How Wall Street Lied to Its Computers by Saul Hansell (NY Times)
In addition you may enjoy two relatively recent pieces by Michael Lewis, The End of Wall Street’s Boom and Wall Street on the Tundra
{ 22 comments }
I think the points are wrong. I can not see anything bad on trying to quantify risk. It’s just that it has to be backed-up with cautious. The point of “knowledge” is that it’s a moving target. What is right one day may turn out be be flawed in reallity. I just suggest playing around with the formulars and then you start manipulating them. You found something you like to “support” and you have a model someone believes in as “truth” but with small changes you can give a much different picture.
And no formula is responsible for the debts of politicians. They just do not care or better they just care enough that they get as much debt as possible for their “political” surviving. However it all comes back to “worthless” money vs money value….
Just a couple of quick thoughts: The problem with these models is that they don’t include manipulation by an Invisible Hand (sort of speaking) like the U.S. Govt, Feds (like pundits say, don’t bet against the Feds), and others……
If the financial markets were completely efficient with no manipulation you could better trust a mathematical model (although several say that you could never predict a fully efficient Market). Working models of an efficient market likely would be modeling human action.
But, again, I think that a good, trustworthy Financial Market mathematical model must include, or be explained, in psychological terms.
There are three conditions that are required to perform a regression analysis. First, the error variable must be normally distributed. Second, the error variable must have a constant variance. Third, the errors must be independent of each other. None of these conditions are satisfied when we are dealing with human action. Regression analysis is therefore not proper when analyzing human action.
The use of probability is also inappropriate when dealing with human action. Using probability is only appropriate if the phenomenon is random, homogeneous, and repeatable. Obviously, none of these conditions are satisfied when dealing with human action (you can’t speak of probabilities with regard to an election because elections are not random, homogeneous, or repeatable). See Hoppe’s great article “The Limits of Numerical Probability”.
It is impossible to quantify risk because risk is subjective. The idea of boiling down risk to a single number is absurd.
@econ guy:
What you have quoted is but the basic assumptions behind elementary regression. There are other techniques to deal with situations where these assumptions do not hold true – in fact, not many assumptions need to be made if “non-parametric methods” are used (Rothbard actually alluded to this in one of his essays). The trade-off in using this superior class of techniques, is that it requires more computational power, and more data to achieve the same degree of accuracy.
When it comes to interpolation and extrapolation of data trends, you can’t avoid making assumptions anyway.
I don’t think people – even Austrians like Hayek – understand the philosophy behind modeling. It is fundamentally an epistemological question – i.e. given the data that we have, what can we deduce from it? Can we find out the underlying principles behind them, and can we predict the future from this limited set of data that we have?
Can we know the “truth”, so to speak, from our limited “knowledge”?
Mathematical modelling has its uses, however they cannot serve as a substitute for a sound theoretical framework. They are useful in suggesting “anomalies” or interesting points for further investigation, but they cannot be taken at face value – they must be interpreted within an appropriate theoretical framework.
If a model was constructed on the basis of a fallacious theory (e.g. Keynesian framework), then the model would be problematic of course.
There is a popular saying among statisticians – “All models are wrong, but some models are useful”. Whether a particular model is useful or not, and under what circumstances – that would have to be left to the acute judgment of statisticians, and the confirmation of reality.
It should also be noted that the veracity of the modelling assumptions isn’t always important – you can have “wrong” assumptions and still get a reasonably good model. So long as the assumptions are not too “wrong”, and that the model is “robust” against poor assumptions.
On the pricing of risk – well, the assumption that errors follow a normal distribution could be quite good i.m.o. under a sound monetary system. Give our fiat monetary regime – there’ll be wild swings, heavy volatility and manipulation of market signals… so the models break down. Thus that underlines the importance of market analysis.
One final note:
Hayek draws a distinction when it comes the natural sciences and in economics – that modelling is acceptable in the natural sciences but not when it comes to the field of econs. Believe me, as a physics MIT grad student – the same modelling/epistemological problems hold in the sciences as in economics.
My opinion (as of now) is that Von Mises’ theoretical framework furnishes us with a proper and sound understanding of basic economic principles, but mathematical modelling – if used deftly, and interpreted within the Austrian framework – can be of much merit.
How else can risk be priced anyway.
@ nick who said “I don’t think people – even Austrians like Hayek – understand the philosophy behind modeling. It is fundamentally an epistemological question – i.e. given the data that we have, what can we deduce from it? Can we find out the underlying principles behind them, and can we predict the future from this limited set of data that we have?”
I think the Austrian body of knowledge does capture models and modeling. Modeling is simply generalizing empirical data into relationships of cause and effect, but mostly just of correlation. It is indeed an epistemological question what we can do with this data. Mainstream Austrians contend that this empirical data is historical data that is accidental in the sense that it could have been otherwise. One cannot deduce anything from it without a prior theory in hand. In this way the empirical data/historical data can illustratie certain parts of the prior theory/theoretical framework.
The mainstream Austrian point would by the way not merely be that the modeling of human action is impossible, but that modeling by itself with empirical data in order to create/deduce sound theory is impossible. This latter point is much more fundamental (epistemologically) than the former and easy to miss.
I’m a quant on Wall Street, and I can tell you, except for a handful of shops (and a handful of asset classes), quants are mainly for show; at best they just quantify what is desired to be heard. These “blame the quants” articles that are cropping up are classic buck-passing; clearly lazy journalists are being fed crap by interested parties, and regurgitating it in print.
@Arend:
What I meant was that the same epistemologically problems pop up in the sciences – so I don’t entirely agree with Hayek’s dichotomy between the natural sciences and economics.
But I believe we’re fundamentally in agreement here.
Looking at mathematical models in Physics and Engineering, a lot of things are “simplified” and the assumptions are not necessarily true either.
Mathematical models at the end of the day are approximations to reality – they are thus “false” in that sense. But that is not to say they are not useful – they are still capable of providing good estimates and aid in decision making.
I agree that “modeling by itself with empirical data in order to create/deduce sound theory” is absurd, but they can still suggest interesting points of investigation. While I claim no familiarity with econometric models, I’m open to the idea that human action could be decently modelled (locally, in the short run, where economic conditions can reasonably be assumed to have little change) – not that they can yield precise quantitative estimates of course, but they could still be of some qualitative use.
My personal take is that a lot of current issues – creationism vs. evolution, global warming etc., comes down to the same epistemological problem too. Creationism assumes an apriori framework, where evolution is an empirical thing.
With evolution: people have some fragments of the past (fossils, astronomical observations, data etc), and try to string them all together into a coherent story (or rather, theory) – sort of like Sherlock Holmes trying to figure out “what happened?”.
Can we know that our guess as to “what happened” is an accurate, or at least acceptable description of the past? Maybe this is a question beyond the powers of human inquiry altogether – especially since “scientific evidence” can be can be interpreted in more than one way, and with non-verifiable assumptions embedded.
In regards to creationism – the question is whether their apriori framework is correct. However, you can believe that the Bible is the literal truth, and yet deduce a different apriori framework.
“I’m a quant on Wall Street, and I can tell you, except for a handful of shops (and a handful of asset classes), quants are mainly for show; at best they just quantify what is desired to be heard.”
Haha, most economists (especially in public/inter-governmental institutions) do the same too.
a.k.a.
“Don’t tell me what to do, tell me what I already want to do.”
Nick: “Creationism assumes an apriori framework, where evolution is an empirical thing.â€
In understand the point you are making about epistemology, but I think that if you investigate creationism you’ll find it relies very much on empiricism and that evolution has much more of an a priori nature to it, too. It’s not as clear cut as you suggest.
Nick: “In regards to creationism – the question is whether their apriori framework is correct.â€
That part is just a matter of logic and easy to check. But there is a very strong empirical part that evolutionists refuse to acknowledge. There are hundreds of scientists who present scientific evidence that evolution through natural selection is impossible on scientific grounds. Most evolutionists ignore these scientists, just as global warming proponents ignore the scientific opposition. Rather than debate the science, they merely snear at the opposition.
Nick: “However, you can believe that the Bible is the literal truth, and yet deduce a different apriori framework.â€
There is a science of interpretation called hermeneutics. It’s nothing more than a subset of logic applied to interpreting communications. If people follow the rules of hermeneutics, the range of possible interpretations of the Bible is narrowed considerably.
In defense of quants, we should remember that businessmen must attempt to predict the future. They have no choice in the matter. The question is whether experts or math models are better predictors and the research demonstrates without a doubt the simple math models outperform experts almost all of the time. A model based on bad theory, or no theory, will always perform badly and that’s the main problem with the models used in mainstream economics and finance. Theory, not modeling is the problem. A model based on the ABCT will outperform any other model, and it will outperform the experts, though it will not be perfect.
Also, I would like to see economists and finance people use data mining methods much more than traditional statistics. Data mining techniques, such as neural networks, have proven many times to be far more accurate than traditional statistics and the don’t require normally distributed variables or any of the other problems associated with regression.
Nick – “Hayek draws a distinction when it comes the natural sciences and in economics – that modelling is acceptable in the natural sciences but not when it comes to the field of econs. Believe me, as a physics MIT grad student – the same modelling/epistemological problems hold in the sciences as in economics.”
I am not familiar with Hayek on this, but the distinction both Mises and Rothbard draw is in degree and not in kind. However, the degree is so large that it might as well be kind. The following is my understanding of epistemology:
All synthetic knowledge falls under two categories, the a priori and the a posteriori. A statement is true, quite simply, if it is an accurate description of reality; and we can only determine truth/falsehood through sense perception. A priori, as I am using the term, then refers to that part of our knowledge that is apodictic, and a posteriori to that part of our knowledge that is empirical. Thus, a statement like “I am conscious” is a priori because it does not need to be empirically verified. However, and this is where Rothbard and Mises diverge from Kant, is that it is based on sense perception because I cannot determine that I am conscious without having first observing that I am, in fact, conscious. The law of gravity, on the other hand, is a posteriori because it is induced from specific observations of reality and it can be falsified.
Based on this understanding, we can see that all fields — economics, physics, biology, and chemistry — are built from a priori knowledge. The difference between these fields, then, is that in the natural sciences, our a priori deductive knowledge is only so powerful that eventually we must attempt other means of knowledge gathering. However, the foundation is still based on a priori truths. For example, the physicist is completely lost if he does not recognize the inviolability of logic — an a priori truth. With natural sciences, our a priori knowledge is sufficiently powerful that we can build an entire system of economics without having to rely on the (less desirable) a posteriori. However, it too has its limits and we may need to extend our understanding of these disciplines by admitting other means of knowledge gathering, as you suggest.
Conclusion: modeling is acceptable in the natural sciences because without it we can’t explain very much. In the social sciences modeling is less desirable and, in fact, near impossible because the assumptions that underlie it are far less applicable — can you really assume the regularity of human behavior?
@Nick: “What I meant was that the same epistemologically problems pop up in the sciences – so I don’t entirely agree with Hayek’s dichotomy between the natural sciences and economics.”
Epistemologically speaking, though, the natural sciences operate totally different than the classical sciences. In the natural sciences, the laws are INDUCED by the empirical data. In the classical sciences, theorems are DEDUCED from apparently self-evident axioms. These are two entirely different methods in the pursuit of “knowledge”, and that is the dichotomy.
I would grant that every science has its apriori assumptions. However, in the natural sciences, the presuppositions are the same, and it is that the methods of empirical science are the means to knowledge. In the classical sciences, each science has its own set of apriori axioms.
The Austrians treat economics as a classical science rooted in the axioms of praxeology. Its concepts are deduced from those axioms. The other branches of economics treat it as an empirical science and try to induce economic laws from models (which to me is a tautology, since the model requires a set of assumptions about the subject being studied) and historical data.
Therefore, the epistemological problems between natural and classical science are quite different. In the natural sciences, they have no means to prove by its methods, once and for all, that a given law is true–it is tentative. In the classical sciences, the question remains as to whether or not the “self-evident” axioms are correct. If they are, the conclusions are certain.
fundamentalist made a great point concerning ID vs. Evolution…
The natural science has in the philosophical framework defining its methods the assumption that the supernatural cannot exist and does not happen. Therefore, intelligent design is not a viable solution because it is supernatural. This is a great example on the epistemological flaws of empirical science.
Intelligent design (different from creationism) is an empirical approach towards the issue that alters the assumption of what is supernatural. Therefore an intelligent being is a viable answer to the origins of life.
However, it must stop at this point. To move beyond this conclusion: (eg. that it is the omnipotent, omniscient, Christian God when it could be multiple, very smart, very powerful aliens) moves the discussion from the physical to the metaphysical, which “empirical science” cannot quantify.
Theology and hermeneutics are classical sciences as well, axiomatic in nature. One may disagree with the first principles, but they cannot deny the strict use of logic by many theologians. (Many other…ehhh.) In fact, Reformed seminaries require training in logic.
There are no constants when dealing with human action other than “time passes” and “man acts”, only variables. That is why Mises said econometrics eludicates nothing on human action.
As for picking stocks, models may make you feel better or prove to be a useful marketing tool to prosepective customers. In the end, the value of the firm and it’s stock price is determined how market participants act.
I think stock pickers would be better served by studying Thymology as opposed to econometric modeling.
Nick,
Your criticism of my post is twofold:
1. There are alternatives to “elementary regression”, so the critique is irrelevant.
2. Despite the violation of the “basic assumptions”, the use of “elementary regression” is justified if it leads to good predictions.
“What you have quoted is but the basic assumptions behind elementary regression.”
The status of the assumptions (basic or not) is irrelevant and does not invalidate the criticism. If the basic assumptions don’t hold, then the use of the technique is unjustified. The use of alternative techniques sidesteps the critique. The use of alternatives is irrelevant if nobody employs the alternative techniques. The fact is that “elementary regression” is used pervasively. Like I said before, regression analysis is not proper when analyzing human action.
“the veracity of the modelling assumptions isn’t always important – you can have “wrong” assumptions and still get a reasonably good model.”
This is the flawed view of Milton Friedman. Austrians are very critical of this stance for good reason (See “Making Sense of Economic Indicators” by Shostak).
“Believe me, as a physics MIT grad student – the same modelling/epistemological problems hold in the sciences as in economics.”
You are incorrect. You have expressed the idea of Scientism: “Scientism is the profoundly unscientific attempt to transfer uncritically the methodology of the physical sciences to the study of human action.” The same modeling/epistemological problems DO NOT hold in the sciences as in economics. Why? Because of human action. I recommend you read “The Mantle of Science” by Murray Rothbard.
http://mises.org/rothbard/mantle.pdf
“There are hundreds of scientists who present scientific evidence that evolution through natural selection is impossible on scientific grounds.”
Bullshit.
@Brent Railey:
“In the natural sciences, the laws are INDUCED by the empirical data. In the classical sciences, theorems are DEDUCED from apparently self-evident axioms.”
The way I think of it is that: the “axioms” in the natural sciences is induced by empirical data. The consequences of these empirically-motivated axioms are then deduced, yielding “theorems”. If further empirical data conflicts with these “theorems”, “axioms” are tinkered accordingly.
Contrast this with the Austrian framework – where (supposedly) self-evident truths about human action are taken to be “axioms”, and theorems deduced from them as you pointed out.
So the two philosophical frameworks are quite similar i.m.o. In both cases, empiricism can suggest inadequacies in our apriori framework, or gaps in our understanding.
Also, when it comes to questions in regards to the origin of life and universe, you can’t really divorce them from the metaphysical. I doubt science alone will ever provide a satisfactory explanation. Perhaps these are really historical questions – unless you were there to witness it, at best you can only come up with “clever guesses”.
@Econ Guy:
I have sufficiently addressed your objections in my previous posts. Even in Physics and Engineering, models are vast “simplifications” of reality with “false” assumptions embedded – yet it doesn’t detract from their usefulness.
The point about modelling is not to construct an accurate description of reality, but one which is “useful” so to speak.
Fundamentalist sums up my view more eloquently that I could have done. I suggest you pick up an advanced Statistics text.
“You are incorrect. You have expressed the idea of Scientism: “Scientism is the profoundly unscientific attempt to transfer uncritically the methodology of the physical sciences to the study of human action.” The same modeling/epistemological problems DO NOT hold in the sciences as in economics. Why? Because of human action. I recommend you read “The Mantle of Science” by Murray Rothbard.”
The key word of course is “uncritically”. Any worthy applied statistician would be highly critical of their models, wary of the assumptions made and the way in which results are to be interpreted.
Some of Rothbard’s criticisms are valid, but he writes as someone without extensive experience in modelling. Otherwise he would have had difficult stepping into any building – whose construction is partly based on estimates derived from “false” equations and “false” models.
Nick,
“Stones, molecules, planets cannot choose their courses; their behavior is strictly and mechanically determined for them. Only human beings possess free will and consciousness: for they are conscious, and they can, and indeed must, choose their course of action. To ignore this primordial fact about the nature of man—to ignore his volition, his free will—is to misconstrue the facts of reality and therefore to be profoundly and radically unscientific” (“The Mantle of Science”, Rothbard).
I don’t object to modeling in the natural sciences. The statistical techniques found in the textbooks are appropriate when applied to the physical sciences. This is because “stones, molecules, and planets” do not act purposefully.
But applying these same techniques to human beings is an entirely different matter. Human beings act and are capable of changing their actions as new information becomes available. The existence of human action (purposeful behavior) makes the use of many statistical techniques found in your beloved advanced statistics textbook inappropriate and useless when applied to the social sciences. The social sciences are different than the natural sciences because of human action.
Interestingly, Hoppe uses a regression equation to make my point (a self referential argument) in the lecture below. I highly recommend this to you.
http://media.mises.org/mp3/MU2005/mu05-Hoppe.mp3
p.s. Murray Rothbard was a mathematics major at Columbia. He knew what he was talking about.
@ Econ Guy:
I’ve heard of the story about Rothbard quitting stats, because much of stats assumed the normal distribution one way or another – and he (rightfully) believed this to be a flawed assumption. But that’s only elementary statistics – the field of statistics has advanced a lot since Rothbard’s time.
Without modelling “human action”, how can entrepreneurs predict the future, calculate risk and price things accordingly?
Any such attempts at predicting the future would have to involve unfounded assumptions anyway.
Just because we don’t have complete knowledge as to what is going on in people’s minds, their (constantly-changing) preferences, and the manner in which they are likely to exercise their free-will – that doesn’t prevent us from coming up with “decent” projections of the future.
Aside, I’m always startled at the vast homogeneity in our “liberal society” which purportedly cherishes “individuality”, heh.
nick: “…the field of statistics has advanced a lot since Rothbard’s time.”
That’s true and one of the things we have learned is that regression results are not affected by lack of normality in the distribution as once thought. Regression is very robust to large deviations from normality. But if you’re worried about it, there are transformations available that can convert a non-normal distribution to something very close to a normal distribution before performing the regression.
In addition, most data mining techniques are not dependent on any distribution and they’re more accurate than regression. Black Swan Taleb has been one of the big promoters of the idea that real world data isn’t normally distributed and that makes all statistics invalid. It just ain’t so.
If anyone is interested in data mining, I highly recommend the free version of Rapid Miner available at rapid-i.com. It has many algorithms for analyzing different types of data, but the support vector machine is most impressive.
@ fundamentalist:
“That’s true and one of the things we have learned is that regression results are not affected by lack of normality in the distribution as once thought. Regression is very robust to large deviations from normality. But if you’re worried about it, there are transformations available that can convert a non-normal distribution to something very close to a normal distribution before performing the regression.”
Most statisticians view transformation methods with skepticism, actually. Non-parametric methods (which incorporate little, or zero, assumptions) are far superior – e.g. using kernels, splines, etc. The massive increase in computing power has rendered a lot of things feasible.
I’ve read a few of Taleb’s popular books. Most of his criticisms are against the manner in which statistics is interpreted and applied by those in public policy and financial institutions, not the usage of statistics per se. He raises some issues about mathematical modelling, but most of them have already been dealt with, and some others are unlikely to be ever resolved. I say this because humans are but mortal creatures with limited capabilities, subjected to the constraints of the universe – and the entire aim of statistics is to “discover the truth”/”predict the future” from the “limited knowledge” that we have. We can only make “clever deductions”.
So frankly, Taleb presents no original insights – but it’s good to be reminded of them. That’s how I read the criticisms of the Austrians too – that mathematical modelling should be guided by the correct theory; models are not truth per se, and are very limited in their application; and watch out for pitfalls. Economics should perhaps be a discipline fundamentally based on verbal logic/Aristotelian syllogism; but mathematical modelling is a legitimate activity which has its place in this world – however limited models may be.
Econ Guy wrote:
“There are three conditions that are required to perform a regression analysis. First, the error variable must be normally distributed. Second, the error variable must have a constant variance. Third, the errors must be independent of each other. None of these conditions are satisfied when we are dealing with human action. Regression analysis is therefore not proper when analyzing human action.”
That is a vast oversimplification. The items above are assumptions for optimality, specifically for least-squares regression. Most risk models, however, are models of probability, with 0/1 outcomes. Human behavior may often be characterized very well this way, such as whether the human customer pays back a loan (0) or not (1). How well estimates of the probability of such things work is an empirical question, not a theoretical one, and there are many examples of this being done in practice, well enough to generate tremendous economic value.
Comments on this entry are closed.