James Sheehan writes that randomness influences all types of human action and helpfully exposes the futility of macroeconomics and econometrics, to say nothing of the attempt by government to plan. This is demonstrated in Nassim Taleb 2001 book that has been read by every major hedge fund manager. He speaks from experience but his conclusions, if not his method at arriving at them, are consistent with what Austrians have said for 50 years. FULL ARTICLE
Source link: http://archive.mises.org/4727/fools-put-faith-in-data-alone/
Fools Put Faith in Data Alone
Previous post: How To Make A Mountain Out Of A Mole Hill
Next post: J.B. Say: An Appreciation
{ 20 comments }
On the other hand, a paper by Michael A. Bishop and J. D. Trout
Iowa State University and Loyola University, show that simple statistical techniques can dramatically improve the predictions of professionals in a variety of fields. Here’s the intro:
Our aim in this paper is to bring the woefully neglected literature on predictive modeling to bear on some central questions in the philosophy of science. The lesson of this literature is straightforward: For a very wide range of prediction problems, statistical prediction rules (SPRs), often rules that are very easy to implement, make predictions that are as reliable as, and typically more reliable than, human experts. We
will argue that the success of SPRs forces us to reconsider our views about what is involved in understanding, explanation, good reasoning, and about how we ought to do philosophy of science.
The paper is 50 Years of Successful Predictive Modeling Should Be Enough: Lessons for Philosophy of Science. You’ll have to google for it because the link is too long. The gist is that experts become arrogant, which damages their judgment. Simple statistical models usually outperform the experts. Should we follow statistical models like slaves? No. They have to be based on sound theory. But when used properly, they can help people look at issues more objectively, eliminate arrogance and bias, and assist us when trying to discern between two equally well argued and logical theories.
If Austrian economics is true, and I believe it is, there ought to exist some statistical evidence for it. Would a small amount of statistics be a stake through the Austrian heart? I doubt it. Most articles on this web site feature graphs with two line plots and the author asks the readers to make eyeball correlations between the two. What is that but a poor man’s regression? Why not just do the regression for us and let us see the statistical correlation?
really enjoyed this book and would also recommend “Fortune’s Formula” which also discusses randomness but adds another layer with the discussion of “overbetting” and the Kelly Criterion—
has a good discussion of LTCM — but discusses the failure as not only as an underestimation of risk, but overbetting — that LTCM’s strategy is a 3% to 5% yoy strategy — that was leveraged to extreme– an unappreciation for the Kelly Criterion
Roger M.,
The reality is, most statistical predictions have been dead wrong, as have most economists. Irving Fischer predicted everlasting prosperity on the eve of the great depression. Mainstream economists predicted gold prices would plummet after it was severed from the dollar. Very few mainstream — that is, statistical, econometric — economists predicted the last bust; more than 80% of those with published records predicting the bust were Austrians, although of course that’s entrepreneurship, not pure economics (but shouldn’t it tell you something that entrepreneurs using the Austrian framework of economic understanding are more successful?). Paul Samuelson thought the USSR would overtake the US economically, while Austrians realized this couldn’t happen as long as it was communist, and understood such a society to be a disaster.
I would also mention the flaws of statistical methodologies as used. There is nothing stating that simply because some (possibly coincidental) relationship held in the past, it must always hold in the future; on the other hand, praxeological laws (ceteris paribus statements) hold whenever describing the behaviour of man. There are also numerous data-mining problems with these statistical approaches (see the “Motley Fool” approach, I think called Foolish 8, which is indeed nothing but a bunch of data-mining nonsense).
That was a good read.
Quote:
“As opposed to a Utopian Vision, in which human beings are rational and perfectible (by state action), Taleb adopts what he calls a Tragic Vision: “We are faulty and there is no need to bother trying to correct our flaws.” It is refreshing to see a highly successful practitioner of statistics and finance adopt a contrarian viewpoint towards economics.”
I would say that we are not faulty but wish to have some randomness in our lives so introduce what are apparently perceived as flaws. Perfectibility in prediction would eliminate all risk and risk, the challenge of win-lose is the reason d’etre for investment. A sure thing means end-of-game and must therefore never be achieved.
To Roger M.
Statistics can only go so far and cannot include unknowns. As Mr. Sheehan points out, it is a mistake to use statistics without logic but the reverse is not true.
I believe that there is a type of psychological Heisenberg Uncertainty Principle when dealing with human beings. As soon you correctly model past behavior and try to apply your new knowledge, enough people (and it likely doesn’t require a large number) become aware of your model, the behavior changes and renders your model useless. I like to play poker, and I find this to be true of any table- as soon as my opponents become aware that I have figured out their holdings based on their betting patterns, they change it up on me, and vice versa.
A model may be good if you are the only one aware of it.
Statistical predictions are always wrong, it’s true. If they’re dead on accurate, that’s usually by chance. But absolute accuracy shouldn’t be the issue. The issue should be whether experts unaided by statistics are more accurate predictors than those who use statistics. The research proves the latter are more accurate. Had Keynes not showed so much disdain for statistics, he likely would have modified his theories and paid more attention to Mises. Many of the examples that Heinrich offers above actually make my point because they didn’t use statistical models to make their predictions, but instead relied upon their biases and ego. A lot of the math used by people like Samuelson isn’t statistics, but calculus.
Roger,
Statistics doesn’t eliminate arbitrary bias. It merely formalizes some things to allow us to obtain a clearer picture on some information. For example, consider using a Bayesian Monte-Carlo simulation to figure out various what-if scenarios; or using historical simulation, from historical statistics. This is all fine and well, but we still exercise our own judgement. We decide if past data is relevant to future decisions, to what extent the future will be like the past, and so-on and so-forth.
Creating statistical models to make predictions is merely quantifying your subjective judgements. Period. It is perfectly valid to do this in entrepreneurial function, but it is an art, not a science. It requires judgement about the applicability, usefulness, appropriateness, of the results of a certain statistical techniques to the decision that you have to make.
What statistics really does is tell us about the past. And sometimes, if we’re not careful, it doesn’t even do that. There are all kinds of problems involved with merely the gathering of data: namely, selection bias (e.g., survivorship bias), data mining problems (running various variables through the data again and again until you come up with specious and arbitrary relationships, like butter production in S. Africa having something to do with US economic conditions), and so-on and so-forth. You also need to make a judgement about the applicability of a particular statistical technique to your situation (e.g., a normal distribution; or lognormal; or skewed distribution, or distribution with some kurtosis).
That’s the first problem: making sure you get your analysis of the past right, and don’t draw out arbitrary relations with no real meaningfulness. If you find some correlation that has no praxeological basis, or no basis that could be interpretted in a praxeological understanding with respect to human action, then your correlation is spurious. In short, statistics doesn’t tell us about economics, praxeological reasoning does.
The second problem is judging the usefulness of your new understanding about the past to future endeavors. In short, you have to judge, how relevant is the past to right now? Just because X happened in the past does not mean it’s going to happen again. You have to judge if conditions are sufficiently similar to make the past data meaningful.
David,
Everything you’ve said is true, but it doesn’t change the fact the predictions made with the aid of statistics are far more accurate that those without. If you google the article I mentioned above and read it, I think you’ll agree.
Roger,
This may very well be true. However, it may also be that over-confidence in statistics causes people to make spurreous errors, because of the various mistakes I’ve mentioned above. What this says is that there are tools do not eliminate the subjective judgemental element of action; and that tools must be properly used — their limits, uses, and flaws must be clearly understood by the user, otherwise, it’s like using a hammer to sweep a floor.
I think statistics bears the blame for a lot of faulty theorizing. It’s often said that you can prove anything with statistics. That’s not true. Faulty theory and abuse of statistical methods produce bad results which people then blame on the statistical technique. Your statistics are only as good as the theory supporting them.
As I wrote earlier, most writers on this web site use econometric reasoning in their articles. Any time they write that one thing is caused by another, they have specified a linear regression problem. They even use graphs, which hide the numbers. They just stop short of putting those same numbers into an equation. For example, a lot of writers show graphs of the money supply versus some measure of inflation and ask readers to eyeball the correlation. That is nothing less than an econometric model. By putting the very same numbers in the graph in a simple linear regression, we might discover that the independent variable explained 52% of the variation in the response variable. Since Austrians use graphs with numbers behind them, why all the fuss about using those same number in a regression equation that attaches a number to the correlation we can see with our eyes?
Roger M,
I’d suggest reading Mises’ Social Sciences and Natural Sciences. The problem is not with finding correlations in the past, although for the purposes of these articles, authors obviously judge it not worthwhile; furthermore, as my boss (a biologist) used to say, if you need to use statistics to determine if you have a statistically significant correlation, you don’t. Again, once we get these correlations, we need to deal with all the problems I mentioned. We need to think about if the time-frame was appropriate, and if past patterns will hold in the future (or if that is even relevant for the article’s purposes).
Also, I think you have causation and correlation confused. That A and B are correlated does not prove there is any causal relation. It may just be a spurious correlation, the result of random chance. This is certainly likely if there are data-problems (data-mining, selection bias, time-bias). And corrleations which don’t make any kind of economic sense should be strongly questioned as being real, and not the result of various chance artifacts. That A and B are correlated does not prove that A causes B; or the B causes A; or that some other thing C, causes both A and B, thus the correlation of A and B. All correlation between two things tells us is that there may be a causal relationship between them, or that some third thing may be causing both. It can never, in the social sciences, tell us anything about causation.
Causation can only be determined using praxeology. And appealing to something like granger-causality is wrong-headed, as granger-causality is really just “temporal relationship”, and to assert that this shows causation is nothing more than a mathematically sophisticated post hoc ergo propter hoc.
I’m so far down intellectually than whats been posted so far, I read all the above and I’m more confused than ever. In the few years I have been trying to learn about Free Markets and investing the best advice I ever read or heard as far as deciding whether to invest in the stock market (were talking in terms of Indexing) was this, and it’s not verbatim…was, “if you believe that our Free Markets ( don’t tweak here you Austrians, think of “Free Markets” loosely) can foster economic growth for the time frame you want to invest then you might consider Indexing. That was my statistic and I’ve been indexing for 15 years (Total Stock Market) and all is well. I tried reading ” Fooled by Randomness” and just couldn’t see how I would use what was being said for anything. But I’m gonna try a von Mises book…can anybody suggest one to start with? Feel free to suggest to my email. Thanks!
Perry,
A surfer doesn’t need to understand the scientific dynamics of currents and waves, just a feel for when a good wave is likely to form is adequate..
This also holds true for most things in life.
For stocks and other traded instruments I find http://www.seykota.com/faq and http://www.turtletrader.com excellent resources, and they are free..
Yogi,
I disagree. Take into account your prevailing wind speed, the surface tension of the water at a given temperature, the amplitude and frequency of the waves as a function of time,……etc. Now take the third derivative of the function that describes the numbers of sharks about to bite off you leg…
David,
Everything you’ve said is true. To repeat, sound theory must come first. But it doesn’t change the fact that simple statistical techniques, such as linear regression, produce far better predictions than simply relying on the opinion of experts. Read the article.
Perry,
Have you checked out Ray Lucia’s books on investing? They’re great!
Roger M,
Well, I’d refute the dichotomy as completely arbitrary. Simple statistical techniques for various things are the opinion of experts.
*sigh*… my favorite blog is calling me a fool
Is there someonoe out there who can give me a source on Lenin’s economic policy. My students have come to the conclusion that Stalin’s methods were necessary to bring Russia into the modern age-that the NEP did not work i.e., Capitalist incentives.I remember a book on foreign investments during the Lenin era. In other words It was capitalists that got Russia up and running esp. in oil. ANd of course we know from Pipes work that during Stalin’s era he allowed the German’s to circumvent the Versailles treaty and develop weapons on Russian soil in exchange for technical assistance.
Come on. Do you want the best and brightest at Stuyvesant h.s. in NYC-future leaders of the world to be a bunch of statists? I need intellectual ammunition and I know it is out there. But plugging in Lenin gets nothing on your search engines
Kit Olivi,
This might be a good place to start:
“Master Trader” misses trend…..
A jury last week hit Ed Seykota for big loss he apparently didn’t see coming, completely missing the trend and the chance to cut losses before hitting bottom. Following his BIG loss to IRS in the 90′s, he is batting 1000 for being unable to read trends against him.
Comments on this entry are closed.