Tuesday, February 03, 2009

Power laws, baby

Thanks to reader GS for the link to this excellent article by statistical physicist Eugene Stanley. There's much more to the article, but I decided to excerpt this discussion of security price fluctuations and the observation that they are far from log normal.

Stanley: ...

When they analyzed these data–200 million of them–in exactly the same fashion that Bachelier had analyzed data almost a century earlier, they made a startling discovery. The pdf of price changes was not Gaussian plus outliers, as previously believed. Rather, all the data–including data previously termed outliers–conformed to a single pdf encompassing both everyday fluctuations and “once in a century” fluctuations. Instead of a Gaussian or some correction to a Gaussian, they found a power law pdf with exponent -4, a sufficiently large exponent that the difference from a Gaussian is not huge; however, the probability of a “once in a century” event of, say, 100 standard deviations is exp(-10,000) for the Gaussian, but simply 10-8 for an inverse quartic law. If one analyzes a data set containing 200 million data in two years, this means there are only two such events–in two years!

Now which is better, the concept of “everyday fluctuations” which can be modeled with a drunkard’s walk, complemented by a few “once in a century” outliers? Or a single empirical law with no outliers but for which a complete theory does not exist despite promising progress by Xavier Gabaix of NYU’s Stern School of Management and his collaborators? Here we come to one of the most salient differences between traditional economics and the econophysicists: economists are hesitant to put much stock in laws that have no coherent and complete theory supporting them, while physicists cannot afford this reluctance. There are so many phenomena we do not understand. Indeed, many physics “laws” have proved useful long before any theoretical underpinning was developed . . . Newton’s laws and Coulomb’s law to name but two.

And all of us are loathe to accept even a well-documented empirical law that seems to go against our own everyday experience. For stock price fluctuations, we all experience calm periods of everyday fluctuations, punctuated by highly volatile periods that seem to cluster. So we would expect the pdf of stock price fluctuations to be bimodal, with a broad maximum centered around, say, 1-3 standard deviations and then a narrow peak centered around, say, 50 standard deviations. And it is easy to show that if we do not have access to “all the data” but instead sample only a small fraction of the 200 million data recently analyzed, then this everyday experience is perfectly correct, since the rare events are indeed rare and we barely recall those that are “large but not that large”.

The same is true for earthquakes: our everyday experience teaches us that small quakes are going on all the time but are barely noticeable except by those who work at seismic detection stations. And every so often occurs a “once in a century” truly horrific event, such as the famous San Francisco earthquake. Yet when seismic stations analyze all the data, they find not the bimodal distribution of everyday experience but rather a power law, the Gutenberg-Richter law, describing the number of earthquakes of a given magnitude.


Anonymous said...

"So we would expect the pdf of stock price fluctuations to be bimodal"

This is incorrect. What "we would expect" is one distribution with ever changing parameters.

Distributions over short periods "we would expect" to be inconsistent with distributions over long periods.

This guy doesn't know what he's taking about.

Nicolas said...

I remember reading a good paper explaining the frequent occurence of power laws in many context. can't find it thoug..

I once tried to detect outliers to clean historic data, and the data that don't fit gaussian returns are overwhelming.
Even filtering for 0.0001 quartile turns you over lots of valid data points.

There are many assumption that are completely wrong in finance, not only statistically, but also economically.

The complete lack of use uncertainty for estimates is from my perspective a bigger problem.

Complex models needs to be calibrated (over stationary distribution..)

stochastix said...

I really don't want to "insult" any of the experts, but why is that people care so much about the probability distributions of stock prices?

As every physicist knows, Einstein found it hard to grasp the randomness in Quantum Mechanics. If I remember it right, Einstein considered a probabilistic approach an exercise of "defeatism". In finance, people seem to be quite defeatist. A great example is the fantasy of gaussian fluctuations. Then along came Mandelbrot, Taleb and their homies with new fantasies such as power laws.

IMHO, going for random walks, power laws and the like is a simplistic approach. There are buying and selling orders being issued at various prices, all the time. A "good model" of stock price fluctuations should account for the structure of the process that leads to the formation of the prices. I rarely see academics working on high-quality order-book data, so I am not too surprised that finance has been (and shall always be) a cemetery of theories. In Physics, no one would be able to go for so long with such crude experimental data, but in Finance it seems that anything goes...

Anonymous said...

"I rarely see academics working on high-quality order-book data"

This would apply only to "high-frequency finance".

tc said...

As someone looking at it from the econ side, I kind of agree that fancier distributions won't tell us about the mechanisms of these tail events - and the econophysicists themselves are part of the financial industry, therefore part of the system as a whole. All those mortgage borrowers taking on more debt than they could afford were a tail event - but they, in turn, were affected by the availability of cheap credit, made possible by Wall Street. I'd like to ask an econophysicist one day - can they model the effect that a newly developed formula for pricing mortgages has, on the mortgage borrowers themselves?

Anonymous said...

Soros has made the same comment. Whatever the theory, its application will invalidate it.

Nicolas said...

"I really don't want to "insult" any of the experts, but why is that people care so much about the probability distributions of stock prices?"

From a practicioner pov, there are a few reasons. obviously, if you know the ditribution, you can make money, but that is not the main reason as this goal is clearly illusionary.
More importantly, with a credible distribution, you can justify making a business by assessing its "risk" whatever that means. note here that the credibility is more important than the reality.

"I rarely see academics working on high-quality order-book data"

The high quality data are not accessible to academics.
It requires a lot of means, an army of statisticians just to clean and maintain data.
Those means are out of reach to academics. Also if you know something in that area.. why be an academic ? It's not really universal knowledge as real science can be.

PS: awesome blog you have !

Anonymous said...

"More importantly, with a credible distribution, you can justify making a business by assessing its "risk" whatever that means."

The more risk can be quantified the more underwriting volume increases. The sell-side and the buy-side know their risks and can hedge. The more underwriting volume increases the faster econmoic growth.

Av said...

I have a very superficial knowledge about things like this - but the following might be relevant in the context of power laws in empirical data:



Anonymous said...


Do you know of a better deterministic theory which could replace the old quantum theory of the Bohr-Sommerfeld quantization program, which does NOT involve probability?

Anonymous said...

Correct me if I'm wrong, but aren't the eigenvalues of the SE easily represented as the minima of an infinite dimensional polynomial?

In that case, the trajetory of a marble (the state of the sytem) rolling from one minimum to another would be incomputable giving the illusion of random transitions.

A (very) little knowledge is a dangerous thing.

Blog Archive