Diary of a Financier

Bookshelf Updates: Efficient Markets v. Determinism Theories

In Bookshelf on Fri 8 Jul 2011 at 12:20
  • Comparison of EMH v. Deterministic Chaos Theories from recent books.
  • The world’s financial institutions are built upon the fallacy of EMH.
  • Reality says normal distributions are wrong model, but the applied mathematics for a more appropriate kurtosis curve (“fat tails”) are impractical. The EMH school shouldn’t carry on in convenient disregard for this.
  • Markets are a body of a fractional (“fractal”) dimension, more than 2-D, but less than 3-D. This means that their price movements are somewhat deterministic and explained by more than two, less than three, interdependent variables.
  • Liquid markets also show a long-memory of initial conditions, which expires beyond the threshold of each markets’ cycle length.
  • Wide lens & monthly charts filter out noise.
  • Fractal mathematics accounts for the qualitative & quantitative, recognizing that life is messy.

I just finished two books that I had to combine in the same review: My Life As a Quant by Emanuel Derman plus Chaos and Order in the Capital Markets by Edgar Peters. They’re such perfect iterations of each pole in the spectrum of modern capital markets creed. Accordingly, I thought it appropriate to juxtapose them herein. The classical, efficient markets camp against the modern, deterministic camp. In the end, I naturally side with Mr. Peters’ bid for determinism, although I approached Mr. Derman’s text with optimism. I thought Mr. Derman, a former physicist, might apply some academic accountability for confounds in his quantitative modeling. Like the brilliant minds at Long Term Capital Management, he showed me little recognition (in practice) for the limitations of mathematics applied to the physical world.

To be certain, Mr. Derman does mention the limitations of his models, but he and his fellow quants apply models as gospel nonetheless. In his professional autobiography, he almost accidentally incriminates the fallacy of classical models, into which I now lump quantitative analysis:

If you own 100 shares of Microsoft… there is no uncertainty about their value [price], only the risk that their value will change in the next instant. When you own an exotic, illiquid option, uncertainty precedes its risk because you don’t know if your model is right or wrong. More accurately, you know that model is both naive and wrong, but the only question is, how naive and how wrong?

First, I sincerely repudiate the Efficient Market Hypothesis that Mr. Derman so nonchalantly passes as factual. It still seems silly to employ a model that so imperfectly and inconsistently fits reality. By definition, that’s just what a model is, I suppose. But, the Efficient Market Hypothesis, the Random Walk, the Capital Asset Pricing Model, Gaussian, and Black-Scholes crowds hardly deliver us half-way to reality. I can’t settle for that as my own best-effort.

After the 2008 crisis, “fat tails” are finally a mainstream market phenomenon. Mr. Derman lingered on the topic of the option pricing “smile” conundrum from his tenure at Goldman Sachs. I prefer Edgar Peters’ treatment of the matter in his own text, wherein he acknowledges that frequency distribution models (like all CAPM constituents named above) would be far better served to operate on a non-normal distribution, or “kurtosis” curve. Yet, the applied mathematics are more convenient under the more imperfect Gaussian assumption, so economists and analysts ignore the shortcomings of their shortcuts. Imagine the entire financial industry built on such a porous foundation!? (Hence floods are commonplace.)

It has always frustrated me that pretty much all of financial planning and portfolio construction relies on the Efficient Frontier, et al. From pension funds to little old ladies, from investment banks to FINRA, everyone’s money is managed in accordance to EMH. I can attest to the convenience, but I can also tell a hundred stories about those who crashed & burned due to the occurrence of one-in-ten-billion probability events. For many reasons, it’s a spectacular advantage to be among a minority who recognize the natural properties of capital markets while the masses still worship the random walk like it’s some monotheistic animism.

I have to record a synopsis of Mr. Peter’s central theories, because they’re quite fascinating.

First, he describes the world according to traditional Euclidean geometry, wherein everything falls neatly into dimensional categories (1-Dimensional, 2-D, 3-D). What happens when you take a piece of paper (so thin to be characterized as a 2-D object) and crumple it up? Does it become a 3-D object? Ostensibly, it does. However, a close analysis would conclude that the crumpled ball of paper is not a simplistic 3-D object:

If you were to detach the two-dimensional sheet from the book and crumble it into a ball, the ball of paper would no longer be two-dimensional, but it would not exactly be three-dimensional either. It would have creases; its dimension would be less than three. The tighter the ball got crumpled, the closer it would get to becoming three-dimensional, or solid…

The crumpled ball has a fractional, or “fractal” dimension. It is non-integer…

A true three-dimesional object is solid; that is, the object has no holes or gaps in its surface… Most real objects are not solid in the classical, Euclidean sense; they have gaps and spaces. They merely reside in three-dimensional space.

If the paper were neatly two or even three-dimensional, you could replicate it using a simple mathematical formula. In the context of capital markets, you could extrapolate-out a pricetrend using geometry/trigonometry/calculus if it were characterized by such a clean, Euclidean dimension. But because the paper is something in between (“fractal”), the formula is too dense and sensitive to derive.

Via mathematical renderings, Mr. Peters determines the fractal dimension of multiple capital markets pricetrends. For example, the S&P 500 pricetrend rears a fractal dimension of 2.33 according to his study. Think of the S&P 500 plotted–a line chart more appropriate than candlesticks, et al. The fractal dimension of 2.33 means that the S&P 500 (SPX) resembles a bit more than a 2-D figure, residing in a 3-D space–kind of like a loosely crumpled sheet of paper. In other words, the SPX chart has the X & Y axes to which we’re accustomed (time & price), but there’s another variable at play that’s scarcely noticeable: the Z-axis/3rd dimension. So, movements in SPX can be explained by a bit more than one independent variable. Obviously, time (X-axis) is one of these independent variables; Mr. Peters asserts that the fractional variable remaining (Z-axis) is not known. The lower the fractal dimension, the better a model can attempt to describe the function, since it’s less complex with less influence asserted by the fractal variable. In the end, ‘the movement of market prices are highly random with a trend component.’

The Efficient Market Hypothesis school (along with the Random Walk & CAPM departments) chooses to ignore the dimensionality of securities. Dimensionality means rhyme, reason, determinism. Dimensionality means investors can–to a certain extent–determine priceaction from a function of limited variables. A line is one-dimensional, explained by no independent variables (although embedded in a two-dimensional space). A line will look the same no matter where you focus. There’s no doubt where a line will look like at any place in space, because lines are linear. They’re not affected by any variables like time. Nature, people, markets do not follow linear paths.

In contrast, a triangle is 2-D, explained by one independent variable. It would be nice if markets followed such a simple model, with priceaction merely a function of one variable, like time. Unfortunately, the actual market’s priceaction incorporates a fractal explanatory variable in its function. While the fractal dimension makes the short-term movements of the market complex, a discernible trend exists in the curve at-large. (More to come on this topic too.) Nevertheless, EMH classifies the market as non-dimensional, meaning day to day, week to week, month to month price movements are independent data, plain points in space. This is the edifice upon which our financial credo has been built.

Here is where I can swoop in with my two most important, most facile takeaways from Mr. Peters:

  1. Markets show extraordinary sensitivity to initial conditions- Like the Soros theory of reflexivity, initial conditions predicate a self-reinforcing trajectory of a pricetrend. As time progresses beyond a decipherable threshold (the series’ “natural period” or “cycle length”), that trajectory is inevitably compromised since the series’ memory of initial conditions erodes. Mr. Peters mathematically illustrates this by successfully testing for significant interdependence in multiple, rolling time series within each of many capital markets. This strength of interdependence wanes the more time progresses for each data set. (4 years are usually the cycle length for most liquid markets like SPX.) Further, he scrambles the data for each market in these tests and analyzes it for the same interdependence, but irrespective of chronological time. This scrambling renders a shockingly insignificant result for every data sets’ level of interdependence, and it confirms the role of both time & price as a variable influencing pricetrends. This would suggest that an S&P 500 with a fractional dimension of 2.33 doesn’t have one dependent variable (price) and 1+ independent (time & maybe interest rates), but rather all its variables are reflexively interdependent.
  2. Wide lens charts & monthly fractals filter out noise- Of all the time series fractals a chartist can analyze, monthly charts show the most deterministic trends because they filter-out the noise of outliers. Mr. Peters conducted his dependence test on data sets of quarterly, monthly, weekly, daily & hourly market prices. The monthly data were most significant due to the low noise of [outlier] traders. Further, his study suggest viewing such a monthly chart from the widest lense possible. Like a shoreline from an airplane looks a lot smoother than from the ground, chart fractals are much less noisy from the 10,000 foot view.

As far as I’m concerned, Mr. Peters’ findings in Chaos and Order in the Capital Markets are a [quiet] vindication of behavioral finance, to wit technical analysis. Summarily, he concludes that while markets visually and mathematically exhibit non-linearity, their movements can be described as both interdependent (self-reinforcing) and deterministic. He concludes the case well himself:

Why do these models fail? They simplify reality by assuming random behavior, and they ignore the influence of time on decision making. By assuming randomness, the problem is simplified and made “neat”; the models can be optimized for a single, optimal solution. Using random walk, we can find “optimal portfolios,” “intrinsic value,” and “fair price.”

Fractal analysis makes the mathematics more complicated for the modeler, but it brings the result closer to those experienced by practitioners. Fractal structure in the capital markets gives us cycles, trends, and many possible “fair values.” It returns the qualities that make the capital markets interesting, by returning the qualitative aspects that come from human decision making, and giving them measurable, quantitative attributes. Fractal statistics recognizes that life is messy and complex. There are many possibilities.


  1. […] a review of the ideas analyzed by Edgar Peters in Chaos and Order in the Capital Markets, which I discussed back in July: Of all the time series fractals a chartist can analyze, monthly charts show the most […]

  2. […] reiterate a bit of my trading philosophy here, for my own good. On the technical side of things, a daily […]

  3. […] it took only a few months in 2007? I would suggest the market is still under the influence of its long memory of initial conditions. A natural market must complete its cycle length before it outruns emotional demons of the last […]

  4. […] in 2007) will alter the contour and trajectory of our wormhole higher. Recall the market’s sensitivity to initial conditions, in that slight alterations to sentiment and psychology have a profound resonance. The 2007 […]


Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s