The Myth of the Rational Market

Justin Fox is an economics reporter for Time magazine, and he is the main contributor to The Curious Capitalist blog, which is one I read a lot. So I was excited when he announced that he was coming out with a book called The Myth of the Rational Market, which, I gathered, was an attack on the efficient market hypothesis. I just finished it, and I have to say it wasn't what I expected. Instead of marshaling the evidence for and against the theory, what Fox has written is a history of how the EMH arose in the 50s and 60s, how it caught on first in academia then on Wall Street, the over-reach of the hypothesis, and how researchers and practitioners started finding flaws in the theory in the 70s, 80s and 90s.

I specialized in finance when I got my MBA. I finished it in 2008. My finance professors were EMH partisans--or at least they presented themselves that way in class. In any case, our core learning was around basic EMH ideas--Modigliani-Miller, portfolio theory, Black-Scholes, CAPM, beta, etc. In my real estate finance class, we learned practical rules of thumb rather than theory, in my risk class, we were taught that risk controls had limits, and in Futures & Options 2, we learned about implied volatility and volatility smiles. But those were the only cracks in my EMH education. One professor (I won't say who) specifically defended EMH by contrasting it to chartism or technical analysis.

The arguments in favor of EMH versus chartism are profound and irrefutable, but chartism is, of course, a straw man. It's like arguing that Newtonian mechanics is the last word in physics by comparing it to Aristotelian and Ptolemian physics.

In any case, even though I was pretty dazzled by the logical arguments of the various parts of EMH, I had one nagging doubt. The idea that the best portfolio was a combination of debt and a market-weighted portfolio of the entire market made logical sense. But then, I thought, why would anyone trade stocks? Why wouldn't everyone just invest in index funds? And if everyone did invest in index funds, how would prices be determined? Where would the market be made?

These nagging questions, plus questions about risk and about the instability of volatility (we learned about GARCH in my risk class) encouraged me to look beyond EMH, which almost invariably leads one to behavioral finance. Essentially, Fox's book is about how theorists and researchers made that particular journey. What he doesn't do is try very hard to explain what the theories were. If you haven't studied finance, I wonder what you will make of this book. For example, in discussing Modigliani-Miller, he tells us that they theorized that capital structure doesn't matter for a corporation. But he doesn't explain what capital structure is (how much of the company is funded with equity, and how much with debt), or that they were talking about capital structure in a kind of idealized world in which income taxes and bankruptcy didn't exist. He certainly doesn't burden his readers with any math, and typically gives only the most basic description of each theory or advance. Partly this is because he has a lot of material to cover, and to do so, he made a decision to concentrate on the people, not the theories. If he explained each theory, even in a non-mathematical way, the book would have been three times as long. Fox is great at portraying the arrogance of the EMH theorists. He digs up some choice quotes.

One must realize that these analysts are extremely well endowed [sic]. Moreover, the operate in the securities markets every day and have wide-ranging contacts and associations in both the business and the financial communities. Thus, the fact that they are apparently unable to forecast returns accurately enough to recover their research and transactions costs is a striking piece of evidence in favor of the strong form of the [efficient market] hypothesis.

This was said by Michael Jensen, who invented "alpha." (Perhaps this was the source of the phrase "big swinging dick"!)

But he is also good at how EMH was slowly built up, and how logical it seemed, especially in contrast to what had happened in the past.

But researchers started finding exceptions to the rule. After Robert Shiller discovered that stocks were way more volatile than dividends, he was willing to declare that EMH had made a huge logical error.

The leap from observing that it is hard to predict stock price movements to concluding that those prices must therefore be right was, he declared at a conference in 1984, "one of the most remarkable errors in the history of economic thought."

Even the intellectual author of the EMH, Eugene Fama (father of cartoonist/political blogger Gene Fama) came to realize that CAPM wasn't enough, and worked hard to come up with an alternate theory (CAPM was a linear regression with one beta--specifically Beta, the measure of the volatility of a stock with respect to the market as a whole; his new model added additional betas, or factors as he called them).

One of the problems researchers started noticing is that many of the ideas built out of EMH only work if a small number of investors were doing them. This was the thing that worried me back in my first finance class. What happens if you have a bunch of S&P 500 index funds? It means that when a stock enters the S&P 500, it gets a huge boost in stock price. How is that kind of thing consistent with EMH? This kind of thing gets to the core of criticism of EMH--how can prices be rational if everyone jumps on the same bandwagon? How can prices be "right" in a bubble? Why don't bargain-seeking arbitrageurs prevent bubbles?

It was precisely when the market was at its craziest, [Andrei] Shleifer and [Robert] Vishny argued, that those who try to end the craziness by placing bets against it would have the hardest time keeping their customers or borrowing money. "When arbitrage requires capital, arbitrageurs can become the most constrained when they have the best opportunities, i.e., when the mispricing they have bet against gets worst," they wrote. "Moreover, the fear of this scenario would make them more cautious when they put on their initial trades, and hence less effective in bringing market efficiency."

This kind of thing--even though it was empirically observable--was starting to sound suspiciously like psychology rather than the actions of rationally self-interested markets!

The book is divided into two parts. The first part is about the rise of EMH, and it's the best part. The slow build-up of the theory is a clear story--perhaps because handsight makes it easier to form the story. But also, the EMH is a fairly unified idea, which makes the progress in building it more unified.

The second half, covering more recent events, is more scattered. Not enough time has passed for a narrative of "anti-EMH" to really gel. Also, there is not an alternate theory to EMH, just a bunch of critiques. Behavioral finance is a collection of theories and experiments and observations that don't exactly gel the way EMH did. Nonetheless, it seems that no serious financial economist can believe that EMH accurately describes the market. Nonetheless, we still have pundits and politicians who believe markets are, essentially, perfect--despite the evidence in front of their eyes. Fox references Thomas Kuhn, and what we are seeing here is a paradigm shift. There are a lot of people who won't let go of the old belief because 1) it worked pretty well, and 2) their salary depends on them believing it. It will take time.

This book is well-worth a read, but only in conjunction with learning more detail about EMH and behavorial finance, either in classes or through some of the other excellent books available about modern financial theories.