If the Random Walk Is Real, the Efficient Market Isn’t.
People who believe in an efficient market generally also believe that stock prices follow a random walk. It can hardly be that both concepts apply (at least not at the same time), can it? These are opposite sorts of phenomena.
An efficient market is one in which prices are in some important sense “right.” For prices to be right, they would need to be set by some rational process. Things set by a rational process are not random; they are predictable.
Mathematics is rational. The results of mathematical calculations are not random.
If you are trying to drive your car in such a way as to make efficient use of your fuel, you do not choose your driving speeds randomly; you choose them by a rational process telling you what speed generates the best results in various sets of circumstances.
Random prices are unpredictable prices. A market in which prices follow a random walk is an irrational market, an inefficient market.
The results of pure gambling (gambling in which the odds are not tilted to favor the house) are random. The results of a coin flip are random. There is no rational process that can be said to be determing the results of a series of coin flips. Coin flipping cannot be said to be an “efficient” phenomena.
Random Walk Proponents Have Failed to Specify the Time-Period in Which It Applies.
It appears that the confusion stems from a failure of the adherents of the two theories to specify the time-period to which they are referring when they make their claims. There are indeed circumstances in which stock prices appear to follow a random walk. There are also circumstances in which the efficient market theory appears to apply. Stocks are the Certs of InvestoWorld; they are two, two, two investment classes in one!
Stocks follow a random walk in the short term (time-periods of less than 10 years). The efficient market theory applies for time-periods of greater than 30 years. During the time-period in which prices follow a random walk, the efficient market theory does not apply. During the time-period in which the efficient market theory applies, prices do not follow a random walk.
There is a sense in which the claim made in the headline of this article is not true. Both the random walk and the efficient market are real. But both cannot ever be real at the same time, as is suggested by adherents of the ideas as they are usually formulated. The random walk is real only in the short term and the efficient market is real only in the very distant long-term.
Random Walk Proponents Do Not Understand Its Cause.
Burton Malkiel, author of A Random Walk Down Wall Street, does not appear to possess a good understanding of what causes the random walk. He has said: “It’s not that stock prices are capricious. It’s that the news is capricious.”
The suggestion is that it is news events that cause price changes. This claim brings to light another conflict in the thinking of random walk/efficient market proponents. Proponents of these theories often argue that investors should tune out the “noise” of reports of investing news developments, that it is only the distant long-term that matters. In the distant long-term, U.S. stock-market returns have been remarkably stable for a long, long time. U.S. stocks reliably provide a distant long-term annualized real return of about 6.5 percent.
I agree that it is the long-term that matters. I question whether Malkiel and others who argue that it is news developments affecting stock prices in the short run understand the implications of believing that it is the long-term that matters. If rational investors know that it is the long-term that matters, why would rational investors care about news developments that only affect short-term prices? News is noise.
There have of course been many news events that have transpired over all of the many time-periods in which the 6.5 percent return has applied. None of them seems to have left much of a mark. What seems to be the driver of stock prices in the very distant long-term is the productivity of the economy in which the underlying companies operate. The U.S. economy has long remained sufficiently productive to support a 6.5 percent annualized real return. So that’s the return we have seen, news developments be darned.
If the short-term price doesn’t matter, and news developments affect only the short-term price, then news development don’t matter either. Do they? News developments are really of interest only to short-term investors, gamblers, irrational investors, emotional investors.
I think that Malkiel would have been more on the mark had he said that it’s emotions that are capricious. Stock prices jump up and down in crazy ways in the short term; that much is so. But it’s not because news developments are unpredictable that short-term prices changes are unpredictable. It’s because investor emotions are unpredictable that short-term price changes are unpredictable.
If all investors were rational, few would even follow news developments from an investing standpoint. Except for a few exceptions (news developments that have a permanent effect on the productivity of the overall economy obviously do matter because they affect the continued viability of the 6.5 percent return), news developments just do not matter. Market prices are random and unpredictable in the short term because investor emotions are random and unpredictable.
News can cause shifts in investor emotions, of course; so news is not entirely lacking in influence on stock prices. It is the emotional reaction to news events that is the driver, however, not the news events themselves. If it is emotions that matter in the short term rather than news, the effect of news events is highly unpredictable. Good news can cause bad emotional reactions. Unimportant news can cause big swings in emotion. Emotions are irrational. Anyone who has ever gotten angry with a loved one for some meaningless and unintended slight (and that’s the entire population of humans who have loved) knows this.
Malkiel is suggesting that the market can be said to be efficient even in the short term because it is rationally responding to news developments. I am saying instead that the market is inefficient in the short term because it is emotionally responding to news developments. We agree that a random walk applies in the short term, but we come to this conclusion for very different reasons.
Short-term prices are capricious, Burton!
The Random Walk Does Not Apply in the Long Term.
Understanding the true cause of the random walk is important.
If you believe as Malkiel does that the random walk applies in the short term because the market responds rationally to news developments, you could be led to believe that long-term prices are set by the same process. As time goes on, the market just continues taking in information bits, continues behaving rationally, continues walking randomly.
I say “no.”
Why? Because the historical stock-return data shows us that stock prices are not random in the long term. Please take a look at the Stock-Return Predictor (see tab at left). The calculator reveals a strong correlation between valuation levels and long-term returns. If prices followed a random walk even in the long term, there would not be such a correlation. If prices followed a random walk even in the long term, prices could end up anywhere in the long term. Yet they never do!
What’s going on?
What’s going on, I believe, is that the influence of emotion is weakening as the time-period being examined grows longer. It’s only in the short term that investor emotion is the dominant influence on stock prices. As time stretches out, the economic realities become dominant. The market becomes more efficient and prices become less random.
It Is Important to Know the True Cause of the Random Walk.
There are important strategic implications that follow from a conclusion that it is Malkiel’s understanding of the random walk that is correct or my understanding of the random walk that is correct. If my understanding is correct, stock prices will almost certainly return to fair-value levels in time (this article was posted in December 2007). If Malkiel’s understanding is correct, it is at least theoretically possible that prices could remain at today’s lofty levels indefinitely; all that would be necessary for that to happen would be for us to see an exceptionally long string of positive news events.
My sense is that many of today’s investors follow Malkiel’s view of what causes the random walk. Many seem dubious as to whether long-term prices can be effectively predicted, possibly because they do not see how long-term news events can be effectively predicted.
However, if my understanding of why a random walk applies in the short term is correct, there is no need to be able to predict news events to be able to predict stock prices. In my model, news events do not matter. It is the gradual diminishment of the influence of emotions that causes stock prices to return to reasonable levels. Under my understanding of how stock investing works, the gradual diminishment in the role of emotions in setting prices is something that always happens and something that always must happen.
News events often trigger emotional reactions, to be sure. Emotions change because reality intrudes on our fantasy worlds through a string of unsettling or disturbing or unanticipated news events. It is not right to identify the news events as the primary cause of the change in prices that follows, however. News events are only a secondary cause. When prices get far out of line with the economic realities, adjustments are inevitable. News events can affect how soon adjustments take place; they cannot affect whether or not adjustments take place. The cause of price changes is the need for stock prices over time to reflect the economic realities.
An investor who placed his trust in the Malkiel model might decide to remain highly invested in stocks even at today’s prices on grounds that “you never can tell.” An investor who believed in the Bennett model would be foolhardy not to make some downward adjustment in his stock allocation when the price got as out of whack as it is today. Under my model, some sort of price adjustment is all but inevitable.
Under my model, there is no amount of “good news” that can stop stock prices from falling at some point; it is the clash between short-term emotional rationalizations and long-term economic realities that causes the gradual shift from a random walk to an efficient market. At some point, the emotional strains caused by the disconnect between market prices and the economic realities grow so great that investors interpret even nominally good news as bad news for stock prices. Dishonest prices (all bull markets are fundamentally dishonest) cause so many economic distortions that they are unsustainable for more than 10 years (or at least this has always been the case in the market history available to us today).
Stocks are held by humans and stock prices are set by humans. So we need to look to human behavior for clues to understanding how stock-return patterns play out. Alcoholism is an act of dishonesty. Do alcoholics find their way back to the reality principle by means of an “efficient” or rational process? They do not. An alcoholic can be threatened with the loss of his family, his job and his health and yet fail to respond to pleas to change. Then he will read a letter from his daughter about the joy she is finding in studying a course that he too enjoyed in his college days and all will suddenly be made clear to him. Humans are not rational. They’re human!
Humans do not turn into entirely different sorts of beings when they come to own stocks. The false confidence in our financial futures brought on by bull markets causes acts of dishonesty as doomed and determined as the false confidence in our personal lives brought on by drink. Investors return to normalcy though a process not dissimilar to the process by which their fellow humans recover from the other form of drunkenness. We do it by fits and starts. For years we turn away from the rational case made by the historical record of stock performance through the years. Then one day we become sick of our own lies and our hearts are turned. It is not news events that cause price changes in the short-term. It is shifts in mood caused by — only the Shadow knows the strange twists and turns that cause human hearts to change.
There are Two Random Walk Models — A News-Driven Model and an Emotion-Driven Model.
The Malkiel model of the random walk and the Bennett model of the random walk are very different models. Although Malkiel argues that investors should ignore the short-term “noise,” he places more importance on news developments than I do. My experience is that even investors who say that they “do not know and do not care” how stocks will do in the short term often take comfort in Malkiel’s idea that it is news developments that cause price changes. Why? I think it is because the idea that news developments are significant gives stock-drunk investors a rationalization for ignoring the strong message of the historical stock-return data — that stock allocations must be lowered when prices reach the sorts of la-la land levels that apply today.
Under my model, investors betting that valuations will not come down are like gamblers kidding themselves into thinking that because it is possible to beat the house in 20 pulls of a slot machine it is also possible to beat the house in 2,000 pulls. We all understand that the longer a gambler continues pulling a slot machine lever, the greater grows the edge possessed by the house. I say that the laws of probabilities apply in the same way for stock investors. It is possible for an investor who ignores valuations to remain ahead for three years or five years or even 10 years. After 10 years, the odds become incredibly long against that continuing to remain the case. Eventually, the laws of probability catch up to the investor who insists on betting against the house.
If you understand why the house stands a good chance of losing to a gambler who makes only a few pulls of the lever but is virtually certain to end up ahead of a gambler who makes 2,000 pulls, you will be close to understanding the terrible mistake that adherents of the random walk and the efficient market have made in developing their model of how stock investing works. The reason why the house is almost certain to be ahead after a large number of pulls is that it takes time for the laws of probabilities to come to dominate results. The probabilities always favor the valuation-informed investor. But the probabilities can be beaten for one year or three years or five years.
Can the probabilities be beaten for 10 years? Anything can happen in this crazy old mixed-up world of ours. I sure would not want to be betting my retirement money on that long-shot bet coming through for me, however. I want the probabilities on my side in the long run.
Those Who Believe in a Long-Term Random Walk Misunderstand How Risk Is Rewarded.
The Efficient Market Theory has sprung a lot of leaks in recent decades. The historical data simply does not support today’s dominant investing model. However, its proponents have been reluctant to cry “uncle!” They continue to believe that they will come up with some way to square this theory with the facts. I see this as a futile quest.
When evidence is put forward that investors who make intelligent choices are able to do better than those who do not (this would not be possible if the market really were efficient), the standard response of Efficient Market Theory proponents is that those investors must be taking on more risk (if they earned greater returns as compensation for taking on more risk, that would not violate the rules of “efficiency”). The obvious problem with this argument is that it renders the theory untestable. If all evidence that the market is inefficient is dismissed as a case where more risk applied (but did not cause unappealing results), there is no way ever to disprove this theory. Theories that cannot be disproven are not the products of scientific investigation; they are mere unproven assumptions.
Again, what’s really going on here?
Just as the random walk enthusiasts fail to distinguish the influence of news developments from the influence of investor emotion, the efficient market enthusiasts fail to distinguish different types of risk. It is probably true that risk is related to return in some ways, but I do not see any evidence that this is so in the way in which the theory’s proponents claim it is.
The theory’s proponents fail to distinguish real risk from perceived risk. Investors often fail to properly assess the risk associated with stock investing. The Return Predictor shows that the risk of stock investing has been sky high in recent years while the likely long-term return for stocks has been a good bit lower than what can be obtained from far safer asset classes.
Efficient market enthusiasts dismiss today’s reality as a logical impossibility. They imagine a law of the universe that ensures that stock investors always be rewarded for taking on extra risk. The reality (as seen from my perspective, of course) is that investors are often compensated not for real risk but for perceived risk. When stock prices are high and risk is high, stock investors receive no compensation for taking on risk because they do not demand any (stock investors are generally most complacent about risk when prices are high and risk is great). When stock prices are low and risk is low, stock investors receive an extraordinary amount of compensation for the little risk they take on (stock investors are generally unwilling to consider stocks when prices are low and can only be enticed to test the waters by the promise of long-term returns far in excess of what stocks provide at times of normal prices).
The Most Important Time-Period for Investors Is the Time-Period In Which the Random Walk Is Being Transformed Into an Efficient Market.
Both the random walk and the efficient market are important concepts. We have learned much from the work done by Burton Malkiel and Eugene Fama and their many supporters. Unfortunately, their failure to specify the time-periods in which the two conflicting models for understanding how stock investing works apply threatens to bring about a wipe-out of middle-class wealth in days to come.
Most of today’s investors seem to believe that it is possible for there to be both a random walk and an efficient market at the same time. I sure do not see how. If you are able to make sense of claims that both concepts can apply at the same time, I would be grateful if you would drop me an e-mail letting me know your thinking. The question I need answered is — If the market is efficient even in the short-term, why is it that investors who change their stock allocations in response to price swings are able to gain a long-term edge?
If I am right that the two concepts apply at different time-periods, then the next logical step in our effort to come to a realistic understanding of how stock investing works is to focus on the time-period from 10 years out until about 30 years out. We know how stocks perform after the passage of 30 years (there has been much written on the efficient market theory). And we know how stocks perform in the short term (there’s been much written about the random walk). We know little about the all-important time-period that stretches from Year 10 to Year 30.
Those years are the years most important for the middle-class investor. It is how a middle-class investor’s portfolio performs in those years that determines whether he will be able to walk the buy-and-hold walk as well as to talk the buy-and-hold talk. Please take a look at the three calculators now available at this site (The Stock-Return Predictor, The Retirement Risk Evaluator, and The Investor’s Scenario Surfer) for an overview of the work that I have done with John Walter Russell to begin to provide investors a sense of what sorts of investing strategies those seeking financial freedom early in life need to be following in this all-important-but-as-of-yet-rarely-examined time-period.