Surprisingly, belying conventional wisdom, the link is too tenuous.

It is the most commonly used tool in the market. And often it is useless. If there is any consensus about investing tools in the stock market, it is two magical letters PE. From the most informed fund manager to the uninformed investor to the finance professor and the polished TV anchor, everybody seems to believe that PE is one of the most important determinants of investing. Implicit in the god-like status of PE is the belief that earnings and markets move up and down together. The greater the change in earnings, the greater is the movement in stock price. When earnings are up, it is time to buy, and when earnings are down, it is time to sell. When PE is low it is time to buy and when PE is high it is time to sell. Real life examples of the belief that stock prices are correlated to earnings are available everyday for all to see.

For instance, before the recent market decline fund managers have been vociferous in claiming that since the earnings are expected to go up 15-17% over this year, investors should expect a return of 15% over one year. How pat! The Sensex, instead of living up to this belief acted like a suicidal monomaic, shedding 3500 points over a month. The fall in Sensex did not diminish this belief. But does it actually work that way in real life? Here are some facts about Hero Honda during various periods. Look at the table:

• In September 01 and Dec 01, HH's net profit grew by 63% and 91% but   the PE hardly budged.

• When growth slowed, P/E started rising. In fact, when the net profit in   March 2005 and June 2005 declined by 2% and rose by just 8%   respectively, the PE went on to hit its all time high of 16. Maybe a quarter   is too short a time period to judge. So, consider some longer term   correlations:

• Net profit rose 88%YoY in the FY 02, but its P/E went on declining from   the high of 14 to 9 in the FY 02.

• In FY03 net profit rose 25% but the PE remained at 9. For the same 25%   rise in net profit next year, PE jumped to 15. In FY05, net profit growth   slowed down to 11% but PE remained at an all time high level of 16.   Clearly, far from earnings following valuation, the actual movement of   stock prices is random. The reality is in complete contrast to market the   theory. Assumptions about earnings being correlated to stock price are   completely opposite to the empirical evidence. PE as a predictive tool?   Think again

The “best money manager on earth” is a mathematician.

"T"he advantage scientists bring into the game is less their mathematical or computational skills than their ability to think scientifically. They are less likely to accept an apparent winning strategy that might be a mere statistical fluke”- Jim Simons, President, Renaissance Technologies

Can somebody who holds seminars on topics like “Generalized Chern-Simons Invariant's as a Generalized Lagrangian Field Theory” on his 60th birthday, be the best market player in the world?

In the previous information, we had mentioned that there are at least two great investors (I use the term loosely to include traders), who have very successfully used quantitative tools to record stunning market performances over a long period. One of them is Jim Simons, about whom the Indian media is totally unaware. That is not their fault. Few even in Wall Street, including his fellow-travelers in quantitative trading, know the full dimensions of Renaissance Technologies Corp. that Simons heads.

Over more than two decades, Simons has been the leading light in marrying math and markets. Simons is supposed to be a cryptanalyst, mathematical physicist, academic - and a billionaire investor. Simons graduated from Massachussets Institute of Technology at 20 and completed his Ph.D. in Mathematics from the University of California at Berkeley, barely three years later. By 23, he was in the MIT faculty. From 1961 to 1964 he taught mathematics at MIT and Harvard University. In 1968, he became the chairman of Stony Brook’s math department and was beginning to be famous in academia. Simons did breakthrough discovery and application of certain geometric measurements that resulted in the Chern-Simons form (also known as Chern-Simons invariants, or Chern-Simons theory). In 1974, his theory was published in Characteristic Forms and Geometric Invariants, co-authored with the differential geometry expert Shiing-Shen Chern. The theory has a wide use in theoretical physics, particularly the famous string theory, used to explain the birth of the universe, black holes and super gravity.

In 1976, at 38, Simons won the American Mathematics Society’s Veblen Prize that is awarded every five years, for his work on differential geometry. It is the highest honour in the world of geometry.

Simons got interested in trading only in the early 1970s. In 1973, a tile company that he had invested in got sold and Simons gave his money to a mathematician who was trading in commodities. In eight months, Simons saw his money go up 10 times. Intrigued, Simons began trading in currencies. In 1978 he left academics, to form a private investment fund called Limroy. In 1982, he founded Renaissance Technologies Corporation, a private investment firm based in New York. Simons is still at the helm, as president, of what is probably the world’s most successful hedge fund.

In 1988 Simons decided to launch a fund that concentrated on pure trading. He launched Medallion in March 1988. The fund earned 8.8% in 1988 but lost money steadily thereafter until Simons stopped trading. For six months Simons and Princeton mathematician Henry Laufer (still Renaissance’s research chief), rebuilt Medallion’s trading strategy, shifting from fundamental analysis to a mathematical approach and rapid-fire trading strategy.

According to Institutional Investor magazine, Simons earned $1.5 billion in 2005 and $670 million in 2004. The only major article about him published by the Institutional Investor magazine in 2000, said that “Simons may very well be the best money manager on earth.” In March 1988 Simons started his flagship $3.3 billion Medallion fund. By 2000, the fund had notched returns of 35.6%, compared with 17.9% for the Standard & Poor’s 500 index. For 11 full years ending December 1999, Medallion’s cumulative returns were 2,478.6%. Quantum fund of George Soros was the next-best with a 1,710.1%. Want to invest in his fund? Sorry, it has been closed to new investors since 1993. How did Simons manage to do it?

How a mathematician runs the word’s most successful hedge fund

In our above issue we profiled Jim Simons, one of the most successful traders of the world, who remains virtually unknown even in his own country. His firm, Renaissance Technologies, is intensely secretive even though it manages $12 billion. It had a website showcasing basic facts about the firm. The site no longer exists. How has it run up a 35% return per annum since 1989? Renaissance is like a college campus. The headquarters has a gym, lighted tennis courts, a library with a fireplace and large private offices for every employee. The centerpiece is an auditorium with exposed beams that seats 100 and features biweekly science lectures.

Among fund’s key professionals are Henry Laufer, the fund’s Vice President for research, who has been with Renaissance since 1989. He was a professor of mathematics at the State University of New York (SUNY) at Stony Brook for 20 years. He has also taught at the Massachusetts Institute of Technology and at Princeton University. Another key executive Robert Frey has a Ph.D. in Applied Mathematics and Statistics from Stony Brook. He researches on systems for stocks.

Peter Weinberger has a Ph.D. in Number Theory from Berkeley. He has taught mathematics at the University of Michigan, and then spent most of his career in systems and software research at Bell Labs. He heads Technology. Paul Broder runs the 20-member trading group which trades 24-hours a day across three time zones. He came to Renaissance via JP Morgan and Chase Bank and looks for people with an appetite for risk and a keen sense of survival. Peter Brown earned his Ph.D. in Computer Science from Carnegie-Mellon University. Before joining Renaissance in 1993, he worked for 10 years at IBM Research on speech recognition and automatic language translation. These are the kind of people who run Renaissance.

The only publicly available piece of detailed information about the firm is a 2000 article in Institutional Investor magazine. According to it, the Renaissance office “resembles nothing so much as a high-powered think tank or graduate school in math and science. Operating out of a one-story wood-and-glass compound near SUNY Stony Brook, Renaissance has 140 employees, one third of who hold Ph.D.s in hard sciences. Among their ranks: practitioners in the fields of astrophysics, number theory, computer science, mathematics, physics and statistics from Japan to Cuba. In notably short supply are finance types. Just two employees, including the head of trading, are Wall Street veterans.” Simons was quoted as saying: “I have one guy who has a Ph.D. in finance. We don’t hire people from business schools. We don’t hire people from Wall Street. We hire people who have done good science.” Renaissance has been recruiting experts in computational linguists who have worked on speech-recognition systems. This is because investing and speech recognition are similar in a sense. In both, one is trying to guess the next thing that happens.

Renaissance builds statistical models that can predict the future movements of traded products. When trading starts, the models rule, specifying that trades pay off within a defined time. Guided by these models, its traders execute rapid-fire trading. The traders never override the models. Simons, estimated to be worth $2.6 billion, donates generously. He has given away $7 million to MIT, $2 million to Institute des Hautes Etudes Scientifique in Bures-Sur-Yvette, France, $1.5 million to the Institute for Advanced Study in Princeton, New Jersey, $1.7 million to the Mathematics Science Research Institute in Berkeley, California. In early 2006, he led a group of directors of Renaissance and of Brookhaven Science Associates in donating $13 million to fund a budget shortfall of the Brookhaven National Laboratory that would have shut down the Relativistic Heavy Ion Collider. Simons plans to spend $100 million on autism research after his daughter Aubrey was diagnosed as autistic.

In 300 years of human evolution, stock markets lagged behind other streams of activity. This awareness helps one to make successful investments.

When you buy a call option of Tisco, you are buying a financial instrument that has been in use for centuries of a company that is 90 years old, making a product that in some form or the other has been used by human beings for thousands of years. Why is this significant?

Very little seems to have been changed in the world of markets. From trading in options of East India Company in 1690 in London Change Alley to those of Tisco in 2005, the only difference is where it is traded. Today, we have an electronic market to trade in, while in the 1690s, trading was in smoke-filled rooms and dark narrow alleys. So the change, over almost 300 years is not in the market elements, such as issuers and instruments and the reasons why these are bought and sold. The change is in the technology. The real area of change was not brought about by finance but by the relentless march of scientific progress.

This says a lot about the progress of financial markets as an area of human activity. Consider instead the progress made by medicine and astronomy over the last 300 years. Or even by the economics of capitalism, the "dismal science" that started its formal journey as a discipline in the late 18th century when Adam Smith wrote his path-breaking treatise. By the way, even at that time, Smith was not called an economist. If Adam Smith were carrying a calling card then, it would say, ‘Moral Philosopher’.

Slow Progress, Much Regress

For all the headlines it occupies and the attention it gets, the stock market as a field of human endeavour has been slow to develop and is still beset by the same motivations, practices and instruments as were prevalent centuries ago. Indeed, there has even been much regression. Financial innovation that worked fine at one time has been junked and then revived - sometimes centuries later. Stock options were formally introduced in India in 2001 - almost 300 years after it was a flourishing instrument in Amsterdam and London, and decades after they stopped being informally used in India. Commodity futures were banned in 1969 in India and brought back in 2003. This kind of time warp has affected even other aspects of finance. For instance, today, we are going back and forth on the issue of private financing of infrastructure projects. One of the major projects at the turn of the previous century, the Panama Canal, was privately funded. After seven decades of misguided ideas of socialism, we are rediscovering simple financial ideas that have worked so well in the past.

This is unthinkable in physics, medicine and other streams of science. It is as if penicillin was discovered and used extensively, then forgotten for 100 years and revived again as a useful drug to fight bacteria. Or that Newtonian physics was found to be useful, like stock options, when Newton was alive, then fell into disuse until the 20th century scientists found it useful again to explain many natural phenomena.

New drugs and new ways to fight disease are being discovered, drawing from the knowledge of earlier generation of drug discovery. New materials such as carbon composites are being developed as relentless advancement to discover tougher and lighter material. We are not stuck with steel and aluminium. Current scientists always stand on the shoulders of their earlier generations of scientists.

In short, scientific knowledge is accretive, accruing more or less systematically and progressively over a period of time. Financial knowledge is random, not progressive and sometimes even regressive. What if scientific methods and approaches are applied to markets? The results are very interesting, as we shall see below.

What has a statistical tool called multivariate analysis got to do with understanding where the stock market is headed.

In the above issue, We had initiated the thought that very little seems to have changed in the world of stock markets. Over the past 300 years of stock trading, there has not been much change in the main market elements: the issuers, instruments and reasons why stocks are bought and sold. This is in sharp contrast to, say, scientific progress. Today’s scientists stand on the shoulders of earlier generations of scientists. Scientific knowledge is accretive. But progress in financial innovation has been random. There has been too much of back and forth, too many twists, turns and retracements.

What if scientific methods and approaches are applied to stock markets? The towering figure of this approach is ace speculator Victor Niederhoffer, a legend on Wall Street. He has pioneered what is called the quantitative study of the market, which relies on a scientific approach. According to Niederhoffer, the starting point of this approach is defining the scientific method itself, which the Oxford English Dictionary defines as “a method of procedure that has characterised natural science since the 17th century, consisting in systematic observation, measurement and experimentation and the formulation, testing and modification of hypotheses.”

Victor, who is extraordinarily erudite, points out in his book The Education of The Speculator that the flavour of scientific methods “was described by Davy as analogy confirmed by experiment; by Stanley Jevons as discovering identity among diversity; by Walter Pater as the analysis of rough and general observations into groups of facts that are more precise and minute; and by Herbert Spencer as finding sequences among phenomena and grouping them into generalisations.” In essence, scientific work involves classification, observation, questioning, testing, measuring, collecting information, experimenting, modeling and revising theories.

How many stock brokers or fund managers employ this kind of complex approach in dealing with stock markets which are ever-changing and mysterious? How many of them proceed by observation, insight, hypotheses and trial, rather than relying on hunch or crude tools of price trends and earnings forecast? They don’t, which is why scientific theories stand the test of time while stock market predictions don’t.

How does a scientific process help? In an uncertain world, it helps reduce the margin of uncertainty. It helps in overturning incorrect ideas or market myths as we call them. As Victor puts it, “errors in judgment are particularly frequent in choices involving uncertainty and risk.” The scientific tool designed to deal with uncertainty is statistics, mainly probability, and Victor has done path-breaking work in applying probability to markets.

Applying mathematics to markets is not rare. Victor is only the most celebrated example. There are boutique research firms that apply high-end computers to crunch market data applying sophisticated mathematical tools. Victor trades strictly on the basis of statistical anomalies and the quantification of persistent psychological biases. He owes his influence to Francis Galton, the inventor of weather maps. Victor came across Galton while studying the influence of weather on sporting outcomes. Galton’s favourite motto was, “Wherever you can, count.” He traveled with a pin and an index card so that he could tabulate anything that struck him as noteworthy.” As Galton himself said it in his book Memories of My Life: “I frequently make statistical records of form and feature, in the streets or in company, without exciting attention, by means of a fine pricker and a piece of paper…The holes are easily counted at leisure, by holding the paper against the light, and any scrap of paper will serve the purpose...”

If only market experts took the trouble to incorporate even a fraction of this approach, the public would be losing much less money. In the US, two speculators have taken the scientific path and have been amply rewarded.

“Earnings will be good and therefore stocks will go up”. The market trashes this kind of facile short-term punditry of media and brokers time and again. But few investors seem to be bothered

For weeks before the earnings’ season began, the entire media, especially the television media, has been peddling the view how the market direction will be set by the nature of corporate earnings. This piece of insight is usually followed by the assurance that since corporate earnings will be good… blah, blah, blah and since the P/E is 14 or 18 or 12 (depending on the earnings of 2008 or 2009)… expect the market to be on an up trend or be firm (pick an adjective of your choice).

Well, corporate results are great. Infosys announced a 50% jump in profit. Sensex hit almost 11,000. And what happened? It lost 1000 points thereafter. The mood suddenly turned sombre. There was consternation all around. The prediction of corporate performance was right but that of stock price was wrong. Surprisingly, there is no correlation. Soon thereafter, dire predictions of a bear market resurfaced. The Sensex rose 900 points. Surprise again.

If you have read earlier pieces of Earning Curve, you would not be surprised. Earning Curve, among other things, is a myth-buster and just two issues back we pointed out that changes in short-term prices are not linked to earnings. We also pointed out in the last issue that what you think is a long term, may really be an extended period of short term in nature. Irrational market behaviour that “should” end quickly, can go on for a long time.

There are millions of examples around us to prove that short-term market movements are random and are often irrational. Could it be that trying to impose our sense of rationality on market movements, to seek cause and effect (earnings will be better and therefore stock prices will go up), is itself irrational but self-serving? After all, a 24-hour TV channel, a 24-page newspaper or a stock broking network with hundreds of branches putting through thousands of transactions everyday can hardly be run by openly admitting to such short-term irrationality. What would they live on?

Indeed, just imagine switching on your TV sets in the morning and finding that your favourite 20-something anchor asking your broker, “What will the market do today?”, and getting the following response: “I am sorry I don’t know. In fact, I have kept a count of all my opinions made here and I find that I was right 50% of the time. But within that too, on many occasions, I was right about the outcome, not about the reasons behind the outcome”. Or imagine, the younger anchor interviewing his older colleague, about where the market is headed and getting a reply that “we have not been able to establish a correlation between stock prices and myriad nuggets of opinions we collect all through the day.”

But then, admissions such as these will take away the basic reason for their existence. After all, a broker cannot tell a client anything but the precise reason behind the movement of a stock or the market as a whole. And if business media admits that it may be thrilling and engaging to dissect the myriad factors behind changes in short-term prices, although having little predictive value, then what would they do? After all, they have acres of newsprint and hours and hours of airtime to fill.

This is all the more glaring at market turning points. Since prices very often change much before fundamentals do, we get misled by a consistent fall in prices when corporate results are good, or firm prices when the results are still bad. So, next time you eagerly look for market opinions at the end of the day or the next morning, know that while it may be interesting to be aware of the chatter, it is irrelevant to what the market will do subsequently.

Some technical analysts rely on an arcane 13th century arithmetic series, called the Fibonacci sequence. A recent study says that it is false belief

Many traders believe that clues to where the market is headed lie in the work of Leonardo of Pisa, a mathematician who lived in the 12th and 13th century. Leonardo, widely known as Fibonacci, had produced a sequence by adding consecutive numbers in a series starting with 1 and 2. It goes as: 1, 2, 3, 5, 8, 13 and so on. Intriguingly, the numbers in this series crop up frequently in nature. There are leaves that grow in this sequence as also shells that are patterned along this sequence. Most interestingly, if you divide each number with the next one, you inexorably move towards 1.618. This is supposed to be the golden ratio in architecture and design, most famously exemplified by the nude man with his arms stretched in Leonardo Da Vinci’s painting of the Vitruvian Man. The Fibonacci magical sequence has also been glorified in “The Da Vinci Code”. Financial traders have long invested a great deal of belief in the golden ratio and the Fibonacci numbers. They believe that markets will retrace 61.8% of a previous leg or go up to 1.618% of the previous move. When this does not work, they use 50% and 38.2% as well, which are supposed to be other important Fibinacco percentages.

Fibonacci worshippers are part of a gang called chartists, or more pompously, technical analysts. They believe that past patterns in prices can provide clues to the future price movements. Some detect and follow patterns such as “cup and handle” or “double bottom”. Some others follow waves, which supposedly move according to the magical Fibonacci numbers and ratios, the most arcane and mysterious of the TA tools. The belief in Fibonacci sequence is so great that the chartists track whether the number of weeks that have elapsed following a move, is a Fibonacci number. For instance, if it is the 13th week after the market has started rising, it could mean that it is time for a correction. Well, none of this is ever tested. The believers selectively remember the times when it has worked and not when it has not worked.

What if someone statistically tested the usefulness of Fibonacci sequence? Professor Roy Batchelor and Richard Ramyar of the Cass Business School did and found no evidence that Fibonacci numbers work in American stock markets. They looked at all major moves of the Dow Jones Industrial Average between 1914-2002 and found no indication that moves reverse at the 61.8% level, or follow any other pattern.

The researchers go beyond the failure of Fibonacci to predict prices and reject the idea that TA works at all. “The root of the problem is the failure of technical analysts to specify their trading rules and report trading results in a scientifically acceptable way. Too often, rules are so vague and complex as to make replication impossible,” they write.

Indeed, one of the biggest beets noires of TA, Victor Niederhoffer, has tested every TA indicator over his 45 years of trading. So far, not one has passed the test of statistical significance. What Victor found instead were haphazard anecdotes, confident assertions, and appeals to authority. He too believes that practitioners and advocates of TA fail to follow standard scientific procedure in presenting and evaluating its techniques.

If you divide each number with the next one,
you inexorably move towards 1.618.
This is supposed to be the golden ratio in architecture and design.
Financial traders have long invested a great deal of belief in this ratio

Is your “long-term investment” just another form of speculation.

You buy an apartment or a house to live in or to rent it out and earn some income. It may give you great pleasure to check about the resale value from time to time but you have not bought the house to flip it over in a hot property market. (If you tried to do that in 1992, you made a bad investment. Over the next 13 years, chances are that checking the resale value would have caused you distress.) You may buy an orchard to take a walk in it. Or for the income the fruits would fetch you. But not to flip it over when nearby railway tracks are broadened and the land value of your orchard goes up. But shares are another matter. It is the only asset that we buy to flip, even long-term investors among us.

Your apartment has a market-determined value. Your orchard has a market-determined value. How about stocks? The problem about using market prices to value stocks is perilous. The question is, which price? The one that was quoted three weeks ago or the one that your broker quoted two minutes ago? What if the shares were valued, not for what the market is paying for them just now, but just as another income-generating asset like orchards and houses? Talking about an income stream from stocks may seem ridiculous when they bob up and down like turbo-charged puppets. But like it or not, seeing stocks this way, helps us understand the difference between speculation and investment.

Stocks, like many other assets, generate an income stream - just as your orchard and rented out property. How to determine a stock’s stream of income and from there how do we derive its actual worth? In financial theory there is something called Dividend Discount Model (DDM), which values stocks by the income they are expected to generate. It says that the value of a stock is the present value of its future earnings. Such earnings for shareholders come mainly in the form of dividends. Now the rate of dividend itself may change, in which case the shareholders earn more. So, the value of a share simply ought to be Dividend Yield + Dividend Growth. Is this all theory that only seems intuitively accurate or does it make any sense in real life? Look back at the largest and the most liquid market in the world, where this theory had a chance of being played out over a 100 years. The Dow Jones Industrial Average earned a return of 9.89% between 1900 and 2000. Dividend yield from stocks during this period was 4.5%. Another 4.5% came from growth in dividend rate. That accounts from 9% of the 9.89% return. So, in the end, the giant real life laboratory of Wall Street did produce exactly what the theory said it would.

There are several problems with the DDM model. One, it is really a very long term valuation tool. It cannot predict the value of a stock even a few years from now. There are other flaws too, which your finance textbooks will not tell you. DDM does not take into account the hugely positive impact of buyback's, splits and bonuses that cannot be captured by dividends. Nevertheless, it helps us focus our attention on a boring but logical way of valuing stocks - as financial assets to earn income from and not speculative assets to be flipped over.

How does a long-term investment for, say, 3-5 years, look through the prism of DDM? It may sound incredible but there is no logical tool to forecast returns for such short periods of “long-term investment”. So, even a long-term investment made for any purpose other than solely for its stream of income, is really speculation. The implication of this is ominous.

The line between “long-term investment” and speculation is thinner than you think.

“The market can stay irrational longer than you can stay solvent” - John Maynard Keynes
“One of the world’s most famous speculators, Warren Buffet, … .”
- Victor Niederhoffer in The Education of a Speculator

In the above paragraphs , we had discussed a tool that helps us determine what should be our long-term expectation from stocks. Long-term return is captured by the idea of Dividend Discount Model, which essentially means that you should expect stocks to fetch a total return that is the dividend rate plus the growth rate in dividends. But this has worked only over a very long term. There is no tool to pin down expected return over, a ‘shorter’ long term, say, three years. Can anyone really say what a basket of equities should fetch by 2008? We mean, anybody with a sense of accountability, not the TV pundits who are never asked to face their confident predictions gone embarrassingly wrong.

But aren’t there many well-known ideas about projected valuation and expected returns? Here is one, for instance. Since India will record a growth of 8% and inflation may be about 5%, the nominal growth will be 13%. This will be the minimum growth for the Indian corporate sector and the better ones will grow at 20% and so…. What is left unsaid are three (untested) assumptions. One, GDP growth automatically means equivalent corporate growth and more. Two, this will automatically lead to a corresponding reflection in stock price. Three, market is always fairly valued.

All three are myths. A GDP growth of 4.5-7% between 1994 and 2003 did not reflect in the corporate performance. Stock prices during this period had no relationship to GDP growth. And with the great wisdom that comes from hindsight, pundits now say that stocks were undervalued for much of this period. Similarly, another market myth is that high P/E stocks will fetch lower returns and low P/E stocks. We have proven the irrelevance of P/E in expected stock returns in the July 7 issue.

This piece is not about the myths of popular valuation methods. We will throw light on valuation myths in subsequent issues. We highlight these myths here only to illustrate the fact that the valuation tools widely used and propagated by the media, fetch random results. Keep a track of all the forecasts you see today to know what we are talking about. The more important point is, returns in stocks can be expected in a general way but cannot be anticipated within specific time periods. US stocks have delivered great returns over decades but the broad market index was totally stagnant between 1965 and 1982. Now, 17 years is 2/3rd of the equity investing period of an average individual. It cannot get any longer term than that.

If for 17 years, the market goes nowhere in the most market-intensive country on earth, then what happens to the cherished idea of “long-term investing”? Is long-term investing then simply another kind of punt on the future? Is this why Victor Niederhoffer, one of the most erudite and legendary speculators of the world, calls Warren Buffet, the supposed God of long-term investing and value investing, a speculator? Speculation is betting on an outcome that is random and uncertain. Often, both the process and outcome of long-term investing ends up being so random and uncertain that there remains no difference with a shorter term speculative bet. This is because of the behaviour of market participants, level of liquidity and so on - factors that have little to do with calculations based on which the long-term investment bet was laid. As Keynes said, the market irrationality can override investment rationality. Indeed, the best of speculators are aware of such irrationality and change their position quickly as opposed to the fixed systems of long-term investing.

If you want to be a serious investor, start with a serious study of market history. It will surprise you how risky equities are.

In the third week of May, thousands of housewives, college students and new part-time traders felt a rude shock. As the Sensex collapsed by over 2500 points over 10 days, there was bewilderment and panic all round. 'What was wrong? Why was the market behaving like this?' was the mute cry. After all, it was supposed to only go up and if it did come down, it should have been only a soft decline before another inevitable rise started.

Those who have been in the market for at least a decade should not have been surprised that the general expectation of how markets should behave was wrong. The answer to why the market was 'behaving like this' is secondary. The first thing to understand and accept is the fact that markets always behave 'like this'. Benoit Mandelbrot, who invented an entire field of mathematics, calls it The Misbehaviour of Markets and has written a book with the same title. Newcomers learn of such unexpected misbehaviour at their own peril and cost through experience. A cheaper way is to study some real history of the markets. That history is littered with booms and busts. This is not what financial theory teaches you. This is not what brokers tell you. Mutual funds, which make a percentage of the amount you put into the fund, are usually happy to present the sunny upside in equities.

One of the ways to figure out whether the market movements are "normal" is to take recourse to statistics. Using statistical tools financial theorists have examined whether market prices are "normally distributed". The results are interesting. There are too many days when the market goes too much to one side, than what statistical theories of "normal distribution" would imply. According to Mandelbrot:

• If the market followed a 'normal' pattern based on statistics over the last   century, there should be 58 days when the Dow moved more than 3.4%.   Instead, there were 1,001 such days. 

• Similarly, as per the "normal" assumption of Wall Street there would be six   days of index swings beyond 4.5%. In fact, there were 366.

• Index swings of more than 7% should come once every 300,000 years. In   the 20th century there were 48 such days. In India it has happened twice   in two years.

The normal market movement theory is flawed, as anyone who lived through "the booms and busts of the 1990s can now see", says Mandelbrot. Indeed, this 81-year-old founder of fractal geometry goes on to say that the entire field of investment analysis is founded on a few shaky myths such as market moves are normal and random. The result of following such myths can be quite devastating - like the shock from the recent sharp market decline. According to him, accepted wisdom substantially underestimates the potential for loss when markets go down. What do you need to know to deal with the sharp rises and declines that markets undergo periodically? According to Mandelbrot we need to know the four ways markets differ from popular perception.

Markets are Risky:

They are much riskier than most imagine - the recent decline drove home this point.

Markets move in Streaks:

Contrary to the accepted wisdom, big moves are often followed by big moves. There are long streaks - witness the continuous rally since October last year.

Behaviour can Override Fundamentals:

Markets are driven by the actions of people, not by fundamentals alone. The collective mania of foreign investors followed by locals piling into the Indian market drove it to peaks in 1994, 2000 and 2006.

Markets Mislead:

Investors love to find patterns and correlations. These work for a while and then suddenly break down with a severe force. Markets love to surprise.

In short, frightening declines like we saw in mid-May are not rare. It has happened in 2000 (dotcom crash), 2001 (9/11) and 2004 (after the general elections). That's just six years of recent history. The reasons were different each time!