Q&A on trading

Rusty, a good friend from my college days (and a bit beyond), has been asking some interesting questions in one of my posts below on trading. Here’s his latest comment:

Hi Jay –
The Vanguard funds have worked out well for me. Besides capturing the market return, they’ve had the other benefits of requiring absolutely none of my time, and I sleep a little better at night.
Regarding efficient markets, I think it depends on what you’re investing in. Are you looking at very small companies with a relatively small number of transactions? I might be convinced. If you’re trading GE and Microsoft, or more generally stocks in the S&P 500, I’m less inclined to agree. With millions of shares traded per day, that’s an efficient market. The people trading GE (as a whole) know a lot more about GE than I do.
If you actually generate the returns you’re expecting trading stocks in the S&P 500, that is an amazing feat.
Two other comments, if you will. I think it’s one thing to be able to say, “We’re in a tech bubble” or “These tulips cost way too much.” But it’s really hard to invest against a bubble. Greenspan was warning of irrational exuberance a couple of years before the bubble popped. Shorting too early would have cost quite a lot, not to mention passing up the rest of the ride up.
Besides that, I can’t think of how you’d try to exploit popularity in the 1-5 day time frame. That’s mindblowing to me, and I’d love to hear more about that. Are you researching these companies, or is your selection based on some kind of technical analysis or trend detection? I’m intensely curious.
Finally, the big question for me. Why aren’t others discovering this, and competing away the returns? Why isn’t some mutual fund company advertising a fund that returned 70% on average over the past 6 years?
Greatly enjoying this,

I started writing a response in the comments, but thought I’d put in a post so others can join in if so inclined.

Okay, one point of clarification. When I discuss my system, I am not referring to Tarzan. Tarzan was/is an experiment, but I do not trade it currently, though there are a couple very good ideas cooked into it (I think). If you read the original post on Tarzan, you will see a reference to my early attempts at Collective2 which performed great but were ill received. That system was the alpha of my current trading system, the one I talk about when referring to my own trading.

A point of agreement. I think it is of the utmost importance to sleep well at night and have that as one of the bedrock requirements of my system design. For me, that entails being fully in cash every night, even though that tends to degrade the performance a bit in the long run. I’m just very prone to frustration at being burned by the overnight news cycle, even though it helps overall. So I simply don’t mess with it, and have actually tried to make it an advantage.

Another clarification. It seems that some of my comments are being taken to refer to macroeconomic conditions. Another of my goals, however, is to build a system that is as uncorrelated to the broader market as possible. So, for instance, in the past few days, when the market was moving up, I was almost entirely short, and did okay… which is not to say that happens every time.

Which brings me back to that post I need to write on my basic approach. Suffice to say I find it MUCH easier to gain an exploitable edge in the 1 day time frame than any other time frame I’ve evaluated. And no, I don’t actually know the names of the companies I’m buying or shorting day by day.

The last question is pertinent (though once again, those results are against a system I don’t actually trade)… I’ll probably need to address it at some point as well. But I think the answer is roughly this: 1) fast-trading systems degrade with the amount of capital invested, so my technique would probably be terrible for a mutual fund; and 2) it took me a couple thousand hours to get here, so though others may be able to get there much more quickly, it is probably a reasonable barrier to entry for the average personal investor/trader.


  1. Steve
    Oct 1, 2007


    I’ve loosely followed your web site for quite a while (since Biblical Horizons was hornes.org/biblicalhorizons), and I have enjoyed the trading posts immensely. This last post drew me in enough to venture a few comments.

    In response to some of Rusty’s comments: it may not be worthwhile to argue too much about the efficient market hypothesis, but I think there is good reason to believe that it does not largely hold for the financial markets. (I have a number of friends who are professional traders—I haven’t done a poll, but I think I can guess the direction they would go.) One of the foundations of economics is the assumption that people are rational actors; this is a great assumption as assumptions go, because you can make almost no formal conclusions without it. However, the truth is that people are _anything_ but rational, professionals and laymen alike. As an example, consider the current housing market. How many people claim to want to sell their house (i.e., they list it on the market) but refuse to sell for anything except the listed price, even though their house has been on the market for months? I’ve known people who flat out refused to sell their property until they could get what they paid for it; this is not rational behavior. The same behavior was seen widely as the dot-com boom started to bust.

    Consider the financial markets in the last couple of months. Were the prices before the meltdown rational? What about after? I would argue that a significant component of each is “collective perception”: everybody seems confident, so I’m confident; once a few begin to panic, it spreads like wildfire. My own analysis is that most of the collective error was before the meltdown, but I have heard people talk more recently about fairly good credit investment products that nobody will touch, just because they’re scared of them now. I think that people collectively reinforce each other’s ideas of good or bad, risky or not risky, going up or going down, and that this leads to strong trends of collective irrationality (even though each individual thinks they’re being rational based on what the people around them are doing).

    I recently read “Way of the Turtle”, written by one of the original Turtles, Curtis Faith. It’s a fascinating read, not only from the technical trading system point of view but also due to the historical narrative. It turns out that a lot of the original Turtles didn’t make money using the system, although some did very well. What was the reason? The ones who failed didn’t follow the system; they made “judgment calls” based on their instincts (read: emotions) and it destroyed their profitability. Some even claimed that the successful turtles had been given secret rules that the others hadn’t been taught.

    This is really inappropriately long, so I’ll address a few other issues briefly. I think a lot can be said about how to invest when you think the market is significantly irrational (e.g., tulip mania), but that would take too long. I would only wholeheartedly agree that you should be careful shorting unless you believe that the market correction is imminent; good ideas and bad timing can easily lead to bankruptcy. Regarding why others like mutual funds aren’t doing this: there are a lot of factors that go into whether an idea works or not in a fund setting. One is drawdown; funds simply cannot tolerate the kinds of significant drawdowns that historically successful systems have. Investors leave, managers get fired, it just doesn’t work. Second, as far as mutual funds go, the long only restriction really limits what you can do. Third, economies of scale can go either way. A lot of trading is only successful with a lot of money (and sometimes a lot of leverage). On the hedge fund side, take quant funds as an example. The final factor is that the “respectable” position in many groups (e.g., academia, mainstream media, investment advisors) is to dismiss technical strategies. “Noise traders” is just a little derogatory, I think. But maybe that’s a good thing; you are right that if too many people attempt the same strategy, the available returns vanish rapidly.

    Jay, I’d love to chat with you sometime about how you’ve developed your systems over the years. As I have the time, I’d like to start playing with some ideas of my own (stocks, futures maybe). My wife and I just moved to Dallas recently (attending Park Cities Pres.); send me an email if you’re interested in corresponding at all.

  2. COD
    Oct 2, 2007

    I tend more to the Rusty theory, diversified, low cost index funds that I don’t have to think about. Reading Bernstein had a big impact on me. I simply don’t have the desire to put in the kind of time you have to beat the market. I’m not convinced that I could beat the market even if I did try. I’d rather take the market return and spend my time on stuff far more pleasurable than financial analysis.

  3. Jay
    Oct 3, 2007

    Chris, I have to admit, I love working on my trading. I didn’t discover this until my early 30’s, but it is a real passion for me. However, I can completely understand not wanting to put time into something that is drudgery for unspecified, and potentially non-existent, gain.

    However, I get the feeling that you and a couple others have in mind a specific notion of what “beat the market” means. And I’m guessing that correlates with what you feel is a worthwhile investment of time.

  4. Steve
    Oct 3, 2007

    It is indisputable that “beating the market” is not easy (what is it, something like 70% of mutual funds underperform their targets?). So for almost everyone, I would say the lowest cost index fund is the way to go (some people would advocate international diversification, but I don’t have an opinion on that at the moment).

    As an engineer/computer scientist with an interest in finance, using my technical skills to try and develop strategies that outperform the market is as much of a hobby or a game than anything else. With a great way of keeping score…

  5. John
    Oct 5, 2007

    The efficient market concept has been disproven by many including Louis Navellier (dissertation on alpha), Al Frank (Prudent Speculator newsletter), Peter Lynch (Fidelity Magellan), and Warren Buffet (Berkshire) who have all consistently beaten the market for periods over 20 years. These folks have invested large sums of money with their strategies and investments published.

    I prefer the long term investing strategy of Buffet taking advantage of the market psychology that causes some stocks to be priced well beneath their true value. However, this does take considerable time to implement so now I’m finding myself attracted to the leveraged index funds such as QLD and SSO.

    Jay, is there some tool you use to ease the tax preparation burden of technical trading?

  6. Jay
    Oct 5, 2007

    Absolutely. I couldn’t do it without Tradekeeper.

    It spits out a flawless schedule D that can be imported into your tax software such as Turbo Tax or TaxCut. Last year, my schedule D printed out at about 40 pages and took about 20 minutes to prepare. One of those amazing bang for the buck applications.

  7. Rusty Burlingame
    Oct 8, 2007

    Wow. A 40-page schedule D. You’re really doing this, aren’t you? Absolutely amazing.
    With the point of view I have had for years about the financial markets, prior to your posts I would have viewed such claims as I would claims of cold fusion or a perpetual motion machine. The fact that you’re honest, not trying to sell me something, and are actually still doing this with success for this long completely blows my mind. I am really going to have to think about this some more.
    I’m trying to find my copy of Random Walk, which is either not yet unpacked or was sold at Half Price long ago.
    A few posts in the comments that I would take issue with. One seems to be saying (as I read it) that the existence of irrational people means markets can’t be efficient. Far from it – who do you think is losing all that money to Jay? :) Efficient markets just mean that the irrational and clueless will take a beating in the market, or (alternately) that those with knowledge of the market can make money exploiting their knowledge.
    Re: Navellier, Frank. I don’t know these gentlemen and will have to do some investigating.
    Buffet invests in companies where he thinks he can add value, and then actively gets involved in the management of those companies. That’s investing and managing, not trading. I’m not sure how that would disprove efficient markets.
    Taking nothing from Peter Lynch, who I only know by name, if the efficient market/random walk theories are true, some fund managers will indeed beat the markets. Some will do it for several years. There are thousands of funds, so this isn’t hard to believe. Out of thousands of competent fund managers all rolling the dice bunches of times, somebody will roll 7 many times in a row every once in a while. This person typically becomes famous for their management style. :)
    Harbor Capital Appreciation (HACAX) is one the 401(k) investment options at Southwest. For years after I started, the fund selection committee pointed out how HACAX had beaten the S&P 500 for something like 7 years running. Then it turned in a real stinker of a year a few years ago – so bad it almost wiped out all of years it was ahead. Same fund manager, I believe.
    Hopefully this isn’t too long. I find this terribly fascinating. A few questions for thought.
    If you were trading a system, how would you convince yourself you’re not trying to find patterns in random data?
    There are at least tens of thousands of active traders, I’ll guess. How would you convince yourself you’re not just lucky like the dice roller mentioned above?
    If something changed in the market and your system stopped working, how would you know?
    Suppose I came up with a system, Jane, uncorrelated to Tarzan, but with the same returns when run over the last 79 months. How would I choose one to use for trading in the future? What basis would I have for expecting one to do better than the other in the future?

  8. Jay
    Oct 8, 2007

    Rusty, you’ve posed some good questions… I’d even go so far as to say the right questions. I’ll give a short answer to a few of them, though they deserve more careful treatment, and I’ll try to address some of the details in future posts.

    In my opinion, backtesting is where it’s at. You have to really get to know your system, and yet you also have to remain a bit distant. My approach to trading has been to try to become excellent at backtesting, balancing that “getting to know” and “remaining distant” paradox.

    You have to keep some distance to minimize (I don’t think it is possible to entirely avoid) backfitting the data. My actual system, which, oddly enough, I call System, went through a pruning process the past few months. I started with some good ideas but not enough detail. By April of this year I came out of that beta phase with a solid system, well articulated, that traded reasonably well. But I knew I had far too many variables with far too much risk of backfitting. So version 2 of System finally came out around August and had about 70% fewer variables but with similar performance characteristics. It’s like you have to unfocus your eyes and see the trees not the individual leaves.

    But you have to have an intimate knowledge of what to expect from your system and what you are going to do in response to the unexpected.

    Regarding luck versus a real (e.g. profitable) system, generally the number of trials is considered crucial to ascertaining the validity of a sample (as well as the distribution of the results within the sample). That’s the huge advantage of trading quickly. For all of 2007, I’ve held 10 to 15 positions each day and am entirely in cash at night. You get a lot of samples quickly, and then it’s more a matter of waiting around for new market conditions and gathering yet more samples.

    So if you had a method to tilt the odds of rolling a 6 on a die up to about 20%, and you had 10,000 samples to prove it, you’d start to feel confident, until you changed the surface on which the die was rolling, and then you’d have to collect more samples and see where you stood.

    But, once again, I have found the most important aspect of all of this is being very good at backtesting. And, to be perfectly honest, it took me probably 3 years and change to get where I had any demonstrable skill at doing useful backtesting.

  9. Steve
    Oct 9, 2007

    Rusty, you make some great points. To your first, I would agree that the presence of irrational people is not inherently opposed to the idea of efficient markets, but I think a little more nuance is helpful. Take an example: IBM is currently trading somewhere around $117, and let’s assume that this is the efficient, rational price. I buy a large amount of IBM stock, and send out email spam with fake rumors about how IBM’s profits are going to skyrocket. Lots of people believe my fiction, and the price goes to $120. A few more people notice that the price is going up, think that there must be some information in the market that they are unaware of, and also buy IBM. End of the day, IBM is up to $125. Is this also an efficient/rational price? In some tautological sense you can say it is, because it is the sum total of everyone’s beliefs and knowledge in the market (but, again, I think this is more tautological than anything). The next day, I’m exposed as a fraud, and the price is back to $120. Back to efficiency/rationality?

    So I think there are two important considerations with respect to efficiency/rationality. First is time scale. I believe that prices have both rational and irrational components, and that the rational component generally operates on a longer time scale than the irrational component. This implies that a buy-and-hold investor is not negatively affected by “noise traders” or speculators, even if their behavior is irrational (not that I think it always is). In the long run, the noise/irrationality tends to cancel itself out.

    The second factor is whether the actions of irrational actors in the market tend to cancel each other out. If they do, then perhaps then net effect really is zero, on all time scales. With respect to this, I would say that human psychology is much more prone to exhibit behavior patterns that reinforce, rather than cancel out. People look at what the people around them are doing, and they do the same thing; it feels safer. Monkey see, monkey do, right? So I think that we really do see short time scale irrationality because of collective perceptions that are not accurate, or collective behaviors that tend to propogate.

  10. Steve
    Oct 9, 2007

    I highly recommend the Way of the Turtle book for its discussions of building systems, benefits and perils of backtesting, etc. With respect to building robust systems, here are a couple of important guidelines.

    First, you need to have tested your system against a wide variety of market conditions. These include high and low volatility markets, and trending (up or down) and flat. It is also beneficial if you’ve tested against catastophe scenarios (perhaps not as bad as 1987, but bad). If your system does well in all of these situations, then you’ve covered a pretty broad part of what the future might be like.

  11. Steve
    Oct 9, 2007

    A second consideration is the statistical properties of your trade return distribution. If your testing shows tons of small negative returns and a few super huge returns that generate all of your profit, that is a big concern, because who knows if those few big hits are likely to happen in the future? On the other hand, if your testing shows that you make positive returns (some small, some large) most of the time, with only relatively few (and small) negative trades, then it is more reasonable to expect that your system will continue to be effective in the future.

    This would be one of my concerns with some of the trend following systems, like the turtle system. From my understanding, they tend to have too great a reliance on rare positive events to generate their returns.

    As Jay mentioned, taking care to avoid over-fitting is really important as well. Jay, how many variables are we talking about in your system? And how far back do you usually backtest?

  12. Steve
    Oct 9, 2007

    Sorry for the excessive posts, but my daily read of the New York Times picked this up, which is relevant to the issue of erroneous collective perceptions: