Algorithmic Trading

'Algorithmic trading' is trading done by computer programs (algorithms) that pick up on trade-worthy bits of information from streams of market data and financial news. According to a recent article in The Economist, algorithmic trading accounts for a third of share traded in the U.S. and analyst project that algorithmic trading in some form or fashion will grow to account for a majority of trading in the coming years.

One of the advantages to algorithmic trading is sheer speed. In trading, especially in 'statistical arbitrage,' even a few milliseconds can mean the difference between exploiting a profitable opportunity or missing it altogether.

There are problems with this type of trading. One of the problems is that the algorithms may mis-interpret data or not have enough or the right kinds of information to make the proper trading decision. For example, as the article points out, does "surprise" mean the price is going up or down? One solution to this is to make sure that algorithms properly combine pieces of information (e.g., matching "surprise" found in a news piece with relevant price data). Another solution is tagging the news items fed into the computers (but then how do you know whether the tags are correct?) -- which is being done by financial news providers like Dow Jones.

I believe that there are more profound and more troubling problems than those pointed out by the article. First, we might wind up seeing similar problems that we got with 'programed trading' (which is not necessarily the same as algorithmic trading) that contributed to the October 1987 crash that rocked the financial market; i.e., you can get massive herding effects since many algorithms (even if superficially different from one another) will often behave in similar ways to each other.

The second problem is that algorithmic trading is essentially data mining. Now, most people in the business world think that data mining is a good thing. In some cases, it can be on a practical level. But, to thinking mathematical statisticians, data mining is a highly dubious practice. As Nassim Taleb might put it, algorithmic trading might be mining data to confirm whatever 'white swans' that one hopes will provide trading profits. But what if a black swan happens? It's doubtful that trading algorithms can pick up on black swans any better than their human trading compatriots can.

A third problem -- which I won't elaborate on since there are mountains of books written on this subject -- are the ones faced by anyone working in artificial intelligence ... which algorithmic trading is sort of aiming at.

Thin Sliced Elections

The NBER (National Bureau of Economic Research) Digest's latest issue (June 2007) has an article -- TV Appearance and Electoral Success -- that I found both fascinating and disturbing at the same time. In their paper, Thin-Slice Forecasts of Gubernatorial Elections, Daniel Benjamin and Jesse Shapiro, conducted an experiment on a group of Harvard students to see whether or not the physical appearance of political candidates alone could lead to correct predictions of the outcome of elections. These subjects were shown 10 second clips of televised debates of candidates for governors in several U.S. states. Some subjects were shown clips without sound while others were either shown clips with muddled sounds or with full audio.

The researchers found that those who saw silent videos had greater success at predicting who would win the elections (58% of picks actually won their elections) compared to most other factors used to make electoral predictions. The students who made their predictions after seeing the videos either with muddled or full sounds could do no better than had they randomly guessed who would win (ranging from 52 to 48% success rates). Furthermore, purely visual forecasts were considerably more accurate than electoral predictions based on other measures such as per capita income, unemployment rates, and state fiscal health.

The results of this research is consistent with similar experiments carried out on subjects outside the U.S. to find out what factors people actually used when picking their political leaders. For example, two studies carried out in Europe (Romania and Finland) found that subjects basing predictions based on physical appearance tended to do better than those who had based their predictions on other seemingly more 'politically correct' factors such as competence in economic and policy matters.

Contrary to expectations, adding policy information to the predictive mix seems to worsen the chances of making correct predictions. The findings of this body of research may explain why 'experts', "who are highly informed about and attentive to policy matters, are often found to perform no better than chance in predicting elections" (and, frankly, experts often do worse than chance in making predictions and this mis-predictive 'ability' is applicable to other fields as well).

These findings are consistent with the idea of 'thin-slicing' which was popularized by journalist, Malcolm Gladwell, in his best-selling book, Blink. Malcolm Gladwell's contention is that making rapid, intuitive decisions are often not much worse than -- and may often be superior to -- making more deliberate decisions. Although I really admire Malcolm Gladwell and, to some extent, agree with this idea, this is the part that I (and I'm sure others) find somewhat disturbing. As I recently wrote up in a blog post on the Travelers' Dilemma, those who make snap decisions tend to make more emotional or random rather than rational or strategic decisions. Thus, while in a positivist sense 'thin-slicing' may be the method used to make decisions in political elections (and in other situations), in a normative sense this type of decision-making is a recipe for choosing incompetent and disastrous leaders.

Benjamin and Shapiro's research seems to contradict econometric models of elections done by researchers like Ray Fair, who found that economic and other policy factors can play significant roles in making electoral predictions. Having said that, all three economists agree that incumbency is the factor with the most predictive power in trying to forecast electoral results. (Campaign spending is another factor that has great predictive power.)

The research discussed in the NBER Digest both complements and contradicts research carried out by Philip Tetlock, a political scientist at U.C. Berkeley. Prof. Tetlock -- most recently in his book, Expert Political Judgment: How Good Is It? How Can We Know? -- eloquently debunks the predictive 'abilities' of experts. Tetlock's research seems consistent and complementary to Benjamin and Shapiro's findings in that all three would agree that experts are bad at making predictions. Tetlock's research is at odds with the reasoning laid out in the Digest in that Tetlock, as Isaiah Berlin would have put it, believes that 'foxes' -- those who are intellectually curious and have wide-ranging interests (i.e., knows at least a little about a lot of things) -- tend to do better than 'hedgehogs' -- those who are focused on a few matters (i.e., knows (a little? a lot?) about a few things ... usually one thing) -- in making predictions. Benjamin, Shapiro, et al., seem to suggest that such a distinction may not matter at all -- i.e., it really doesn't matter whether or not an expert is Berlin's intellectual fox; in fact, narrow-minded hedgehogs -- so long as they are focused on the right 'few' things (such as visual cues of personal appeal in this case) -- may make better predictions than Tetlock's foxes.

Perhaps the most fascinating aspect of Benjamin and Shapiro's findings -- beyond how the electorate is susceptible to making snap decisions based on 'looks' -- is the complexity of how people thin-slice during elections. Although the experimental subjects did predict better with purely visual cues (without sound or policy information), these visual-based predictions do not seem to be simplistically correlated to factors that are obviously associated with physical attractiveness.

Instead, there seems to be a sense of some intangible quality -- we can call it 'charisma' -- that somehow can be picked up visually (and perhaps is muddied a bit when we start considering the substance of what the candidates have to say) that seem to be the key factor in the predictive success of those who based their decisions on silent videos. Thus, personal charisma of a candidate -- as vague a concept as that may seem -- is a superior to the predictive power (or lack thereof) of more concrete factors like policy matters and competence.

As Benjamin and Shapiro aptly conclude, "Adding policy information to the video clips by turning on the sound tends, if anything, to worsen participants' accuracy, suggesting that naïveté may be an asset in some forecasting tasks."

eBay 'Sniping' and the Power Law

'Sniping' -- where a bidder waits until the last moments of an auction to place a bid -- is a tactic used by some eBay auction aficionados to gain an upper hand in bidding for an item.

According to an article (June 23, 2006) in the New Scientist, two South Korean physicists, Byungnam Kahng and Inchang Yang, found that a power law equation explained the observed patterns in eBay auction bidding. The power law reflects the fact that most of the bidding for items tend to take place near the end of an auction. Thus, according to their article in Physical Review E (vol. 73, p 067101, 2006), 'sniping' is a rational strategy to employ in eBay bidding.

This research builds on the work of experimental economist / game theorist, Alvin Roth. His article in the American Economic Review (vol 92, p 1093, 2002) came to similar conclusions.

Traveler's Dilemma: 'Irrational' Game Theory

I just read an interesting article in Scientific American on game theory -- The Traveler's Dilemma (June 2007) -- written by Kaushik Basu, the economist who created the aforementioned game paradigm. The Traveler's Dilemma, as the Sci Am article describes it, is very similar to the Prisoner's Dilemma except that the payoff matrix of the typical PD scenario is a kind of subset of the payoff matrx of TD (specifically, when the players are restricted to the lowest two strategic choices, TD is equivalent to PD).

The Traveler's Dilemma (TD) basically involves the following scenario: Two players can individually choose a figure (usually couched in terms of some monetary amount or price) without in any way conferring or coordinating with each other. If both players choose the same amount, then they will each be paid that amount (hence, like Prisoner's Dilemma (PD), there are meta-incentives to 'cooperate'). But if they choose different amounts, then the player who chooses the lower amount will get that amount plus some reward/bonus while the player who chooses the higher amount will get the lower amount minus a penalty (which is usually the same size as the reward). Thus, as in the better known PD, the TD's payoff structure provides myopic incentives for players to 'defect' and try to undercut one another.

Under the standard assumptions of 'rationality' (I put it in quotes here because I find economists' and game theorists' definition of rationality to be too straight-jacketed and narrow) and the logic of backward induction (a solution concept in game theory), the players in TD should defect and choose the lowest number possible (similar to the result in PD). Of course, this result is to the deteriment of both players since both of them would have gotten a higher payoff if they had 'cooperated' and chosen a high amount.

Prof. Basu argues -- providing ample experimental/empirical evidence and common sense observations -- that, in reality, the theoretical result of TD does not hold. In practice, most players who participate in TD inspired game theoretical experiments tend to choose figures in the high range of options. These empirical results fly in the face of the standard assumptions of 'rationality' (although, as Prof. Basu helpfully points out, it is consistent with a sort of meta-rationality) in game theory and economics.

The article makes some other interesting observations. One of them are the results of an experiment carried out by economist/game theorist, Ariel Rubinstein. Rubinstein found that people who based their decisions in Traveler's Dilemma style experimental games using strategic reasoning or more formal 'rationality' took the longest time to respond to prompts. Conversely, those who made decisions on spontaneous 'emotional' or untrained intuitive responses or who made "random" (i.e., inexplicable or perhaps even crazy) choices tended to take the least amount of time in making decisions.

Another interesting observation made in the Scientific American article is whether or not the economist's/game theorist's concept of rationality needs to be modified and expanded. Prof. Basu argues that the standard notions of rationality are inadequate to explain the empirical tests of the Traveler's Dilemma as well as the Prisoner's Dilemma (especially, iterated PD). The problem that the article zeros in on is the assumption that rationality is "common knowledge" in its philsophical, formal logic sense of that term (i.e., that each of the players in TD know that the other player will act 'rationally' and they are each know that the other knows). Prof. Basu argues that people may not always conform to this standard assumption, especially in repeated games such as iterated (or repeated) Prisoner's Dilemma.

As the article points out, perhaps these observations from empirical tests of TD and PD will lead to a rethink of game theoretic logic. Perhaps there is a "meta-rational" consideration(s) that lead people to (without direct collusion and communcation) choose mutually beneficial options. This type of reasoning should give us some reasons to hope in humanity at large.

Timber! Money Can Grow on Trees

A few years ago, I came across an alternative asset class that institutional investors -- especially university endowments -- that struck me as being really out-of-the-box: timberland. At the time, I asked myself, "Why are sophisticated investors investing in trees and forests?"

A recent New York Times article, For Some Investors, Money Grows on Trees (May 27, 2007), answered many of the questions I had about this alternative investment. In a nutshell, investors are counting on revenues from sales of timberland products to lumber, paper, and other companies, along with potential gains from the underlying real estate. Rather than investing in individual lots (which would make little sense for large institutional investors like pension funds and university endowments), investors invest through TIMOs (timber investment management organizations) and timber REITs (real estate investment trusts).

Historically, investing in timber has done well. An index of returns on timberland investments since 1986 (when the index was created) to the first quarter of this year rose at an annualized rate of 15.09%. In the last three years, the return was 14.63%, which is higher than the returns on the S&P 500 over that period (12.25%).

The most appealing aspect of this asset class is that it has had low correlation with the performance of stocks and bonds. 'Low correlation' is important to risk management under conventional financial economics portfolio theory. I should note that that I am usually highly skeptical and suspicious of claims of 'low correlation' between asset classes and markets since 'correlations' are (a) dynamic, and (b) there might be less obvious links between investments that simple measures of correlation don't pick up. However, in this case, this idea does seem to pan out at this point in time.

Note: The best books I'm aware of dealing with the role that alternative assets can play in managing an investment portfolio are the two books written (thus far) by David Swensen. I'm not sure if Yale's endowment invests in timberland, but I would be shocked if they didn't. I am aware of other university endowments that do invest in timberland, e.g., Caltech.