Team Rankings, a business I started in 2000, takes a quantitative approach to understanding and predicting sports events. Over the years, we’ve built products targeted at a number of different groups within the world of sports enthusiasts.
Two products we’ve built — one to help people beat the Las Vegas line, another to help them win their NCAA Tournament office pools — are both data-driven tools to predict outcomes of sporting events. However, they reflect two very different sides of the world of quantitative decision-making.
The Skilled Opponent
When I was working at PayPal in the early 2000s, I was spending most of time building predictive models to detect fraud. At home one night, I decided to see if I could apply a similar methodology to sporting events to use on Team Rankings. My system incorporated a large number of inputs — how each team had been playing, what their respective strengths were, how far each had travelled — and used them to find a blackbox (i.e., really complicated) predictive model to assess the likelihood of a bunch of events related to a game. Among these: how likely is each team to win, how likely is each team to cover the point spread, and what is the expected final score.
It turns out that this approach works well enough to consistently beat Las Vegas. For the current models — which use essentially the same methodology — here are the results for college basketball since 2008-2009:
For these two bet types — against the spread and over/under — Team Rankings’ picks have been correct 5214 times and incorrect 4452 times. The odds of this level of success by a dart-throwing monkey or TV commentator — who would expect to be correct 50% of the time — is less than one in a billion.
However, while 50-50 constitutes “break-even” from a statistical perspective, the break-even point for a Las Vegas gambler is higher. The house takes an extra cut: in most cases, a gambler bets $110 on a game to win $100. That means that you need to be correct at least 52.4% of the time just to break even.
Team Rankings’ results beat that threshold too: 5214 wins and 4452 losses constitutes a 54% winning percentage. And that means that someone who gambles strictly based on Team Rankings picks will make a profit over the long term. Bet $100 on each game, and you’ll make an average of $7,000 over the course of a season; bet $1000 per game, and your expected return will be $70,000.
That’s a terrific return, and I’m proud of it.
But still, it begs a question. Team Rankings has a great team (I’m a tiny part), some cool technology, and lots of data, and the best we can do is 54%? That’s barely better than a coin flip! It means an expected return of about 3% on each bet — a good investment, but hardly earth-shattering and a far cry you get from the “100% GUARANTEED PICKS!!!” you see on sleazy gambling picks sites.
[Disclosures: Team Rankings' picks are for entertainment purposes only. Of course. And while I'm capable of thinking like a scientist (constantly skeptical), I'm also capable of thinking like a writer (cherry picking to prove my point). These are among the better-performing of our models, but overall our results are likewise very strong and profitable.
At some point, I'll take Anthony from Kaggle up on his offer to run a contest to predict games, so people smarter than I can have a crack at it. If we did that, we might improve the number to 55-56%.]
The Unskilled Opponent
Some of Team Rankings’ recent product innovations tell a very different story.
For many years, we’ve provided odds and analyses related to the NCAA Tournament. Our products have included tools to match up two teams, along with probabilities for each team to make each round of the tournament. So, for instance, you can see the odds of Gonzaga making the Sweet 16, the Crazy Eight, the Final Four, etc.
A few years ago, I realized that those features could help people win their NCAA Tournament pools, but only if they used our numbers in the correct context. Say, for instance, that Kansas is the team most likely to win the tournament, with 20% odds, but Kentucky is just behind at 19%. Kansas would be a better pick to win than Kentucky, right?
Not necessarily. Let’s say Kansas is a really popular pick, but Kentucky is not. 50% of people are picking Kansas to win and only 10% are picking Kentucky. If you picked Kansas, you’d have a slightly better chance of being right on the winner, but because you’d be competing head-on with many more people, you’d almost certainly have a smaller chance of winning your pool. In entrepreneur-speak, picking Kansas would be like trying to build a mobile photo sharing app in 2011, a group buying site in 2010, or a Facebook game in 2009: good business or not, you’re setting yourself up for a lot of competition.
So, with some help from Brad, we built a tool that allowed us to answer the question of what picks would maximize someone’s odds of winning their pool. To do that, we combined teams’ win probabilities (which many others do) with data on how many people picked each team (which no one else does). We simulated millions of tournaments, randomizing both the outcome of the games and the picks made by people in your pool.
The end results are pretty astounding. Our top brackets — based on a moderately conservative set of assumptions — had an expected return on investment (ROI) of around 800%. This bracket, for instance, would have had about a 0.9% chance of winning a 1000-person pool, nine times higher than the average participant. By picking as our champion Ohio State — a very strong team not given enough credit by the public — our odds would be much better than if we picked Kentucky, the best team but also one strongly favored by the general public.
Our 10-person bracket looked quite different: with a smaller number of people to beat, our simulations indicated it was a stronger strategy to pick mostly favorites, with Kentucky as the eventual champion. This bracket had a less ridiculous but still quite impressive 186% expected ROI.
Kentucky wound up winning the national championship. Most of our users in small pools did quite well and won their pool; our users in large pools did not. We won’t be successful every year, but over time our results have been very strong. And this approach works: I feel comfortable with the assertion that the average yearly return for our strategy will be at least 200%.
How do we synthesize all of this and bring it back to the non-sports world?
A bet on a game at a Las Vegas sportsbook and an entry in your colleague’s NCAA tourney pool both constitute a wager on sports. However, the underlying market dynamics could not be much more different.
Las Vegas is more or less efficient: if lots of people bet on one side of a game, they’ll update the odds. In contrast, your friends and colleagues in the NCAA pool are likely making impulsive decisions that are economically irrational.
Hence there’s one world (Vegas) where 3% returns are celebrated as amazing wizardry, and another (friends’ pools) where you can have expected returns of 900% without anyone really paying attention.
If you’re looking for a financial return, though, there’s a catch. There’s only one NCAA Tournament per year, so your opportunity to make money is limited. In theory, you could enter lots of pools with distinct but complementary undervalued picks, perhaps giving yourself a 50% chance of winning with only 15-20% of the pot. But you’d be putting your marbles in one basket.
By contrast, each year there are thousands of regular games on which you can bet against Las Vegas. Adding those together can yield a solid expected ROI over the course of a year. Quantitative hedge funds generally take this against-Vegas approach: they find short-term inefficiencies, and bet on them again and again.
Though difficult, Team Rankings and hedge funds show that betting in Vegas-style almost efficient markets can be extremely profitable. From the actor’s perspective, it’s a bunch of bets in established markets with positive ROI. Yet from the world’s perspective, it’s a bunch of minor market efficiency improvements, but a world that hasn’t really improved in any meaningful way. In other words, something that’s more Wall Street than Silicon Valley.
By contrast, the pool of your buddies — while itself not a world-saving problem — represents a far larger and more profound inefficiency. The large-scale decision-making of Las Vegas and Wall Street is close to being economically efficient, but one-off decision making by individuals and businesses is not. Most choices — companies deciding whom to hire or where to put resources, government choices on how to run cities and schools, individual choices on where to invest, and which teams to pick for your NCAA pool — are made haphazardly and could be improved a lot.
It’s tougher to build a business to address these massive inefficiencies: to build something large, you need to find important, quantifiable decisions that have associated data and haven’t already been examined properly. That type of problem is more interesting, has more upside (+800% vs. +3%), and can be much more impactful than its Vegas-style alternative. And it’s why, without hesitation, I choose Silicon Valley over Wall Street.