The Epidemiology of an Internet Business

Imagine life as a bacterial plague.

Your world is pretty simple. You aren’t motivated by fancy cars, attention from attractive people, or the respect of your peers. All that matters is the propagation and survival of your species.

But being good at both propagation and survival is tougher than it sounds. Say you’re really successful at spreading. You quickly infect everyone in the world. Awesome news, right? Well, maybe not: one of two bad things might happen. Either you’re deadly and the source of an epidemic that destroys the world’s population… and with it your source of places to hang out! Or you aren’t deadly, and everyone’s immune system figures out how to get rid of you at once… and then you also don’t have anywhere to hang out.

Successful plagues therefore need a more nuanced approach: propagation at the right speed, a non-lethal effect on targets, and a means of mutating to adapt to changing conditions and antibodies.

As it turns out, Internet businesses are a lot like bacterial plagues. In both cases, results vary widely.

MySpace excelled at propagation: it launched in August 2003 and grew to 100 million users within three years. But its survival mechanisms have been less effective, meaning that those 100 million users have largely managed to expunge the MySpace bug from their system. Hence MySpace usage in 2012 is a small fraction of what it was in 2007.

Some of the first Facebook apps were even better at propagation and even worse at survival. Circle of Friends, which I developed, launched in September 2007. By November 2007, without any semblance of press, we were adding half a million users a day. Today, Facebook reports that Circle of Friends has 300 monthly active users. Not three hundred thousand. Three hundred.

By contrast, Facebook itself has excelled at both propagation and survival. Eight and a half years after launching, Facebook has around a billion monthly active users. To get there, they’ve done astoundingly well at both spreading and maintaining their role in existing people’s lives.

Most aspiring Internet entrepreneurs know about viral coefficients. But a viral coefficient mainly reflects the propagation piece of Internet epidemiology.

The other half, equally important for a sustainable business, is frequently referred to as retention. This number is usually expressed as the percentage of registered users active in the last month. Five million registered users, one million of whom have on the site in the past 30 days? That’s a “retention rate” of 20%.

In the world of epidemiology, a retained user is like a person still being affected by bacteria.

The retention number doesn’t explain a lot, though. On the Internet, are those 20% super active, or are they only using the product once or twice a month? Were those 20% the same 20% as the month before?

It’s possible to look at the entire system from a fairly simple epidemiological perspective by instead asking two key questions:

1) What percentage of users this month (or week) fall into one of a few key categories? For Facebook, those might be inactive users (no visits), lightly active users (1-3 visits), moderately active users (4-7 visits), highly active non-contributors (8+ visits, 0-1 comments/posts), and highly active contributors (8+ visits, 2+ comments/posts).

(I’m sure there are biological parallels reflecting the progression of bacteria through someone’s system, but my lack of knowledge means crafting an analogy would almost certainly entail me putting my foot in my mouth… if I haven’t already done so.)

2) How are users transitioning between the different states? For example, what percentage of last month’s inactives stayed inactive, and what percentage jumped into each of the other four categories?

Here’s what these numbers might look for Facebook (note that these are completely fictional):

In this world, Facebook has a billion active users, but still another half billion inactive users. The billion users are divided relatively evenly between the four active categories.

And here are the transition probabilities between states of activity. This (again fictional) matrix defines Facebook’s epidemiological framework:

I make the fairly conservative assumption that 90% of inactive users will stay inactive, while 4% will become lightly active, 0.4% highly active contributors, etc. Meanwhile, nearly half of highly active contributors will stay in that bucket next month.

If this model is correct, here’s how Facebook’s world will look the next two months:

As you can tell, with these transition probabilities, Facebook’s bucket stats will stay stable.

So let’s see what happens if we change the transition probabilities just a bit. In the matrix below, Facebook gets a bit better at keeping lightly active and moderately active users from becoming inactive — moving from 12% and 5% to 4% and 2% — and also better at keeping highly active contributors in the high contributor bucket — increasing from 48.5% to 75%. Everything else stays the same.

The effect seems significant but not overwhelming. For next month’s stats, here’s what we see:

In other words, a slight decrease in inactive users, from 500 million to 472 million, and a larger increase in highly active contributors from 200 million to 260 million.

But like viral growth, that difference gets a lot bigger as its effects fan out. Here’s what things look like two years from now:

Inactives have been cut almost in half; super-actives have more than doubled.

And this doesn’t take into account the effect of that increase activity on other users — both those already on Facebook and those who aren’t.

In other words, small changes in transition probabilities can have large long-term effects on a business’ usage patterns.

So how do you define the buckets for your business? The details aren’t really that important — I could have defined “lightly active” as 1-4 visits a month. What’s most important is separating users who are meaningfully different from a business perspective. So if a tiny number of contributors are generating 90% of the content for everyone else, it’s important to give them their own bucket.

It isn’t just users who matter in an ecosystem. Tweets and the way they travel around are extremely important to the way Twitter works: a minor change in the way retweets work could completely change Twitter’s tweet ecosystem. Even without any changes on Twitter’s end, the dynamics could shift because of changes in users’ culture.

Once defined, the transition probabilities allow a company to understand how people are moving through their system. More important, they serve as a set of baselines to try and beat. If I were Facebook and saw the probabilities in the initial matrix above, two numbers would stand out to me: 90% of inactive users stay inactive, and 53% of highly active non-contributors will become less active the following month.

In a company Facebook’s size, I’d probably have a person or small team focus on each of those two areas. Each could dive into the data to search for any easy wins, then spend a few months building features aimed at improving those percentages.

In a few months, we’d hope to see better results for those two metrics. By then, the ecosystem of users and technologies will have changed, and some new problems will have surfaced. This is much like bacteria suddenly confronting new medicines or even changes in their host species.

In the search to find a perfect system that can propagate and sustain itself, bacteria have a big advantage: orders of magnitude more scale. There are billions and billions of them, and many, many opportunities for subtle mutations to come up with the perfect plague. Even fast moving Internet companies can’t compare.

On the other hand, Internet companies can be guided by smart, data-focused leaders who, unlike bacteria, aren’t governed purely by randomness. For that reason, I’d rather be the founder of an Internet company than a bacterial plague… even if the company was MySpace.

No Job Left Behind / How to Not Shoot Yourself in the Foot

It was the day after we launched LinkedIn’s Jobs product in early 2005. Matt Cohler, the GM of our the new jobs product, was giddy. A day into the product’s life, companies were already posting hundreds of jobs on LinkedIn.

Upon seeing the numbers, Matt made a confident prediction: this would be my second consecutive Internet startup success. LinkedIn had over a million users, and our first path to real revenue — charging hiring managers to post job listings — looked extraordinarily promising.

Matt is frequently correct, and his prediction about LinkedIn wound up being a good one. But the path to success wouldn’t be quite as straight and simple as I think he anticipated.

For the first six weeks, it would be free to list jobs on LinkedIn. That led to lots more postings, but an imbalanced market: most listings attracted very few applicants and thus meant a poor experience for job posters. Still, we had reason to believe this dynamic would shift.

At the end of that period (March 2005), we started to charge $75 for job listings. Not surprisingly, this planned change led to a dramatic decrease in new listings. That drop meant our revenue wasn’t what Matt (who had just left to join a tiny startup called Thefacebook) and others had hoped for.

But that wasn’t the worst problem. Even with a small number of jobs in the system, most jobs were getting no more than a few applicants. That meant that hiring managers and recruiters were shelling out seventy-five bucks and getting relatively little in return; that didn’t bode well for the product’s future.

If employers had a great experience, they’d post more jobs and tell colleagues to do the same, and the market would even out. But with a poor experience, they’d try something else, and our job listing platform (and revenue stream) would suffer.

How could we remedy this problem and generate applicants for our job listings? I leapt at the challenge, which seemed like a great place to apply some interesting machine learning algorithms to an important business problem. We had millions of users; surely we could match at least a few hundred of them with each of the listings.

Within a few weeks, I built up a set of algorithms to find suggested jobs for LinkedIn’s users. Some of the predictions were quite good; others were clearly off-base. Matching users and jobs is a hard problem.

I shared those recommended jobs with my colleagues, and we decided to email our users with personalized job suggestions. Internally, we called the product No Job Left Behind (NJLB), oh-so-cleverly borrowing language from George W. Bush’s first-term education legislation. The first small NJLB batch was sent out with “Mike Greenfield” as the from name; thereafter they were sent out from “Nick Welihozkiy”, our account rep.

The NJLB results were okay but not great: lots of users clicked through on one or more recommendations, and a handful applied to the jobs. By emailing out suggestions to a targeted set of users, we were able to increase job applications by around ten percent.

However, we also got a handful of complaints about our occasionally off-target suggestions. This came to my attention when a Silicon Valley executive or two complained to my colleague Keith Rabois about their suggestions. As those who know him are well aware, Keith is less obliging toward mediocrity than most people. He of course conveyed his friends’ feedback: we were spamming some high-profile people with low-quality emails, in the process undermining the strong brand LinkedIn had built up.

At the time, I brushed this aside. To me, a user was a user, a job was a job, and a powerful friend of a VP wasn’t necessarily any more valuable to LinkedIn than anyone else. So I put in certain filters to make sure executives wouldn’t get job emails from us, and we continued as we had.

Sometimes, ignoring unsolicited advice from bigwigs is the right strategy. Many people building products are too eager to please a few powerful individuals whose opinions are no more relevant than those of an average user. If you’re building a product for mainstream moms, it’s probably safe to weight the opinion of a VC no more (and often less) than that of a typical mother. Most VCs aren’t experts — or even users — of products for moms, so optimizing for their experience is silly.

LinkedIn had a different dynamic, though. The hotshot execs serve as both the aspirational celebrities that “normal” professionals look up to, and the ultimate decision makers for many purchases which affect LinkedIn’s bottom line. And LinkedIn had a strong brand to protect: the site was (and is) seen as a place for high quality people and high quality (if sometimes boring) content.

If we sent an email to a large company CEO suggesting she apply for a job as a software engineer, we’d undermine that brand with the users most important to our product. In other words, Keith was absolutely correct.

Last week I got an email from Glassdoor that reminded me of that whole experience:

Glassdoor job recommendations

I’ve played a bunch of roles in my career, around both data (from growth hacking to scientist) and entrepreneurship (CTO, co-founder, investor, adviser, etc.), and I know from experience that people like me are a pain in the ass to match jobs for.

That said, Glassdoor’s suggestions don’t even have a unifying theme that suggests they’re making a true guess about what I am. Recommendations to be a retail sales rep, visual designer, and QA tester, all in the same list… seriously?

My assumption is that Glassdoor is sending suggestions purely based on my location, and not doing any advanced targeting. If that’s true, they should be framing the user’s expectations more clearly (DJ Patil, my successor at LinkedIn, explains this well in his e-book Data Jujitsu). Either way, the experience for me is poor. I’d guess that with these emails, Glassdoor generates a few additional applications but undermines their long term brand.

At LinkedIn, we realized that and ultimately turned off the No Job Left Behind emails. They carried substantial risk, and weren’t working well enough to justify that risk.

But progress has happened: the data team at LinkedIn revamped my old models, and seem to be producing a much better set of matches than I ever did. As a result they’ve resuscitated the “Jobs you may be interested in” emails, with a quality level that solidifies the LinkedIn brand.

Note that two of the recommendations are for positions at Glassdoor. Summing things up with a clever joke about that will be left as an exercise for the reader.

Developers are Oil, Marketers are Vinegar: Mix Them Wisely

An emulsion is a mixture of two or more liquids that are normally immiscible (nonmixable or unblendable).


Developers are like olive oil. Valuable in a variety of settings. Full of substance and flavor. The good ones are a little peppery.

Marketing people are like balsamic vinegar. Sometimes a little sweet. Sometimes a little acidic. Always capable of adding flavor and zing.

Let’s say you take out your favorite small mixing bowl, and throw in some oil and vinegar. You can do it however you want: oil first then vinegar, vinegar first then oil, a little of each. The end result will be the same: balsamic vinegar on the bottom of the bowl, olive oil on the top.

The chemistry of oil and vinegar problem is surprisingly complicated, and beyond the scope of this blog. If that’s what you’re here for, please have a look at Wikipedia for an overview of the emulsion process and Cook Smarts for some great vinaigrette ideas.

How about mixing developers and marketers?

At Circle of Moms in 2010, we had a challenge and an opportunity. Our Facebook traffic was drying up and we weren’t doing a good job of re-engaging our users with email. Ephraim and I felt that email could be a great channel for us: we’d seen BabyCenter build a business with email as a central tenet, we had a user base hungry for great content, and we knew lots of valuable facts about our users.

We were sending out weekly newsletters with uncustomized, low-quality content. We had tested one or two small changes to those emails, but were still seeing low clickthrough rates and high unsubscribe rates.

Unfortunately, our existing team didn’t seem poised to solve that problem. On one side, we had olive oil: a strong team of developers who loved building out features. But none of them gravitated toward the email problem: improving the content of an email newsletter feels like a product marketing feature and not a technology one. What developer wants to spend all of her time doing text and UI tweaks?

On the other side, vinegar. Sending out millions of emails is a complicated operation, and understanding the dynamics of populating emails is hard. Moreover, actual testing of email content is more easily accomplished by someone who both understands the technology and can be hands on with data. Some of my more frustrating moments were spent with non-technical colleagues, trying to detail exactly how, when, and why we sent different emails to our users. Several marketing people and a product manager were interested in solving the email problem, but none had the technical or analytical skills to really take it on properly.

With two poor options, I wasn’t sure how to proceed. Short of shifting my role to one where I’d spend 90% of my time revamping our email system, I didn’t see a solution.

Fortunately, the solution found me. While recruiting for engineering roles at a Stanford Career Fair, two interesting people came to talk to us. Both knew how to write code and had an engineering degree or two from Stanford. Both were happy to code a little bit, but neither had the skills or desire to be a pure developer. Each was also quite interested in product and data, wanting to work with others across the company. As such, they weren’t interested in any of the developer job listings we’d printed out, but were obviously smart and seemed like they could be great contributors.

I followed up with one of them shortly afterward, and quickly realized she had precisely the skills we needed to solve our email problem. After bringing her in for formal interviews with people across the company, we quickly made an offer to perhaps the world’s first “Marketing and Analytics Engineer.”

The title sucked — clearly I should have consulted whoever coined the term growth hacker — but the end result was terrific. Erin quickly came to understand our codebase, our usage patterns, our product, and the psychology of our users. On the technical side, she could dive in and write code, run a/b tests, and do SQL queries. For more complex technical tasks, she worked directly with a pure developer to implement a new feature. But she also had as strong a product marketing sense as almost anyone else at the company, and spent time with our editorial/marketing team to understand what they were doing and figure out how to productize and optimize it.

Ultimately, Erin would simplify the product to enable inputs from the marketing team, in effect enabling them to better do their jobs. She would also automate and optimize many of the marketing team’s processes, bringing a great set of analytical and technological skills to their part of the company. In the process, she emulsified the awkwardly separate oil-vinegar mixture; it helped us immensely.

The next year, Erin gave us the unfortunate news that she had accepted a product management role at Workday. Fortunately, we had a natural successor: we’d brought Emma on as a summer intern to play a similar role, and Emma quickly accepted a part-time-turning-into-full-time position as a Marketing and Analytics Engineer.

Emma would ably step into Erin’s old shoes. She had done a lot less software development than Erin had, so it took her some time to get up to speed on that front. Fortunately, it’s manageable for a smart and driven person to learn basic coding skills. Like Erin, Emma focused on technology iterations, while working with other developers who built the deeper core technology. The end results were great: Emma served as an interface and emulsifier between our editorial team and our development team, and became the single force responsible for propelling our email product forward.

The term growth hacker is too specific to describe the positions of Emma and Erin: neither spent much time on growth, instead re-engaging users and building products to serve an emulsion of goals. Marketing and Analytics Engineer is a boring title, but a correct one. Both were developers — formally and culturally part of the development team — with goals more focused on business metrics than technology.

To startup companies: I strongly recommend hiring this kind of person, or several of them. Skills are less important than the ability to learn and execute. But please note the last word in the title, engineer: this needs to be someone who will write code. Otherwise, the emulsion is not likely to happen.

To recent college grads: if you want to be a product-y, marketing-y generalist in technology companies, great. People with product and marketing skills are tremendously valuable. But please, please, please: learn to write a little code. You don’t need to be able to build a scalable back-end system. You may just need to be able to change the text in an email and then write a SQL query to see what effect it’s had. Then you become dangerous in a really good way… and hopefully you’ll be able to emulsify with the best of them.

Predicting The Future is The Future

I spent the first four years of my career building predictive models. For PayPal (my employer), knowing the probability that a user or transaction would turn out fraudulent was incredibly valuable. 60% probability this transaction is fraudulent? Automatically block the transaction. 2% probability this large merchant will “take the money and run”? Get an agent on his account, look at his transaction patterns, talk to him on the phone, keep a close eye on him. 0.01% chance he’ll wind up bad? Move on to the next person: this guy isn’t interesting!

So we spent a lot of time building and optimizing statistical models that would estimate risk. Unlike the maligned risk models from the financial world, our models were completely empirical and built from huge data sets — thus likely to be stable and accurate (barring major product changes).

But because of the size of PayPal’s data set, the process of building a model was fairly cumbersome. Say we were looking at $500+ transactions for fraud by the merchant. We’d go back and gather almost all previous transactions in that range. For each payment, we’d gather a huge number of other features, as of the time the transaction happened: how much money, what day of the week, what time of day, how many payments had the seller accepted in the 24 hours prior, what was the largest payment the buyer had ever sent, what was the historical fraud rate on the IP address the seller was using… plus 1000 other things. We’d use those 1000 features to try and predict whether or not the transaction would wind up being marked as merchant fraud.

Armed with lots of data, we’d build a predictive model that could ultimately score any new transaction by looking at those features. Then we’d see how the candidate model performed against an independent test set. We’d repeat a few times to try and build a better model, then deploy the best model onto production. All in all, it would take a fraud scientist at least a few weeks to build out a new model.

Once deployed, that model would score every transaction (or user), providing an extremely valuable way to highlight the riskiest people and transactions in the system. In many cases, we’d identify (and ultimately stop) half of the fraud in the system while blocking or examining well under 1% of the transactions.

This was a key asset for PayPal: because of our data and our models, we could predict better than anyone else.

I applied this same approach to predicting the outcome of sporting events for Team Rankings. For Team Rankings, the features are things like offensive rebound percentage, power rating in away games, and distance traveled for this game. And we try to predict a score (Stanford expected to win by 4.7 points), a winner (Stanford has a 72% chance to win), or the team that will cover the point spread (Stanford has a 54% chance to cover). We’ve done pretty well over the years, correctly predicting close to 55% of games against the spread (50% is average; 52.5% or higher means you’re actually profitable).

So when I first joined LinkedIn to work on analytics, I found I had this awesome hammer (building predictive models) and went looking for nails. I was a little disappointed to find that there weren’t any obvious ones. Predict how likely someone was to become a premium subscriber? Kind of interesting, but unclear what you actually do with it. Estimate the likelihood someone changes jobs in the next year? A little more interesting — likely changers can be highlighted for recruiters and get more “new job”-focused content — but only really valuable if the differences between high likelihood and low likelihood are large.

For a couple of years, Circle of Moms was much the same story: no obvious nails to slam with the predictive modeling hammer. But then our editorial team launched The RoundUp and was writing a number of articles each week, in large part to allow us to cut the Facebook cord. What if we could estimate the likelihood that a mom would click on an article emailed to her, before actually sending the article? Then we could send out the article she’d be most likely to read.

This time around, I structured things a little differently. Not wanting to spend weeks and weeks re-assembling “at the time of transaction” data as we had at PayPal, we computed feature values (e.g., how old is this mom’s youngest kid, what percentage of food-related emails has she historically clicked on) as of the time we sent the email, and immediately stored those values in the database.

Rather than build one statistical model to predict clickthrough rate for all user-email combinations, we built statistical models specific to each article. If we’d built just one predictive model, we would have used characteristics of the article as model features, and could have started scoring people immediately upon starting to send the article. However, our article-specific understanding of who would click on what would have been limited: no doubt the top predictor of clickthrough rate would have been overall email clickthrough rate.

By building an article-specific model, the unique predictors for this article — e.g., having a child between 12 and 24 months for that “how to start potty training” story — would feature more prominently in the model. The downside? We’d need to create lots and lots of statistical models, and we couldn’t use our models until we gathered enough data to build them out.

To handle building that number of models, we had to automate the process of model creation. Fortunately (okay, intentionally), we were already saving everything we’d use to build a statistical model in a single table we could dump out with one simple query. Building the technology out took a month or two, but once we did it, everything was automated: no data scientists needed. Every week, we’d build ten or twenty models for the new articles we were testing with a small batch of users. Then the following week, we’d generate scores (predicted clickthrough odds) for each article for each user, then send each user the best scoring article.

It turned out that all of this worked well. Our models were pretty good at finding targeted content for our six million moms, and once we tweaked the model building technology, everything scaled pretty nicely.

Automation is incredibly important. It democratizes the process of building and using statistical models, so that a small startup (with lots of data) can build pretty good statistical models without a team of statisticians. These automated statistical models will almost inevitably perform more poorly than their human-built counterparts, but they’re close enough to be competitive.

Kaggle allows companies like PayPal, Netflix, and Allstate to upload data sets; data experts then compete to build the best predictive model to win a cash prize. Somewhat counter-intuitively, this has a similar impact to automated statistical modeling: it commoditizes the process of coming up with predictive algorithms. A decade ago the algorithms were key. What we’ll likely see going forward is that data matter a lot, algorithms are close to a commodity, and clever applications of those algorithms grow in importance.

As it becomes easier to build predictive models — and most companies have a really long way to go to get there — we’ll see more and more applications when the technologies and processes to build models get simpler. Here are some examples.

1) Finding the odds a user will like each of 10 or 20 choices. This is what we did at Circle of Moms, and is a great way to find the best content for an email, daily deal, or small (mobile) screen. It has huge potential for many content and purchase-driven sites, and is a key component of Retention Science, in which I’m an investor.

2) The last mile of a complex recommendation system. Amazon could likely get significant improvements by building predictive models for whether someone will buy (click on?) each of their top 10,000 items (i.e., 10,000 separate models). Then, when I go to the home page and they use other logic to find the top 50 for me, they apply predictive modeling to order those top 50 (scoring the top 10,000 would likely be too intensive).

3) Advanced user segmentation. As I mentioned before, it’s not obvious what LinkedIn gains from knowing my likelihood of becoming a subscriber. But if they can break that out in a few separate ways — odds of me being a premium job seeker, odds of me buying a recruiting package, odds of me being a non-subscriber who nonetheless invites a lot of friends — they can upsell the functionality that I’m most likely to use. If they want to get really clever, they can A/B test different upsells and build statistical models to predict someone’s response rate to each one.

4) Likelihood of being interested in a specific add-on. If Hertz determined that there’s a 0.5% chance I’d actually buy the additional insurance option or the prepaid gas, they likely wouldn’t bother trying to sell it to me. On the other hand, if it was 50%, they would (and should). If they avoided pitching me on something I’ll never buy, they’d go through the line more quickly and piss me off less; meanwhile they could still use upsells to drive profit by pitching customers who are more likely to buy.

5) Predicting someone’s movement from where they are right now. Given who and what they are, and that they’re walking or driving in a certain direction, what is the likelihood that they enter this store or that restaurant?

To be clear, automated predictive modeling is not a cure-all. It’s too expensive (in terms of processing power) to apply to every pair of items (user and book, for example). It doesn’t add much if a user has strongly indicated intent (this friend is in my address book, or I’ve searched for this exact flight) — in that case nothing fancy is required. But for many other areas, expect to see predictive modeling become automated and democratized, as data become more important and algorithms less so.

3 Things I Learned at LinkedIn (and 2 I didn’t)

It’s a neat feeling when a story about your grandfather helps you contextualize your own journey.

When Poppop, my grandfather, was a young man, he worked in his father’s store. He would keep detailed records of what people were buying. Poppop tried various promotional strategies to improve sales, then closely examined the results. He found that he had a natural talent for gaining insight from looking at numbers; he realized that gave him a big advantage in business (he didn’t need lots of data or fancy A/B testing).

Later on, Poppop would bring those skills — and a natural ability to relate to people — to a business he started with his brothers. Their clothing company, Rockower Brothers, would go public in 1962; it was acquired by Woolworth in 1978.

That progression — from data-obsessed junior employee to entrepreneur using data strategically — sounded much like my own. For both me and my grandfather, mastery of data and math would be one step of several on the trail toward starting a meaningful company. Unfortunately, he died when I was young, so I never got to directly ask him about those experiences.

It’s tempting to paint Silicon Valley as a unique haven for progress with no historical precedent. As with many similarly bold pictures, that painting contains elements of truth, but doesn’t tell the whole story.

In the first post in this series, I wrote about my relatively confined role at PayPal; it made me comfortable with startup life but didn’t really prepare me for entrepreneurship.

My experience at LinkedIn — where I was the first analytics employee and led the analytics team — did much more to prepare me to start a company. That difference is more about my role than about the company: despite having a strong vision, LinkedIn was not tactically great and the team was not as strong as PayPal’s (it’s unlikely we’ll ever see a substantial “LinkedIn Mafia”).

Specifically, my time at LinkedIn allowed me to grow in three areas:

1) Product ecosystem and distribution

Silicon Valley’s priorities are changing. In the past few years, we’ve seen a collective appreciation for the importance of user experience on product. In the last year or two, the term “data scientist” has become popular, but almost no one understands what big data really means(here’s my answer). Very recently, the term “growth hacker” has become popular (I’ve been so branded), thanks to an understanding that getting people to use your product isn’t just something to hope for.

But almost nobody talks of the importance of understanding an Internet company’s usage ecosystem. Take a new user, and project her activity out a few months. Is she going to get through the signup flow? Engage with the product in different ways? Invite friends? And more important, how does this behavior affect other users on the site? Growth hacking requires an understanding of a subset of the usage ecosystem, but the usage ecosystem is much more than that. A company that does well on growth hacking but poorly on creating an ecosystem ends up providing very little value for its users. That was where we found ourselves with Circle of Friends — Circle of Moms’ predecessor — and a big part of why we decided to pivot.

At LinkedIn, a big chunk of my time was spent understanding the ecosystem. Before that experience, I had only a superficial understanding of virality, never seeing how small changes can make big differences. I knew nothing of the effects of different channels — transactional emails, marketing emails, social emails, SEO, partner referrals, press — on a product’s ecosystem. At PayPal, I crunched numbers. At LinkedIn, I wrote email copy to boost numbers and realized the importance of understanding user psychology. Those skills and interests would transfer nicely to the experience building out Circle of Moms, arguably becoming my strongest asset as a founder.

2) Leadership

LinkedIn was my first experience hiring and managing. There’s a lot of (mostly superfluous) writing on whether leaders are born or made. I won’t generalize, but I will state that I certainly wasn’t a born leader: 26-year old me was a much weaker leader than 34-year old me is.

26-year-old me was a very awkward manager. I felt bad asking my employees to do things for me and I was overly deferential to both my own team and others in the organization. To some extent, that’s my personality and will never change completely. But in two years managing people I gained a degree of confidence and authority as a leader that is absolutely essential for a co-founder. If in 2004 I’d started a company instead of joining LinkedIn, I would have ultimately ramped up and gotten used to managing people, but it would have been a bumpier process (and Circle of Moms was bumpy in its own right).

At LinkedIn, I hired two full-time employees and one part-time employee. All three were older than I was and more educated (two with PhD’s). Hiring was a new experience, and an educational one. In the hiring process, I saw a contingency recruiter misrepresent both sides to increase her own take; I also had a candidate accept an offer from us, only to change his mind when his then-employer made him a counter-offer. When I started (what became) Circle of Moms, I was hardly a hardened and savvy hirer, but I’d at least been around the block once or twice.

Of course, management experience is a double-edged sword for entrepreneurs. We all know stories of those who easily managed hundreds of people in big companies, then struggled to be hands-on when part of a tiny startup. My pre-startup experience — managing a couple of people at LinkedIn but still being mostly hands-on — was about the right prep to start a company. For those who more naturally gravitate towards management, leading a team pre-startup is probably unnecessary.

3) The hubristic belief that I could do better

I continued to build out my longtime side project, Team Rankings, while at LinkedIn. It was becoming more and more of a business. In 2004, I started to charge for premium NCAA Tournament content. In 2005, I started publishing predictive models on the site and charged people to access them. Later that year, Tom Federico started working on the site with me, and we raised the bar on product polish and the branding. Come 2006, we were really cranking on our March Madness product: we built an awesome tool called BracketBrains to help with office pool picks. That March, I was constantly tweaking the site’s SEO, scrambling to keep the server up as traffic surged, and optimizing our marketing messages in real-time. It was exhilarating.

Meanwhile, things were moving along pretty slowly at LinkedIn. I was firmly planted on the analytics side of things. I couldn’t write production code, and had limited places to make user-facing changes. A few months earlier, my friend Jawed had pinged me about joining his new startup called YouTube. I hadn’t been interested then, but after that March Madness experience, I saw things differently: YouTube’s early growth, quick speed of iteration, and small size (around ten people) had some appeal. I had significant discussions with YouTube but didn’t wind up joining them; I probably would have if I’d found online video as interesting and important as a professional network.

In the months following, I became more frustrated with both the slow speed of development and what I thought were some poor tactical decisions. Because I was so close to the numbers, LinkedIn’s decision-making bugged me, and made me think I could do better. Because I had built up Team Rankings and been able to quickly iterate on it, I figured I could also build products better and more quickly (to those who have had the misfortune of working with my code: insert a snide remark here).

What I didn’t learn at LinkedIn

When I started Circle of Moms, I was a crappy software developer (I’m still no superstar). I could write code pretty quickly and solve scalability problems in clever ways, but I didn’t do a very good job architecting a system. That wasn’t the end of the world — we had lots of good engineers who eventually made our code base fairly solid — and I’d argue that for very early consumer Internet startups, an A+ engineer is only a nice-to-have.

I also didn’t have much of a business focus at LinkedIn; I didn’t know or understand much about raising capital, equity, finances, or the role of sales on LinkedIn’s success. At Circle of Moms, my co-founder Ephraim — who was CEO while I was CTO — focused more on these areas. My lack of knowledge here certainly didn’t help, but probably wasn’t the end of the world: it’s much easier for a good product person to learn about business than for a good business person to learn about product.

Moving on

In January 2007, it was time to move on. I told Sarah Imbach (my boss) and Reid Hoffman that I was planning to resign and ultimately start my own thing. Reid sat me down, seemingly to convince me to stick around. In a similar conversation nine months prior, Reid had convinced me that LinkedIn — and not YouTube — was the place to be (it’s nice when your fork in the road leads to honey on one side and sugar on the other). But the dynamic for this discussion was very different: I’d found the honey for myself at the end of the LinkedIn road and wanted to move on to climb a big but as yet undefined mountain.

In that conversation, Reid revealed to me that LinkedIn was very close to bringing on a CEO to replace him, describing the candidate as a very well-respected #2 person at a public company. I was somewhat taken aback, but it made sense: Reid understands his skills (product vision, many different types of strategy) and weaknesses (operational details) as well as almost anyone. It was probably a last-ditch effort to get me to stay, the implication being that LinkedIn would soon be better led and better run. The new CEO, who it turned out would be Dan Nye, would hopefully address the weaknesses I saw in the company and its culture.

Alas, I’d made up my mind and knew where I wanted to go. I’d grown in three very important areas. Like my grandfather who moved on from his father’s store, I was ready for a new challenge.