Data, Analytics, & Finding Your OKM: Mixpanel Goes Beyond Page Views
What’s the true measure of success for a startup? That depends on what you’re trying to accomplish. If you work at Path, the San Francisco-based mobile social network, it’s probably not useful to obsess over how many people have downloaded your iOS or Android apps, or even how often the apps are opened or how many posts people view.
Given that Path is supposed to be a shared personal journal, a more pertinent question would be how many people are actually capturing and sharing “moments” such as photos, videos, or check-ins. If people aren’t posting, after all, they’re not contributing to Path’s community or its business.
“That is the actual metric for Path,” says Suhail Doshi, co-founder and CEO of San Francisco-based Mixpanel. “Every time you share a moment with friends, that is much more defining than a download or a page view, because that’s someone saying ‘I am interested in this app and I would like to engage with it.’”
Doshi and his co-founder Tim Trefren should know. They started Mixpanel just three years ago inside the Y Combinator startup accelerator, and now they provide analytics wisdom to Path and dozens of other consumer-facing Web and mobile startups, including Airbnb, Quora, Posterous, BitTorrent, Bebo, Ubisoft, Jawbone, Twitch TV, and Leviathan Games. If there’s one message the 12-employee startup tries to impress on all its customers, Doshi says, it’s that “page views are not very important. Simply because you are producing page views doesn’t mean you are growing or doing well as a business. There are much better, more objective ways to determine whether you are doing well.”
Pressing the point home, Doshi says every company ought to identify the OKM—One Key Metric—that sums up its health and progress. “It’s the one thing you would bet the company on,” he says. For a publisher, the OKM shouldn’t be simply how many people came to the website in a given month—that might not signify much if most of them bounced right on to some other site. A more meaningful number would be how many people stayed for at least one minute and shared at least one article via Twitter, Facebook, or e-mail. “It doesn’t have to be a big number,” says Doshi. “Just something that’s harsh enough that you will know you’re succeeding as long as the number keeps going up.”
The problem with finding your OKM is that it might not be a number that even shows up in traditional analytics packages from the likes of Omniture (now part of Adobe), Coremetrics (now part of IBM), or Google Analytics. And if it’s there, running the query to find it could take hours. In contrast, giving tech companies the power to track and dissect more types of events in near-real time is the whole reason for Mixpanel’s existence. “There are all these companies that track page views, so from the first line of code I wrote I said I was going to try to build something different,” Doshi says.
The taste for metrics was something Doshi acquired while doing a summer internship for Slide, the app development company founded in 2005 by PayPal co-founder Max Levchin. Slide was extremely numbers-driven, and Doshi said he’d heard that Levchin “identifies with people like him who stay up until 1 am and code like crazy,” so he barely left the office that summer while working with the analytics team to improve the code for a new product. He wound up making enough of an impression that Levchin became the first contributor to Mixpanel’s $1.75 million seed round after the startup exited Y Combinator in the summer of 2009. (Levchin was joined by Square chief operating officer Keith Rabois, Bebo co-founder Michael Birch, and Sequoia Capital.)
But the biggest validation that Silicon Valley movers and shakers think Mixpanel is onto something came this May, when Andreessen Horowitz led a $10.25 Series A investment. Levchin, Salesforce CEO Marc Benioff, and Yammer CEO David Sacks also participated. “We’ve watched Suhail and his team at Mixpanel transform the company from an innovative startup to one that delivers the most advanced solution we’ve seen for mobile and Web analytics,” said Andreesen Horowitz partner Peter Levine in a press statement.
Once the data is in Mixpanel’s system, the real analysis can happen. The startup has built online tools that let companies slice and dice their data without having to learn SQL or write database queries. To demonstrate—and to prove that Mixpanel eats its own dog food—Doshi showed me an analytics report on how quickly Mixpanel’s graph-rendering package was doing its job for various users. The news was good: for most customers, graphs were showing up in less than a second. But the report was interactive, allowing Doshi to segment the analysis to his heart’s content. “If I want, I can view a distribution of response times where the time is less than one second, the browser is Chrome, and the user is from the U.S.,” he says. “These are incredibly sophisticated things that Kissmetrics, Omniture, and Coremetrics cannot do.”
The other thing that sets Mixpanel apart from its competitors in analytics, according to Doshi, is that its tools can parse incoming data points in real time—or as close to it as you can get. If a Path user shares a moment, in other words, it will show up on Path’s internal dashboards almost immediately. Omniture users have to wait 3 to 24 hours for data like that, Doshi says—and it’s not just building the reports, but querying them, that can be time-consuming. “Our metric is three seconds,” he says. “When you change the behavior of analytics like that to make it real time, customers start exploring their data a lot more than they used to. They don’t feel like they are going to get penalized if they ask the wrong question and then have to wait another day for the result.”
To make Mixpanel so fast, Doshi and his engineers had to throw out off-the-shelf database tools and write their own. He says the company started out using MySQL and Cassandra, an open source distributed database program first developed at Facebook, but abandoned them because they were too hard to modify. In mid-2011 the company switched to its own data store; it’s called Arb, short for arbitrary. “It can do pretty much do any kind of analysis we need it to. It’s the big reason a lot of our customers use Mixpanel.”
Recently, the company has been sharpening its focus on individual users. So-called “funnel analysis” has long been one of the tricks its database can perform, meaning it can track whether a user completed all the steps in a procedure—say, picking a username, inputting a password, uploading a profile photo, and connecting with friends when joining a social network. But recently the company launched “Mixpanel People,” a new menu of options that lets customers tie analytics data to specific users and compare behavior across groups of similar users. Such capabilities are also available from services like Performable (now part of HubSpot), but it made sense for Mixpanel to add them, since customers were already collecting so much demographic data in the form the properties associated with each event data point. Doshi calls the people feature “the next evolution of our product line.”
As for the startup’s business model, it’s simple: customers pay by the data point. An early-stage startup with a few thousand users might generate 500,000 data points a month, at a cost of $150; enterprises might generate up to 20 million data points at a cost of $2,000 per month. Doshi says that Mixpanel tracks more than 6 billion data points per month overall. The company has been cash flow positive throughout 2012, and it’s got “massive deals with two companies in the Fortune 100 list” in the works, Doshi says. Another new customer offers a mobile game that’s “one quarter the size of Angry Birds,” he says.
Of course, it’s one thing to measure the performance of your app or website, and something else entirely to change or improve it. Mixpanel can’t actively help with that part. But if you don’t have One Key Metric—or, to be more realistic, Half a Dozen Key Metrics—you won’t even know if you’re making progress. Plus, there’s just something cool about data and dashboards, which provide at least the illusion of knowledge and control. “Analytics is the only objective way to determine whether you are growing, whether people care about what you’ve built, and how you can improve what you’re doing,” says Doshi. “Otherwise, it’s all just intuition.”