Bluefin Labs, Named After a Sushi Bar, Tracks Social Media Around “Every Show on TV”

9/12/11Follow @gthuang

Deb Roy doesn’t strike me as a Jersey Shore kind of guy. Yet he can tell me exactly what viewers are saying about the estimable Deena Cortese during any episode, or how many people are annoyed by Mike “The Situation” Sorrentino and are tweeting about him (no, I haven’t been—too busy getting my GTL).

I’ll spare you the sordid details, but Bluefin Labs won’t. The Cambridge, MA-based tech company, which Roy co-founded in 2008, is all about understanding social-media conversations around TV programs and ads. Not just what’s being said and the sentiment behind it—as many other companies are tracking in various fields—but also the precise demographics of who’s saying it, what else they’re saying (and watching), and how their profile and comments are correlated with other people and programs, in an aggregate sense.

Bluefin Labs has gotten its share of press for its technology approach. But the company seems to be coming into its own as a business, and it has been garnering interest from TV network executives and big brand marketers as of late. So I sat down with Roy, Bluefin’s CEO, to learn more about the company’s history and approach.

Roy is a professor on leave from the MIT Media Lab. He’s an expert in artificial intelligence, machine learning, and data mining, among other things. His company, which now has north of 30 employees, has deep computer-science research in its DNA (for better or worse). Based largely on its technology and team, it has raised more than $7 million from Redpoint Ventures, Acacia Woods Ventures, Lerer Ventures, and other investors.

My first order of business: where did the name come from? Well, the story of Bluefin Labs (not to be confused with Bluefin Robotics, another MIT startup) goes back to Roy’s research at the Media Lab, where he has been a faculty member since 2000.

As he puts it, his research program was “built on understanding the relationship between words and context.” (Roy is known for having recorded a quarter-million hours of video of his young son’s early life, in part to understand how a computer might learn human language.) Around 2007, a PhD student of Roy’s, Michael Fleischman, was working on a project to analyze video footage of baseball games and audio of announcers talking about the games—a convenient (and fun) database for associating words and context. Conceptually, the goal was to teach a machine to understand what a “fly ball” or “home run” looks like, for applications such as video search.

A program director at the National Science Foundation happened to see an article about the baseball research. He called Roy and suggested he apply for a Small Business Innovation Research (SBIR) grant. Roy had no business experience—it was a “part of my brain I didn’t know existed,” he says—but he and Fleischman (who “has entrepreneurial blood running through his veins,” Roy says) wrote up a proposal. And in 2008, they were awarded a $100,000 SBIR grant. They had 48 hours to pick a name for their company, so they named it after a sushi bar in Porter Square (Blue Fin) where they had dinner. The name stuck.

Bluefin Labs started out by analyzing other types of sports programs, like football. Social media was taking off, so Roy and company ran a test with the National Football League to track what people were saying on Twitter about specific plays in games. They weren’t thinking about the ad industry yet, Roy says. This was about crowdsourcing highlight reels automatically, and tie-ins to fantasy football.

But the sheer volume of social-media content about television was astounding. “This looks bigger than a sports thing or football plays. This is about every show on TV,” Roy realized. “We went from, let’s do this fine granularity of comments on plays in football, to let’s make a machine that watches all of television and listens to all the conversations.” The result, he says, is “the most comprehensive database of what people [say] about what’s on TV. We call it the TV genome.”

(I must admit it would be frightening if the first sentient machine gets its worldview from television and social media. On the other hand, kids these days…)

Fast forward to today, and Bluefin has amassed a database of tweets, blog posts, and other public comments—from Facebook, YouTube, and so on—from nearly 20 million people, to go along with its video analysis of programs and ads across 117 TV networks. Roy says the company tracks 15-16 million comments a month that are about live TV, with about 5 percent month-to-month growth, in the U.S. (Bluefin processes the video—checking that a scheduled program is actually on-screen, for instance—and then throws out the raw footage.)

So where’s the business here? Well, it’s all in how you slice the data, Roy says. If you’re a network exec, you might want to know which programs—and time slots—are generating the most comments, so you can make decisions about scheduling, he says. Or you might want access to the demographics of your audience—gender, age range, whether they’re a parent, and what else they’re watching (all based on their social-media comments and public profile).

If you’re an ad exec, you might be interested in these demographics as well—and in things like what the audience overlap is between The Daily Show and other comedy programs and talk shows, for example. Bluefin has gotten interest from big soda and candy companies, as well as TV networks and ad agencies, Roy says, though he declined to name any yet.

It’s still very early in the product game for Bluefin. My hunch is social media analytics is where the future of Nielsen ratings, polls, and TV-ad placement is going—and the question is whether the analysis that a company like Bluefin can provide is worth paying big bucks for, compared to other options. Bluefin points to the size of its database and depth of its analytics as differentiators from the competition, of which there is plenty.

However it plays out, the company is clearly entering a new phase—as is Roy, who seems to be embracing his transition from professor to CEO. The last three years were about developing a “deep and rigorous technology stack,” he says. Now it’s about “how to productize the data. You don’t invent this kind of thing overnight. If you want to build a tall building, you dig a deep basement.”

Gregory T. Huang is Xconomy's Deputy Editor, National IT Editor, and the Editor of Xconomy Boston. You can e-mail him at gthuang@xconomy.com or call him at 617-252-7323. Follow @gthuang

By posting a comment, you agree to our terms and conditions.