How Crimson Hexagon Translates the Blogosphere’s Babel Into Wisdom

11/12/08Follow @wroush

File this under “Only in Cambridge.” Before my interview last week with the founders of Crimson Hexagon, a startup using statistical methods to comb the blogosphere for the latest opinion on brand-name products, I had assumed that the company’s name came from its affiliation with Harvard, where its technical founder, Gary King, is a professor of government. Nope—turns out it’s an allusion to a single line in a short story in Spanish by the late avant-garde Argentinian writer Jorge Luis Borges.

The “Hexágono Carmesí” in Borges’ 1941 story “La Biblioteca de Babel” (“The Library of Babel”) is the hidden central chamber in an infinite library; it’s the room containing the single magical book that serves as a perfect compendium and guide to all the other books. The room is supposed to be a metaphor for the company’s search algorithm, which is magical in its own way. As King explains it, it’s able to gather precise summaries of the blogosphere’s sentiment on given products or personalities, but without actually having to understand or accurately classify each mention of said entities.

Or maybe the Borges allusion is just a diversionary tactic. Harvard, after all, is notoriously jealous of its own brand identity. “If we said it was crimson because of Harvard they wouldn’t want us to use it,” jokes Candace Fleming, Crimson Hexagon’s CEO.

I talked last Friday with Fleming and King, who have been working behind the scenes to launch the angel-funded company since 2007 and had their official coming-out on October 14. While a number of other publications have taken note of the startup, I wanted to know what makes King’s algorithm tick, and how the company plans to make it pay.

King’s algorithm, called ReadMe, mines the deep vein of public opinion represented by blogs (about 1.5 million of which are updated at least once a week, according to Technorati). This is a different, and much more focused, variety of opinion than what classical public-opinion polls can get at, the company argues. “Polls are very good for certain purposes, but essentially they are pop quizzes about subjects the respondents may not know anything about,” says King, who directs Harvard’s Institute for Quantitative Social Science. “But if you’re interested in what digital camera to buy, you want the opinion of your Uncle Max, who spent a lot of time figuring it out. The opinion of ‘the American public’ is not interesting to you.”

Crimson Hexagon report viewBrand-monitoring agencies have long been aware that the blogosphere is a treasure trove of consumer opinion, and there are a number of Web-analytics companies (including some here in Boston, like the Cymfony and Compete divisions of TNS Media Intelligence) that purport to automate the process of distilling that opinion. But for the most part, these companies’ tools for measuring blog-based sentiment are clumsy, inexact, and labor-intensive, King and Fleming say.

“With current online brand monitoring solutions, you can find out how many people are talking about your product, and how many people are saying positive and negative things,” says Fleming. “But it’s a little bit like going to the doctor, and having the doctor say ‘You’re sick,’ but then he doesn’t tell you what’s wrong. It can be frustrating.”

Crimson Hexagon’s technology lets the company go several layers deeper, according to Fleming, who walked me through a quick example using the Apple iPhone—a product Crimson Hexagon was asked to study by a client (not Apple). “We can tell you not only how many people are saying positive things about the iPhone, but we can tell you that 30 percent are saying that they love the third-party apps, and 15 percent are complaining that the onscreen keyboard is really hard to use,” Fleming says. The company can also track that sentiment over time. Adoring blog commentary about iPhone apps, for example, spiked after July 11, when Apple introduced new software that allows users to download third-party software.

Fleming, the former CEO of another Cambridge, MA-based Web analytics company called Icosystem, says Crimson Hexagon is already doing consulting work with customers from a range of industries, including computer hardware, mobile phones, travel, finance, advertising, and international aid. “So we’re finding that this is applicable in a lot of different ways,” she says. “The key thing all of these people have in common is that they care and they want to know what people are saying about them online.”

But how does Crimson Hexagon sort through the enormous mess of material added to the blogosphere every day to find nuggets of opinion about specific brands—and more importantly, how does it sort these opinions into the buckets that interest its clients? That’s where ReadMe comes in. King says the idea for the algorithm crystallized from two initially unrelated projects he was working on at Harvard.

The first was a project to track opinion about the 2008 presidential candidates, back before the primaries, when there were quite a few of them. “We figured out how to find and download the information in all the political blogs, but when we tried all of the standard computer science approaches to classifying them into the categories we were interested in, they were a disaster,” King recounts. “Some of them would work as much as 60 or 70 percent of the time—which means, of course, that 30 or 40 percent of the time you’d get a completely wrong answer. We tried method after method, and they were all failing.”

Gary KingAt the same time, King says, he was helping the World Health Organization tackle the problem of obtaining accurate mortality data in developing countries, where autopsies and death certificates are rare. “They had a way of doing this by surveying the relatives of people who had recently died, and asking them a series of uncomfortable questions like ‘Were they bleeding from the mouth?’ and ‘Did they have a stomach ache?’ And then they’d show the answers to physicians, who would decide what the cause of death was. They called this a verbal autopsy. The problem was that if you showed this data to more than one MD, they would never agree. We found a way to automate the sorting that worked much better than showing the data to physicians, and the World Health Organization has now implemented this all over the world.”

At a certain point, says King, “I realized that the mathematics underlying the two problems was equivalent.” The same method that he had used to reach nearly 100 percent accuracy in sorting the verbal autopsy data, in other words, could also be used to accurately categorize opinions about political candidates.

What is that method? That’s the part that’s a little bit like magic. And to understand it, it might help you to recall the debate that raged during the 2000 U.S. Census about “statistical adjustment” versus “direct enumeration.” Statisticians at the Census Bureau argued that traditional methods of finding and interviewing Americans during the decennial census inevitably overcount some groups and undercount others, and that greater accuracy could be achieved by conducting a separate, large sampling survey, then using its results to adjust the traditional count.

Politicians in Congress—some of whom stood to lose their seats to redistricting if the adjusted census found larger minority populations than expected—balked at this proposal. And to the dismay of statisticians, the Supreme Court ultimately declared that the official population of the United States is the number of people physically enumerated by census interviewers.

“If you cared about the real number of people in the country, you would want to use the statistical approach,” says King. But at Crimson Hexagon, he says, “we are not bound by an arbitrary legal ruling. We care about the actual fraction of blog opinions in a particular category.” That means the company is free to use statistical methods to adjust its actual counts—which, as it turns out, hugely simplifies the problem of categorizing large numbers of opinions, whether they’re about presidential candidates, causes of mortality, or digital cameras.

Like the census statisticians’ proposal, ReadMe is all about correcting errors in the original data. Using standard search algorithms, Crimson Hexagon can sort blog posts into categories with about 80 percent accuracy—which sounds good, but wouldn’t be useful to the company’s clients, since at that level of error some categories could by off by 50 percent or more. “But suppose,” says King, “that you knew that 10 percent of the documents in Category One should actually be in Category Three. That might not help you classify individual documents better—but since we’re not interested in that, it doesn’t matter. You just subtract that many from Category One and add them to Category Three. It gets you the right answer, proportionally.” In other words, you can stick with an 80-percent accurate categorization scheme, but still get a nearly 100-percent accurate sense of the proportions among the categories. Or as King rephrases the point: “Classifying every needle in the haystack is very difficult, but we can still classify the whole haystack.”

(The sizes of the errors in each category, of course, are critical. The company determines that by having human coders create a “training set” by initially classifying, say, 100 blog posts into categories. “Then you run your statistical method on those same documents and see how well it does,” says King. “It may be doing really well in Category Seven, but 10 percent of the documents that should be in Category Three get put into Category One. Then we know what the correction is” for the whole data set.)

Undoubtedly, that’s all an oversimplification—but you probably get the gist. Fleming says Crimson Hexagon’s clients put its findings to a variety of profitable uses. She told me a story about “a major networking hardware company” (it’s not hard to read between the lines about which company this might be) that asked Crimson Hexagon to monitor general opinion in the blogosphere about whether its stock was overpriced or underpriced. The company was particularly interested in how bloggers would react to a negative earnings announcement it planned the make. “It shocked them to see that there was a little blip in opinion, but it wasn’t significant,” says Fleming. “A PR campaign they had launched two days prior on a new product they were releasing was completely overshadowing the negative reaction from the quarterly earnings. Two direct actions came out of that—one was that they decided not to do this damage control campaign they were about the launch, because it looked like the impact from the negative earnings was not as bad as they thought. And they also had no idea how much reaction this new ad was generating in the market, and they decided to accelerate that campaign.”

More and more companies, says Fleming, are looking for this kind of “fast feedback” about how a new product, campaign, or piece of bad news is playing in the market. For now, companies have to work with Crimson Hexagon consultants to formulate the questions they want answered, come up with sorting categories for ReadMe, and create an initial training set. But by next year, she says, the startup plans to roll out a self-service, Web-based version of the tool.

“There are so many things that people in the business world want to know about what consumers are saying and thinking, where they like to do a quick opinion poll if they could,” Fleming says. “But you don’t have to go out do an opinion poll, because people are naturally talking about these things on the Web”—today’s real Library of Babel.

Wade Roush is a contributing editor at Xconomy. Follow @wroush

By posting a comment, you agree to our terms and conditions.