From the Lab that Brought You Siri, It’s Trapit—A Personalized Discovery Engine

10/21/11Follow @wroush

There are two kinds of people: Those who already own an iPhone 4S, and those who don’t but have seen Apple’s Siri ads and wish they did. Siri, of course, is the voice-operated personal assistant that can read your text messages, schedule appointments, check on the weather, make waffles, and babysit your toddler (just kidding about those last two). It’s the coolest feature of the latest Apple smartphone, which went on sale last week. Virtually overnight, it has brought state-of-the-art artificial intelligence technology into the consumer mainstream.

Well, it turns out that there’s more where that came from. The same Silicon Valley defense project that gave birth to Siri—called CALO, for Cognitive Assistant that Learns and Organizes (more on this below)—has another, lesser-known spinoff, a Palo Alto news personalization startup called Trapit. The company’s web-based service uses AI to organize the welter of new content that appears online every day into tailored collections called “traps.” And while the technology is still a bit raw, there’s a chance that it could have the same kind of impact in the world of Web search and online news that Siri is beginning to have on mobile interfaces.

I’ve been playing with TrapIt for about three months now. It hasn’t become a part of my daily news-browsing routine, but I can definitely see that happening if the startup continues to refine the interface, improve its search algorithms, and make the site more tablet-friendly. (Trapit took the lid off its service in June, but it remains in closed beta testing, which means you have to request an invitation to get an account. The wait was short when I registered. The company says it’s going to open the beta version of its service to the whole public later this fall.)

The first thing to try when you go to Trapit is either to browse one of the existing, featured traps—which are often related to breaking news, such as yesterday’s killing of Muammar Qaddafi—or start one of your own by entering a phrase or keyword into the “Discover” bar. After a short wait, you’ll be presented with recent news stories and blog posts on your topic, culled from across the Web.

At first the selection may seem pretty random. The neat part is that as you peruse various articles, which pop up in lightbox-style windows, Trapit observes what you’re reading, how long you spend with each article, and what you’re sharing with others. It uses these cues and others to beef up its profile of your personal tastes, so that over time it’s able to surface more articles that fit your interests and fewer that don’t.

You can also train Trapit manually by clicking on the thumbs-up or thumbs-down buttons—and the more you do this, the faster the software will learn your preferences. As you create traps on new topics and train your existing traps, you can end up with a whole gallery of mini-magazines, exclusively tuned to the mix of subjects that you, and you alone, are passionate about. It’s a pretty unique service—the closest comparison I can think of is Google Alerts, which are like standing search queries with the results e-mailed to your inbox every day. But Trapit is more like regular Web surfing. It’s just that you’re surfing the Web you want.

Trapit’s AI-driven approach goes completely counter to the dominant trend in news curation today, which emphasizes the power of social networking and collaborative filtering. News aggregation apps like Flipboard, Pulse, AOL’s Editions, CNN’s Zite, Yahoo’s Livestand, and Google’s forthcoming Propeller platform may seem to provide personalized news feeds, but they’re really just bringing together stories that have been flagged by the people you follow on Facebook, Twitter, LinkedIn, or Google+. The information is only “personalized” in the sense that no one else has the same group of friends as you.

“This whole idea of using your social networks to find information is not really a solution, it’s an accommodation,” Gary Griffiths, Trapit’s CEO, argued to me recently. “It’s a recognition that there is way more stuff out there than you can ever find, so you just say, ‘I’m going to let my friends find it for me.’”

But what if your friends don’t share all of your interests, or aren’t very good at ferreting out the kinds of obscure news tidbits that might make your day, or are just dull people? The view at TrapIt is that the Web is a lot wider and deeper than either you or your friends can cope with—so wide and deep, in fact, that only software can be trusted to uncover the buried content that would be gold to you but dross to someone else. Hank Notthaft, Jr., chief product officer at Trapit, calls it “a discovery engine built for a Web that has grown so large and dynamic that it demands true personalization…While other platforms, from Google to Facebook and Twitter, are limited to what’s popular or what your so-called friends think is important, Trapit delivers what’s important to you.”

Like Siri, Trapit is an offshoot of the five-year, $200 million CALO research project. The goal of the project, which was based at SRI International in Menlo Park, CA and funded by the Defense Advanced Research Projects Agency (DARPA), was audacious: to create “systems that can reason, learn from experience, be told what to do, explain what they are doing, reflect on their experience, and respond robustly to surprise,” in the words of an SRI website.

Siri, which was spun off by SRI as a separate company in 2008 and acquired by Apple in April 2010, represents just one embodiment of that vision. The smartphone app is very good at parsing your spoken requests and assembling answers using the resources and data at hand, such as your contact list, calendar appointments, and current location.

Trapit is a very different embodiment. Its roots are in a program called CALO News Assistant, a prototype system built to help military personnel sift through news articles and blog posts. Users could select topics based on their fields, skills, or assignments and receive related article summaries via e-mail. “CALO learns the kind of articles you read and will highlight articles that best match your interests, focusing your attention on the most important articles,” in the words of one SRI brochure describing the system.

Nothhaft, who had worked as a product manager at WebEx and as a product marketing executive at business-telecom startup LiteScape Technologies, joined SRI in 2009 as an entrepreneur-in-residence, originally with the goal of sniffing out business collaboration and meeting technologies that might be ready for commercialization. He didn’t find anything that was ripe in those areas, but he did learn about CALO News Assistant. Nothhaft says he became “very excited…I’m an analyst by nature, and I’ve always been an information hound, with 2,000 feeds in my RSS reader.”

He thought that if the news assistant could be adapted for mainstream news consumers, it would be a powerful alternative to existing aggregation systems like that RSS reader. But there were two big problems: the news assistant scoured a limited selection of sources, and it was slow—it worked all night just to identify a few articles matched to users’ interests. To be credible, a consumer-scale system would need to scour the whole Web, and it would need to process information in near-real time, ingesting newly published articles and matching them against users’ personal profiles in just seconds.

That was the goal that Notthaft and Griffiths—Notthaft’s old mentor from WebEx and LiteScape—set for themselves once they got SRI’s blessing to spin out the technology. Luckily, they had help from David Schairer, the former chief technology officer at Concentric, who became Trapit’s third co-founder. With both theoretical expertise in adaptive learning and natural language processing and practical experience building high-volume spam filters for Internet service providers, Schairer is a nearly unique commodity in Silicon Valley, Griffiths says. Even so, making the news assistant faster and scaling it up to monitor more sources (some 50,000 now) took more than a year. “It was a lot of work,” Griffiths says.

And the truth is that there’s more work to do. Trapit probably needs to learn even faster than it does, and do better at guessing users’ intentions when it doesn’t get direct feedback. When I created a trap for “Hitchcock,” one of my favorite directors, it took me forever to teach it that I meant Alfred Hitchcock, not Hitchcock, TX (a suburb of Galveston), not the singer-songwriter Robyn Hitchcock, and not the Arizona TV anchorwoman Tara Hitchcock. (I confess that I deliberately didn’t call the trap “Alfred Hitchock” from the start, because I wanted to see how long it would take to train it. But I had no idea how much non-Alfred news there would be.) When I created a trap for “vegetarian recipes,” the second item Trapit showed was a recipe for chicken with pesto and penne. I gave that one a thumbs-down, naturally, but will Trapit know it was because of the chicken? To be ready for prime time, in short, Trapit will need to work reliably even for users who aren’t as willing as early adopters like me to train it.

Notthaft says that the 16-employee company, which has an office in Portland, OR, as well as Palo Alto, is working on a series of feature enhancements that will show up on the site this fall. The service is free, but Nothhaft and Griffiths envision selling annual subscriptions to traps that include content that’s normally behind a paywall (e.g., the Wall Street Journal), or creating “sponsored traps”—imagine golf club maker Ping sponsoring a featured trap on the U.S. Open, for example.

Trapit, which recently raised $5.6 million in venture capital from a group including Horizons Ventures (which also invested in Siri), hasn’t said whether it’s working on native smartphone or tablet versions of its service. But that would be a natural direction to go. I do most of my own news browsing these days on my iPad rather than the desktop Web, and some of Trapit’s features don’t work well in a mobile browser. For me, a best-of-both-worlds news app would be something that’s as pretty and as user-friendly as Flipboard, but with the filtering smarts of Trapit underneath the hood. In fact, I won’t be surprised if Trapit gets scooped up quickly, just as Siri was, by a larger company with an interest in content curation—say, Flipboard, Google, Facebook, or even Apple.

Indeed, the bidding may already have begun. Nothhaft says the publicity around Siri has begun to rub off on Trapit, resulting in “increased visibility and opportunities” for the startup. “Just as Siri is revolutionizing the human-computer interaction on the mobile device, Trapit will revolutionize web search as we know it today,” he asserts. A bold claim—but one with some history to back it up.

Wade Roush is a contributing editor at Xconomy. Follow @wroush

By posting a comment, you agree to our terms and conditions.