I’ve been taking in Google’s recent release of “Search, plus your world” (or SPYW as the cool kids say) over the last several days, reflecting on what it means for Wetpaint and other media companies; but perhaps even more importantly, deeply understanding what it indicates about Facebook and Google themselves. As we all know by now, these most recent changes are meant to make its search more personal by up-weighting social activity in its algorithm, and using each person’s own position within their circles to determine relevance.

You might think that I would be one of the first to jump in the game with Google. After all, my company Wetpaint has been making a massive investment in distributing our content via other social channels, particularly Facebook. We’ve been seeing massive returns. And, I’ve even gone on a limb to predict that Facebook should be implementing its own Web-wide search this year.

Still, when it comes to playing Google’s social games, so far I’ve advocated staying on the sidelines of all their social venues—even their recent business pages. That’s been because even though the stadium lights are on, no one is on the field. More specifically, even though Google has 90 million registered users of the service, we see very little activity of significance among our target audience. But with its new SPYW changes, the question is: Has Google indeed forced companies’ hands?

Unfortunately, they have. And, in doing so, it marks a milestone in the changing mentality of Google. The search company’s great innovation—using the signals of the Web to best determine what the audience really wanted—has now been subverted. The company’s originally unshakable-seeming ethos of mechanistic neutrality has slowly, slowly, slowly, and now all of a sudden given way, and the new precedent is to favor its own business interests over those of the audience.

The result, like it or not, is that companies that rely on search for traffic must hear and obey loud and clear Google’s message that Google will favor those that favor it. It’s a dirty truth, and one far more chilling than the other more technical biases of its algorithm before.

Google has already started infusing search with the content that’s been blessed via Google+. Do a search for “New York Times” and you’ll probably find the New York Times plus.google.com page as the second search result. Search for “Mark Zuc” and you’ll likely see Zuckerberg’s Google+ page (despite the irony) populate as an option in the Google Instant choices.

I haven’t seen this bleed over to news stories yet, but I believe that it’s coming. Soon you’ll do a search for the latest headlines and your search results will be chock full with musings from your friends and non-friends inside Google+.

Google+ may not take off as a real social network, but Google has indicated that it’s throwing its full weight behind it anyway to make the best of what it’s got. Even if consumers don’t adopt it en masse, whatever activity is present will pepper the famous algorithm’s search results.

The irony here is that Google’s pivot toward a social search belies how important that social data is. The company is putting its lock on search at risk to gain a chance at a foothold on social. But what really comes through to me is that a great social search can be a winning product—if it’s populated with the right social data. So far, Google’s is not.

The question is—if that’s what I’m after—won’t I still just go to Facebook, where all my friends actually are (and which Google has adamantly cut out of SPYW)?

While SPYW does force publishers to support Google’s social network, fortunately it will be a temporary sacrifice from publishers during this period of transition from these days of search to a socially wired world. And that forthcoming world looks increasingly like it will be wired not by Google, but by its arch-enemy Facebook. Indeed, by corrupting the quality of their search product, Google may have just opened up a clear product entry into search for their rival as well.

Comments (4) | Reprints | Share:          

A dark day in Boston is a little darker now. Jerome Rubin, the publishing executive and inventor who co-founded the display company E Ink and helped commercialize the online database that became LexisNexis, died from a stroke on Monday in New York City, according to a report by the Associated Press. He was 86.

Rubin worked at the MIT Media Lab in the 1990s, where he helped spin out Massachusetts-based E Ink, the electronic-paper company that makes the display in the Amazon Kindle and other e-readers.

Before that, Rubin, a Harvard University alum trained in physics and law, helped launch an Ohio-based research database for lawyers in 1973. The database search and retrieval system eventually became LexisNexis, specializing in news, legal, and public records information. Harvard Magazine wrote about some of Rubin’s achievements back in 2000.

Rubin lived in Manhattan and is survived by his children, Richard Rubin and Alicia Yamin, according to the AP report.

Comments | Reprints | Share:          

There’s been very little change in top-level internet domains (like .com, .org, .us, etc.) for a long time. But a number of years ago I started thinking about the possibility of having a new .data top-level domain (TLD). And starting this week, there’ll finally be a period when it’s possible to apply to create such a thing.

It’s not at all clear what’s going to happen with new TLDs—or how people will end up feeling about them. Presumably there’ll be TLDs for places and communities and professions and categories of goods and events. A .data TLD would be a slightly different kind of thing. But along with some other interested parties, I’ve been exploring the possibility of creating such a thing.

With Wolfram|Alpha and Mathematica—as well as our annual Data Summit—we’ve been deeply involved with the worldwide data community, and coordinating the creation of a .data TLD would be an extension of that activity.

But what would be the point? For me, it’s about highlighting the exposure of data on the internet—and providing added impetus for organizations to expose data in a way that can efficiently be found and accessed.

In building Wolfram|Alpha, we’ve absorbed an immense amount of data, across a huge number of domains. But—perhaps surprisingly—almost none of it has come in any direct way from the visible internet. Instead, it’s mostly from a complicated patchwork of data files and feeds and database dumps.

But wouldn’t it be nice if there was some standard way to get access to whatever structured data any organization wants to expose?

Right now there are conventions for websites about exposing sitemaps that tell web crawlers how to navigate the sites. And there are plenty of loose conventions about how websites are organized. But there’s really nothing about structured data.

Now of course today’s web is primarily aimed at two audiences: human readers and search engine crawlers. But with Wolfram|Alpha and the idea of computational knowledge, it’s become clear that there’s another important audience: automated systems that can compute things.

There are product catalogs, store information, event calendars, regulatory filings, inventory data, historical reference material, contact information—lots of things that can be very usefully computed from. But even if these things are somewhere on an organization’s website, there’s no standard way to find them, let alone standard structured formats for them.

My concept for the .data domain is to use it to create the “data web”—in a sense a parallel construct to the ordinary web, but oriented toward structured data intended for computational use. The notion is that alongside a website like wolfram.com, there’d be wolfram.data.

If a human went to wolfram.data, there’d be a structured summary of what data the organization behind it wanted to expose. And if a computational system went there, it’d find just what it needs to ingest the data, and begin computing with it.

Needless to say, as we’ve learned over and over again in building Wolfram|Alpha, getting the underlying data is just the beginning of the story. The real work usually starts when one wants to compute from it—so that one can answer specific questions, generate specific reports, and so on.

For example, in our recent work on making the Best Buy product catalog computable, the original data (which came to us as a database dump) was perfectly easy to read. The real work came in the whole rest of the pipeline that was involved in making that data computable.

But the first step is to get the underlying data. And my concept for the .data domain is to provide a uniform mechanism—accessible to any organization, of any size—for exposing the underlying data.

Now of course one could just start a convention that organizations should have a “/datamap.xml” file (or somesuch) in the root of their web domains, just like a sitemap—rather than having a whole separate .data site. But I think introducing a new .data top-level domain would give much more prominence to the creation of the data web—and would provide the kind of momentum that’d be needed to get good, widespread, standards for the various kinds of data.

What is the relation of all this to the semantic web? The central notion of the semantic web is to introduce markup for human-readable web pages that makes them easier for computers to understand and process. And there’s some overlap here with the concept of the data web. But the bulk of the data web is about providing a place for large lumps of structured data that no human would ever directly want to deal with.

A decade ago I suggested to early search engine pioneers that they could get to the deep web by defining standards for how to expose data from databases. For a while there was enthusiasm about exposing “web services”, and now there are all manner of APIs made available by different organizations.

It’s been interesting for me in the past few years to be involved in the emergence of the modern data community. And from what I have seen, I think we’re now just reaching a critical point, where a wide range of organizations are ready to engage in delivering large-scale structured data in standardized forms. So it is a convenient coincidence that this is happening just when it becomes possible to create a .data top-level domain.

We’re certainly not sure what all the issues about a .data TLD will be, and we’re actively seeking input and partners in this effort. But I think there’s a potentially important opportunity, so I’m trying to do what I can to provide leadership, and further help to accelerate the birth of the data web.

[This post also appears on Stephen Wolfram's blog---Eds.]

Comments | Reprints | Share:          

Google is changing its mind about what’s relevant.

In a sweeping technical overhaul that will start to go into effect today, the search giant is altering the way it ranks search results to highlight content that users have shared, or that has been shared with them. If you’re logged into Google (NASDAQ: GOOG), links to this personal and social content—such as photos you’ve uploaded to Google+ and posts shared by the people in your Google+ circles—will now appear at or near the top of search result pages for the appropriate keywords.

For example, if you’ve ever shared photos of your cat Mittens on Google+ and you type “Mittens” into the Google search box, those images are likely to be among the top results, alongside links to Web pages about winter wear.

The personalized results won’t include information users have shared outside of Google’s ecosystem—say, status updates on Facebook or photos on Flickr (or, notably, tweets on Twitter). And users will have the option to turn off the new personalized results if they find that the information is getting in the way of their routine Web searches. But with the changes, Google is betting that most people who use its flagship search engine will prefer to see information they or their friends have created or shared over pages published by faraway people they’ve never met.

“Relevance is what we do,” says Sagar Kamdar, group product manager for search at Google’s Mountain View, CA, headquarters. “It’s our bread and butter. We are blending this content to optimize what we see as relevance.”

A mock preview of personalized search shows results both for chikoo, a fruit that grows in India and Pakistan, and Chikoo, a dog belonging to Google Fellow Amit Singhal.

Which is to say that if Google’s new algorithms have some reason to think you’re actually looking for mittens rather than Mittens, your kitty might get demoted. “If you have a personal result that is not as good as the Web result, the personal result will be visible in the appropriate place on the search result page,” Kamdar explains.

Alongside the personalization features, Google is making it easier to find information about people you know, in part by changing Google Instant to show links to the profiles of people in your Google+ circles. Google Instant is the technology rolled out in 2010 that predicts the search keywords you want as you type a query. With today’s changes, entering the first few letters of the name of someone in your Google+ circles will be sufficient to surface their Google+ profile in the autocomplete list.

In addition, Google will do more to direct users to content from people they may not know personally, but might like to follow via social media. The names of prominent Google+ users will be included in the Google Instant autocomplete predictions—type “Trey R,” for example, and you’ll see a link to the Google+ profile for Trey Ratcliff, a professional photographer who has made extensive use of Google+ photo albums. And in the right column of Google search result pages—the area normally reserved for keyword-based advertisements—the company will begin promoting the profiles of prominent Google+ users associated with certain search queries.

In the example Google Fellow Amit Singhal shared in a blog post announcing the changes today, a search for “music” summons Google+ pages for Britney Spears, Alicia Keys, and … Next Page »

Comments | Reprints | Share:          

Merry Christmas, Boston. You asked for it, and you got it. A big, publicly traded consumer tech company to put us on the map alongside the Silicon Valley bad boys and uppity New Yorkers. I present to you: TripAdvisor (NASDAQ: TRIP).

Sure, we already have Zipcar (NASDAQ: ZIP), Carbonite (NASDAQ: CARB), iRobot (NASDAQ: IRBT), and privately held but well-established companies like Wayfair, Kayak, and Harmonix. But TripAdvisor is different. Although the online travel firm is not new—it’s been cranking here in Boston for more than a decade—it has become one of the biggest consumer-focused Internet companies on the East Coast, with more than 1,100 employees; 50 million-plus unique visitors a month checking out hotel, restaurant, and travel reviews; and, oh yeah, a market cap north of $3 billion. Yet it hasn’t received as much media coverage or tech-community-adulation as you might expect over the years. (An exception to the former would be this in-depth story by my colleague Wade Roush; the latter would be this commentary from investor Chris Dixon.)

I spoke with TripAdvisor CEO Stephen Kaufer on Wednesday, the day his company officially became independent from Expedia (NASDAQ: EXPE) and started trading on the Nasdaq under its own stock symbol. Here are my takeaways from our chat:

1. TripAdvisor wants the spotlight now. “We have generally been very, very surprised at how little attention the press have paid to TripAdvisor,” Kaufer said. He pointed to his company’s size, number of employees, traffic, revenues, and profits, calling it “the $3 billion company in our own backyard.” On the local training and ecosystem front, he says college interns are turning down offers from Facebook and Google and working at TripAdvisor instead.

2. TripAdvisor is global. Seventy-five percent of the traffic to the company’s branded websites comes from outside the U.S. Think about that for a minute. “We’re almost unchallenged in most countries in the … Next Page »

Comments | Reprints | Share:          

It’s probably the quietest Boston-area public offering of the year. But why? Maybe because it’s a spinoff from an already public company, and the deal was announced back in April. Or maybe because New England doesn’t go wild for its consumer-focused tech companies the way some other regions do. Or maybe everyone is just sick of online user reviews (and planning holiday travel).

For whatever reason, tomorrow’s first trading of Newton, MA-based TripAdvisor’s stock on the Nasdaq under the symbol “TRIP”—and the company’s official spinoff from Bellevue, WA-based Expedia (NASDAQ: EXPE)—doesn’t seem to be getting talked about or celebrated that much in local tech circles. And that’s probably just fine with TripAdvisor, which has always been understated when it comes to PR. (Its stock has already been trading under “TRIPV” for the past couple weeks.)

The company, which has more than 1,000 employees (doubled in the past two years), is a major international force in online travel and has been raking in the cash for the better part of a decade. TripAdvisor made $486 million in revenue in 2010 and is on pace for more this year. From what I can tell, the firm has never been particularly chummy with Boston media, marketing, or trade organizations. It does recruit heavily in the area though.

TripAdvisor started in 2000 as a search engine to help people find reviews of travel destinations. The company quickly became a destination in itself for user-generated reviews, and it figured out the lead-generation model before most people had even heard of lead-gen. The company makes money from advertising and referral fees when consumers book flights and hotels through its partner sites. One tidbit: Founding CEO Stephen Kaufer told my colleague Wade last year that TripAdvisor was “a couple months shy of going out of business” in late 2001 before it figured out its revenue model.

In 2004, TripAdvisor was bought by Internet giant IAC (NASDAQ: IACI) and became part of Expedia, which itself was spun out from IAC the following year. TripAdvisor the company has grown to encompass nearly 20 travel sites (including TripAdvisor.com, SeatGuru.com, and BookingBuddy.com) and is expanding aggressively in Europe, China, and other locales.

It sounds like going public should raise TripAdvisor’s international profile, but being independent of Expedia won’t change things much. The companies basically operated separately over the past seven years. One broader trend to watch: whether user-generated content for travel, hotels, restaurants, and other businesses remains a lucrative property to own, or whether it ends up getting lost in the noise of bigger and badder marketing devices of the coming decade.

Locally, there’s an interesting juxtaposition of TripAdvisor with other prominent travel tech firms. Just as TripAdvisor is going independent, Cambridge, MA-based ITA Software is being integrated into Google (NASDAQ: GOOG) as the search giant ramps up its own travel-site capabilities. And Kayak, the Connecticut-based travel meta-search company with a strong Boston presence, filed for an IPO late last year but has been waiting out the market and evolving its own offerings in this super-competitive sector.

Comments | Reprints | Share:          

I didn’t know the full extent to which the Boston area has a thriving data and analytics startup scene.

I had always associated the city primarily with biotech innovation. My company, Chart.io, provides hosted business dashboards to help companies visualize their database data. We’re based out in San Francisco (we were part of the 2010 Y Combinator class), but our investors, Avalon Ventures, call Boston home. When my friends at Avalon-backed Kinvey (mobile backends as a service) and Boston-based SessionM (a platform to spark deeper consumer engagement with mobile content and ads) and I decided to co-host a data visualization and analytics meetup for the local community, we expected to get 20-30 RSVPs at most. Instead, we broke 100 in a flash and saw a steady torrent of emails from data enthusiasts pleading for admission.

In fact, a deeper look into the Boston tech scene reveals quite a rich history of data and analytics companies, including Netezza, Endeca, ITA, EMC, and other giants. And it turns out, the startup scene is equally rich, with companies innovating around NoSQL, data storage, search, healthcare, and a variety of cloud computing ventures. Here’s a quick tour of the Boston- data landscape. And this is only the tip of the iceberg.

As data volumes have exploded in the past decade, so have the number of companies building tools to store, retrieve, analyze, and generally manage the deluge of data.

Two Boston-area companies, Cloudant and Basho, are tackling the big data problem through non-relational databases (NoSQL), designed to handle hundreds of gigabytes and even terabytes of data and enable applications to elastically scale out to meet the demands of millions (or hundreds of millions) of concurrent users. In this vein, Cloudant offers tools to help companies use Apache CouchDB, while Basho developed its own data store called Riak.

Meanwhile, other local firms are focusing on the next generation … Next Page »

Comments (3) | Reprints | Share:          

Boston-based Skyword is moving up in the world. The startup, which makes a software platform for brands to create Web content, said today it has closed $6 million in financing from Cox Media Group.

The company says it helps publishers—which is marketing-speak for retail and industry brands, as well as media companies—create online content “for a search and social driven world.” In other words, Skyword farms out marketing content to writers with experience in everything from medicine to investing to parenting, and tries to make the content discoverable by consumers via search engines and social networks. (Depending on whom you talk to, this is either one of the savviest or most annoying ideas ever. Or possibly both, if you view marketing as a necessary evil.)

Skyword is led by founder and CEO Tom Gerace. The company started up about a year ago, out of the news and social-networking site Gather (which still exists). Skyword’s customers include Procter & Gamble, Everyday Health, and ImpreMedia.

Comments | Reprints | Share:          

Into the lair of beasts strode Adam Goldstein. Armed only with his wits and a mean set of slides, he descended on the Boston area on a warm, early winter day. He was no stranger to the premises. Goldstein had been an MIT undergrad before moving to San Francisco to participate in the Y Combinator startup program with his online travel company, Hipmunk.

Goldstein, the startup’s CEO and co-founder, spoke at Xconomy’s “6×6: Six Cities, Six Big Tech Ideas” conference earlier this month, representing the Bay Area. I say he was among beasts because Boston is the land of heavyweight travel firms such as ITA Software (now part of Google), Kayak, and TripAdvisor, and upstarts like Hopper, WaySavvy, and SilverRail (now based mostly in the U.K.). And Goldstein has been on record saying other travel sites “have really dropped the ball on flight search.” (On the other hand, Hipmunk formed a licensing partnership with ITA about a year ago.)

Indeed, the whole culture of Hipmunk is about coming “into an established industry with a focus on usability,” says Goldstein, in a polite-but-firm jab at the big players who don’t seem to care as much about being user-friendly. (As for the company’s name, let’s just say the cute-animal logo is its main justification.)

I must confess, I was skeptical at first. Since part of me still lives in the ’90s (the early to mid-‘90s, mind you), any bluster from new travel sites tends to fall on numb ears. Most travel sites seem pretty much the same, and even the worst ones are still more convenient than what people like me used to do, which is call up travel agents and individual airlines, get some options, and repeat until settling on a purchase. Hipmunk is about making the whole search process simpler, more intuitive, and more visually interactive.

But there’s only so far that can take you as a business, as my colleague Wade probed a few months ago. What stood out to me most about Hipmunk is its strategy of building a business by focusing first on getting lots of loyal customers—not trying to cash in on every eyeball.

“The entire world, especially in the world of travel, has become sort of addicted to the idea of making as much money as possible from each time someone visits their website,” Goldstein said. “What that’s led them to do over time is bombard their customers with advertisements and pop-up windows and all sorts of other things that just distract them.” Conclusion: Hipmunk won’t make money from ads, just referral fees when people book trips. But it does need to gain users—lots of users.

Here’s a short video interview with Goldstein, conducted by my colleague Lilly O’Flaherty. I like the part at the end where he references a talk by Northrop Grumman’s Bill Walker, also at 6×6, on high-altitude UAVs (unmanned aerial vehicles). Heck, maybe someday an entrepreneur will pitch an idea for a company that’s a “Hipmunk for UAVs.”

Comments | Reprints | Share:          

From its very early days, one of the core appeals of online real-estate marketplace Zillow has been looking up your house (or your neighbor’s, or anyone else’s) on a digital map to see how much it might be worth.

“For us, really more than any other app in the category, we start with a map. And we think location is super-important,” says Jeremy Wacksman, Zillow’s senior director of marketing and mobile strategy.

So when uber-retailer Amazon.com came out with the Kindle Fire, Zillow (NASDAQ: Z) needed to find a way to include mapping with its app for the new entry-level tablet. Pretty simple, right? Zillow’s already got location-enabled apps on the major mobile platforms.

But not so with the Kindle Fire. As a much more stripped-down device, primarily intended for media consumption and shopping, the Kindle Fire doesn’t have GPS capabilities. And even though the Kindle Fire is based on a version of Google’s Android mobile operating system, Amazon has aggressively asserted its independence from Google with the Kindle Fire.

That has meant keeping Google Maps out of Amazon’s Android app store, and telling Kindle Fire developers not to include any features based on Google’s Mobile Services—even though that meant the mega-retailer didn’t have its own in-app purchasing system when the device was rolled out.

So, to make its maps work on the Kindle Fire’s version of Android, Zillow’s app offers a mobile version of its regular online maps—which are already supplied in almost all cases by Bing. GPS-enabled services are turned off on the Kindle Fire app, but users can still search for an address or location to find homes they’re interested in scoping out.

That’s why you saw Zillow making a big deal out of the fact that it was “the only map-based real estate app on the platform” when it announced the app as one of the first to be available for the Kindle Fire. ”A lot of other apps that utilize Google services maybe just took them out or toned them down,” Wacksman says. “But for us, this was kind of core.”

No, getting in as the default mapping service on one app isn’t going to make Bing rocket ahead in market share—the latest figures continue to put Bing-powered search at around 30 percent of the U.S. market. And the search business is still a huge money-loser for Redmond.

But it looks like Microsoft can continue getting small wins for its search service when other big tech companies want to play keep-away from Google. They’ll take it.

Comments | Reprints | Share:          

Stephen Wolfram is not one to be summed up in a few pithy quotes. Well, too bad.

Here is his life in a nutshell:

“I grew up in England and went to all sorts of good schools that I thought were completely irrelevant.”

“By the time I was 20 years old, I was a physics faculty member at Caltech, and I was building a big software system that was a forerunner of Mathematica.”

“Along the way, I learned a lot about what not to do in starting a company.”

“For about a decade I was almost a complete recluse, running the company from a distance, and spending every night working on basic science.”

“I don’t really have a boss. I just do what I want to do. The trick is not to have a private company that gets too weird and too pathological.”

That was a sampling of what the distinguished and controversial Wolfram had to say at our Xconomy Forum in Boston last week, called “6×6: Six Cities, Six Big Tech Ideas.” For those who don’t know, he is the founder and CEO of Wolfram Research, the creator of Mathematica and Wolfram Alpha, and the author of A New Kind of Science. He is also a recently minted Xconomist.

Wolfram, 52, set the table for the theme of our event, which involved some of the biggest ideas in technology and business from entrepreneurs and executives around the country. As only he could, the physics and software guru reflected on his 30 years in the tech industry and his contrarian approach to running big projects and building companies.

A few things really stood out to me in his talk. One was the importance of making mistakes early in his career. While at Caltech in the early ‘80s, Wolfram got into a “grisly early-IP-meets-university” battle over his software tools, he said.

“I ended up deciding I had to start a company around the software system I’d built, and of course I was just a physics kid. I didn’t know anything about starting companies,” he said. “I made lots of mistakes, like not running the company myself, hiring a CEO who was twice my age, and so on. The company quickly started doing things that I thought were silly and boring. In the end, after many trials and tribulations, it did in fact survive and finally went public and was gobbled up by bigger fishes.” (You can read more about his first startup, Computer Mathematics, which was venture-backed and later merged with Inference, here.)

Another thing Wolfram figured out early on was that university work was not for him. After spending time at Caltech and the Institute for Advanced Study in Princeton, NJ, he started a research center at the University of Illinois at Urbana-Champaign to study complex systems and complexity theory. “My plan A was to get lots of other people to help work” on the implications and applications of his findings, he said. “It was OK, but it was really slow. I got kind of frustrated and needed a plan B,” he said. “My plan B was to build … Next Page »

Comments | Reprints | Share:          

Seattle startup Wavii, previously seen taking the concept of “stealth mode” to extreme lengths, is letting slip more details about the product it’s been building for these many quiet months. In a pair of blog posts today, the startup says it’s using machine learning to sort content from across the Web into feeds—as they say here, “Making Facebook out of Google.”

Yes, a variation of the well-worn startup pitch “it’s like X for Y.” To be fair, that’s not a bad way of relating a bunch of really nerdy computer science stuff to garden-variety online content consumers, which looks like the target audience.

Wavii aims to sort through the mountains of information online and present related pieces of content together in a stream, as if there were a person curating the feed. That’s what Facebook (or Twitter) users do for their followers: Call pieces of information out of the ether and pass them along to others.

Wavii’s reference to Google points to the fact that, while they’re very good at finding information, search engines still rely on pretty specific queries that hit the right keywords in more or less the right order. Commercial search isn’t so great at answering open-ended or natural-sounding questions, like “what’s the traffic like on I-5 right now?“

That’s one of the things University of Washington search expert Oren Etzioni has been studying, and kind of harping on, in his spare time. As I wrote in this profile of Decide, the e-commerce startup that Etzioni co-founded, a technology called open information extraction actually does allow computers to comb through written materials and decide whether the content is, in fact, relevant to a specific topic.

Decide uses open information extraction to comb through product announcements, press releases, blog posts, and news reports to figure out whether a newer model of a particular electronics gadget is coming out soon. If you wanted to do that yourself, you’d have to Google a string of keywords and scroll through several pages of results to find the right information. Decide’s machines can do it automatically and continuously by parsing the written material already out there.

It looks like Wavii is using a similar approach to curate content about pretty much any topic a user might want to follow. In today’s post from someone identified only as Dan (stealth mode), Wavii labels its version of this technology “big data machine learning.”

“The key to making this technology work is teaching the computer to recognize the things we care about when they’re being talked about, even when they are expressed in tens or sometimes even hundreds of different ways, all without making too many mistakes. Once the computer has this, it can easily build the feed items you’re used to,” the company wrote.

Wavii hinted strongly at this approach in an earlier guest post on TechCrunch, using the wedding of Prince William as an example. Former TechCrunch editor Michael Arrington also reported some of these concepts in an earlier post about Wavii, based on a test version he saw. So I guess we can officially declare an end to this particular tour through “stealth mode,” or as GeekWire’s John Cook recently put it, “double secret stealth mode.”

I e-mailed CEO Adrian Aoun to see if he could tell me any more about Wavii at this point. At last report, from Arrington in July 2010, the company had raised about $2 million from various angel investors.

Comments (1) | Reprints | Share:          

“Marcus Welby is dead.”

So says West Shell, the CEO of San Francisco-based Healthline. Not that Dr. Welby was ever actually alive—the fictional physician, played by Robert Young, made house calls for ABC TV from 1969 to 1976. But Shell’s point is that few people these days have a family physician like Welby to whom they can easily turn with medical questions. These days, says West, “The doctor is your third opinion”—the first two being the Web and, you guessed it, the Web.

Shell says 170 million Americans turn to the Internet every month for help managing their health. And Healthline, the 130-employee company he’s been leading since 2005, captures a huge portion of them: roughly 100 million per month, spread across the many Web properties, such as Yahoo Health, AARP.com, and DoctorOz.com, that use the company’s search tools and medical content. It’s one of the three largest U.S. providers of consumer health information on the Web, the others being WebMD and Everyday Health.

What sets Healthline apart from its competitors, according to Shell, is its focus on technology. Healthline’s core asset is a health-specific search engine programmed with a vast set of categories and definitions—a “semantic taxonomy,” to use the search industry term. The taxonomy helps consumers find the information they need, even if they don’t know the right keywords to use. To take a simple example, the Healthline taxonomy knows that “Lou Gehrig’s disease” and “amyotrophic lateral sclerosis” are the same thing, or that “brittle bone disease” is technically known as “osteogenesis imperfecta.”

Healthline CEO West Shell

“Doctors speak in Latin, insurance companies speak in billing codes, and patients speak in English,” says Shell. “These terminologies are incredibly difficult. But at the end of the day, the reason we beat Google and Microsoft and Yahoo in terms of being able to search health information and deliver it is that it’s the most complex information retrieval problem there is. Over half of the staff here are engineers, medical informatics specialists, doctors, pharmacologists, and nurses. That’s our unfair advantage.”

Much of the information available on Healthline is written by the company’s own researchers and contract writers. But it’s also a rich source of articles and videos syndicated from other sources around the Web, such as 5min, a library of instructional videos owned by AOL. Healthline also offers one of the Internet’s most comprehensive symptom search services. If you’re suffering from, say, excessive yawning, Healthline will give you a long list of potential causes, from insomnia to sleep apnea to a heart attack, and share information on potential cures and costs, as well as leads on local doctors who specialize in treating these conditions.

Healthline has raised about $50 million since 2005 from VantagePoint Venture Partners, Investor Growth Capital, and a group of strategic investors including General Electric, Kaiser Permanente, Aetna, US News, and Reed Elsevier. It’s looking like those backers made a good bet. The company’s business is growing at a rate of 50 percent per year, according to Shell. This fall, Healthline ranked 75th on Deloitte’s list of the 500 fastest-growing technology companies in North America.

That’s a remarkable turnaround for a dot-com era company that nearly disappeared from existence in the mid-2000s. Originally known as YourDoctor.com, the site was created in 1999 by James Norman, M.D., an endocrine specialist and Internet health education pioneer. YourDoctor.com built the foundations of a semantic taxonomy for health, but after the bust, the startup just couldn’t … Next Page »

Comments (2) | Reprints | Share:          

Today’s main infotech feature on Xconomy San Francisco is about Ask.com, the Oakland, CA-based search company that has been busy transforming itself into a question-and-answer service powered by an army of community members. If you can’t find an instant answer through Ask.com’s Web search tool, you can easily pose it to other Ask.com users. Depending on the subject, you might get multiple answers within minutes.

As I was writing that piece, there was one nagging thing I couldn’t figure out. Why do people bother to answer the questions? Nobody at Ask.com gets paid actual cash, as the “guides” at human search site ChaCha do. There’s no system of points and levels, like the one at Yahoo Answers. There’s no credit mechanism like the one at Quora.

There is one way to earn concrete rewards at Ask.com—but the money goes to breast cancer research, not to you. If you volunteer to support the Susan G. Komen For The Cure Foundation, Ask.com will donate 25 cents to the foundation every time you post an answer. As far as I can tell, something like half of the active users at Ask.com are signed up for this program.

But the question remains: why spend your leisure time contributing free content to Ask.com—a unit of media giant IAC/InterActiveCorp—when you don’t see a penny of the revenue the company earns by showing ads on the site? It’s the exact opposite of the “free rider” problem in economics, where people consume resources without paying for them. In this case, people are adding resources without asking to be paid.

Of course, you could ask the same question about Wikipedia or any of the Web’s many crowdsourced sites. And you could venture all sorts of hypotheses—maybe people are just killing time, or burnishing their online reputations. But I finally got the brilliant idea of asking Ask.com users themselves why they contribute.

Within an hour of posting my question, I’d received more than 25 responses. I quoted a few of them in the feature article, and in this space I wanted to show you the whole set (minus a couple of trollish ones). The answers are pretty illuminating for anyone interested in community dynamics on the Web—especially the longer ones from “Dozy,” “VirginiaL,” and “Barbarabliss.” The bottom line seems to be that people just want to help.

My personal thanks to everyone who contributed answers.

The Question:

Why do people answer questions on Ask.com? I’m not asking in a snarky or ironic way — I’m genuinely curious about what motivates you to answer questions here, especially given that Ask.com doesn’t use points, badges, rankings, or other systems for rewarding or recognizing your contributions.

The Answers:

GuardGirl: i guess we’re just curious to see what other people are thinking and i personally like to help people:)

TheHelpFinder: They want to be helpful, that’s all.

pumpkinman: i guess just to help people with what they need thats all

Daisy46: It’s fun! and i like to give people accurate information that will actually help them. It makes me feel like I am making a tiny positive difference in that persons life.

666_Maggots: to have a purpose in life…

bellydncr: I guess some of us still enjoy trying to help even if we don’t get gold stars.

persiancat: I like to answer questions to help people…. Especially if I have personal experience

… Next Page »

Comments (3) | Reprints | Share:          

What’s the best way to tell my mother-in-law that we’re not going to visit her at Thanksgiving?

My college friend has taken over my friendship with another guy, and now both are ignoring me. Should I ask them why?

What do you do if the girl you asked to marry you says no?

[Questions posed by Ask.com users, November 15, 2011]

Google, the Mountain View, CA-based search and advertising giant, may have a 66 percent share of the search market, but there are plenty of questions its Web crawlers and ranking algorithms just can’t answer. That includes squishy, emotion-laden questions like the ones above, as well as detailed, local queries like “what’s the best place for Mexican take-out in St. Cloud, Minnesota?” For those types of questions, there’s just no substitute for bringing real humans into the loop.

Across the bay from Google, in downtown Oakland, there’s a 15-year-old search company that’s now banking on the power of these human connections over algorithms. It’s called Ask.com, and it’s the perpetual fourth-place finisher in the search engine races, after Google, Yahoo, and Microsoft’s Bing. Last year, Ask.com gave up on its long struggle—financed by Barry Diller’s IAC/InterActiveCorp, which purchased the company for $1.85 billion in 2005—to compete with Google in traditional Web search. Under Doug Leeds, who became CEO in mid-2009 after a stint at the helm of IAC property Dictionary.com, Ask.com decided to transform itself into a question-and-answer service. And as of this fall, the transition is complete: in September the company opened the live Q&A portion of its site, previously in beta testing, to all 63 million monthly visitors.

Ask.com's iPhone app

The change marks a return, in some ways, to Ask.com’s heritage as Ask Jeeves. Early in its history, the company’s claim—symbolized by its butler mascot Jeeves—was that it could find definitive answers to natural-language questions such as, “Who was the world chess champion in 1956?” or “What’s the difference between a leopard and a cheetah?” The problem was that the claim didn’t always hold up; the computational problem was just too hard, often leaving Jeeves at a loss. (Even today, search utilities like Apple’s Siri can only answer natural-language questions reliably within restricted neighborhoods of knowledge, such as the weather.) In 2001, Ask Jeeves bought an algorithmic search engine called Teoma and started spewing out long lists of blue links just like Google’s.

The company continued to welcome questions phrased in natural English, and it tried to siphon traffic away from Google by marketing itself as a question-answering site. But “we started talking to users in a way that was different from the way they wanted to talk to us,” Leeds told me in an interview this summer. “What they really wanted was an answer at the top of the page, not a list of links.” Even the so-called “smart answers” that Ask.com offered at the top of a search result page weren’t addressing the need. “As good as we think we got at this, at least 60 percent of the questions that come in every day are things we can’t answer from the information we crawl,” says Leeds.

It turns out that people are really good at coming up with questions that have never been asked or answered before. So Ask.com decided to shift strategies again. “Toward the end of 2009 and through 2010, we built a platform not just for searching documents, but for searching people—to give them a chance to answer the questions,” says Leeds.

Today, when you arrive at the Ask.com website or fire up the Ask.com iPhone or Android apps, you see a big search box where you can type in your question. If the service can find a definitive answer somewhere in the database of 700 million question-answer pairs it has culled from the Web, it will simply show you the answer. (Question: Is a Granny Smith apple more tart than a Red Delicious? Answer: Yes.) If it can’t, it will show you a list of related Web results. And if those results don’t answer your question, you can click on the “Q&A Community” tab, where you can submit the same question for review by other Ask.com users.

There’s no guarantee anyone will answer your question, but in my limited experiments, I’ve always received at least one answer within an hour. “The biggest challenge for us is how to integrate these two services,” Leeds says—meaning the Web answers that arrive in milliseconds and … Next Page »

Comments (1) | Reprints | Share:          

There are roughly 500,000 iPhone and iPad apps in Apple’s iTunes App Store, and almost that many smartphone and tablet apps in Google’s Android Market. That gives mobile consumers lots of choices, but it has created an untenable situation for mobile developers. Unless you get lucky and your app vaults onto the top-5 or top-10 charts, or is anointed as a “New and Noteworthy” or “Featured” app by a human curator at Apple or Google, it’s virtually impossible to get noticed amidst all the noise. As a result, there’s a very long tail of perfectly good apps that are failing to find their natural audiences, simply because mobile users have no way to discover them short of browsing page after page of poorly organized lists in the app stores.

I’ve been covering technology long enough to remember when there were 500,000 sites on the entire World Wide Web. That was back in mid-1996, when Yahoo-style guides and directories were still considered the best way to find new stuff. As the Web swelled—to 1.7 million sites by December 1997 and 3.7 million by December 1998—the directory model quickly became unworkable, and people started turning to first-generation search engines like AltaVista. But search results in these early days tended to be pretty random, and vulnerable to manipulation through spamdexing schemes. It wasn’t until Google came along with its Page Rank algorithm in late 1998 that Web surfers finally had a reliable way to locate high-quality content.

The app world hasn’t yet had its Google moment—which is more than a bit ironic, considering that Google itself runs one of the two largest app stores. Just try searching on the term “restaurant guide” in iTunes or the Android Market. The top result at iTunes is something called VegOut, and the top result at the Android Market is the U.S. Army Survival Guide. I kid you not.

In any rational universe, the top results for “restaurant guide” in both stores would be Yelp and Urbanspoon. But at iTunes, these apps don’t even appear in the first 180 results. Zagat, which would be a logical number 3 result, does turn up in the 17th position at the Android Market, but that’s probably just because Google now owns it. The overall rankings are so goofy that even AltaVista couldn’t have come up with them.

Chomp CEO Ben Keighran

Entrepreneurs aren’t waiting for Apple and Google to fix the mess they’ve created; several startups now offer alternative ways to find great mobile apps. The one with the biggest lead is probably Chomp, which is based here in San Francisco. Chomp’s own app for searching apps is available for both the iPhone and Android phones. In many ways, it’s what the iTunes App Store and the Android Market should be—a fact that Verizon recognized in September by announcing that it would build Chomp into the Verizon app store that ships with all Verizon Android phones.

Lately I’ve been getting to know Chomp co-founder and CEO Ben Keighran, an Australian-born programmer-entrepreneur who moved to the Bay Area about six years ago. “Search is really broken on both Apple and Google for searching for anything other than the name of an application,” Keighran says. “It’s just like the Web. Search wasn’t important at the beginning of the Web; what people needed was a curated directory. Search wasn’t important at the beginning of the app store revolution. And now it’s an incredibly broken feature.”

Before Chomp, Keighran was best known as the creator of a Java-based text messaging service called Bluepulse, which, at its peak around 2006, was handling 10 million messages per day for mobile subscribers in India, South Africa, and other countries. Bluepulse actually started out as an app store, so Keighran has been thinking about the problem for a long time. “Technically, it was a lightweight, 63-kilobyte browser, and it downloaded a list of apps you had said you wanted to use, and you would launch the apps within the browser,” Keighran recounts. “But it wasn’t used as much as the messaging feature, so it kind of got rolled into the messaging product.”

Keighran moved Bluepulse from Sydney to San Mateo, CA, in 2006 and raised almost $10 million in venture funding for the company in hopes of … Next Page »

Comments (11) | Reprints | Share:          

Updated 11:30 am 11/3 with slides
Sorry, experts: Information consumers don’t particularly care about all your highfalutin’ degrees and special titles. For a surprisingly large swath of online queries, friends and other real-world users are the key sources of opinions, says Stefan Weitz, Microsoft’s director of search.

“People don’t know who to trust. They don’t know who experts are. I can claim to be an expert in search, but you have to believe me,” Weitz said Wednesday in a presentation at the first Seattle Interactive Conference.

Like a true data geek, Weitz had a truckload of statistics, research, charts, and graphs to illustrate the importance of social signals in influencing user behavior online. But the short version is summed up well by this quote: “Humans crave other humans inside of search.”

There is an exception to the no-experts finding—when it comes to technical problems, people online actually want to see expert opinions, Weitz says.

But for every other example of some very common queries that Microsoft studied, the opinions of friends or other regular users were vastly preferred. Here’s that key slide from the presentation, and the rest is embedded below via Slideshare:

That’s significant for Microsoft, of course, because of the company’s deal with Facebook that gives Bing access to public Facebook data. Bing now incorporates that information into its search results.

“The bottom line is people are using social networks, in this case Facebook is the dominant one, to pose questions, to get opinions from people they know,” he said. “And in case you didn’t know, Bing has a relationship with Facebook.”

It’s an interesting relationship. Facebook hasn’t gone full-force into Web search, instead relying on its Bing partnership. Some who study the sector, though, say Facebook will eventually do search itself—potentially leaving Microsoft behind.

That would be a big letdown for Bing, which is growing in share of the search market (and powers search for poor old Yahoo) but is still well behind the search market leader, Google.

For his part, Weitz said the tracking of quality social signals will become even more important over time, as the amount of information online and the volume of search traffic increases, Weitz said.

For instance, search in general still expects people to put the right combination of keywords into the search box when they’re looking for something they don’t know anything about. “If you think about that, it’s chaos,” he said.

And while search gurus are trying to figure out how to make algorithms better at answering questions, Bing finds that users are going to their social graph when they want to ask a question and get an opinion.

If your garden-variety Internet user gets entrenched in that kind of split—social networks to answer questions and search to gather raw data—you can see how hard it might be for straight-up search tools to break Facebook’s hold and truly catalog all the valuable information out there.

“The cold math that we have used for 15 years now to help refine results to your particular keyword is not something which was sustainable long-term,” Weitz said. “You need that emotional connection to actually help you rationalize your way through the decision you have to make.”

Comments | Reprints | Share:          

There are two kinds of people: Those who already own an iPhone 4S, and those who don’t but have seen Apple’s Siri ads and wish they did. Siri, of course, is the voice-operated personal assistant that can read your text messages, schedule appointments, check on the weather, make waffles, and babysit your toddler (just kidding about those last two). It’s the coolest feature of the latest Apple smartphone, which went on sale last week. Virtually overnight, it has brought state-of-the-art artificial intelligence technology into the consumer mainstream.

Well, it turns out that there’s more where that came from. The same Silicon Valley defense project that gave birth to Siri—called CALO, for Cognitive Assistant that Learns and Organizes (more on this below)—has another, lesser-known spinoff, a Palo Alto news personalization startup called Trapit. The company’s web-based service uses AI to organize the welter of new content that appears online every day into tailored collections called “traps.” And while the technology is still a bit raw, there’s a chance that it could have the same kind of impact in the world of Web search and online news that Siri is beginning to have on mobile interfaces.

I’ve been playing with TrapIt for about three months now. It hasn’t become a part of my daily news-browsing routine, but I can definitely see that happening if the startup continues to refine the interface, improve its search algorithms, and make the site more tablet-friendly. (Trapit took the lid off its service in June, but it remains in closed beta testing, which means you have to request an invitation to get an account. The wait was short when I registered. The company says it’s going to open the beta version of its service to the whole public later this fall.)

The first thing to try when you go to Trapit is either to browse one of the existing, featured traps—which are often related to breaking news, such as yesterday’s killing of Muammar Qaddafi—or start one of your own by entering a phrase or keyword into the “Discover” bar. After a short wait, you’ll be presented with recent news stories and blog posts on your topic, culled from across the Web.

At first the selection may seem pretty random. The neat part is that as you peruse various articles, which pop up in lightbox-style windows, Trapit observes what you’re reading, how long you spend with each article, and what you’re sharing with others. It uses these cues and others to beef up its profile of your personal tastes, so that over time it’s able to surface more articles that fit your interests and fewer that don’t.

You can also train Trapit manually by clicking on the thumbs-up or thumbs-down buttons—and the more you do this, the faster the software will learn your preferences. As you create traps on new topics and train your existing traps, you can end up with a whole gallery of mini-magazines, exclusively tuned to the mix of subjects that you, and you alone, are passionate about. It’s a pretty unique service—the closest comparison I can think of is Google Alerts, which are like standing search queries with the results e-mailed to your inbox every day. But Trapit is more like regular Web surfing. It’s just that you’re surfing the Web you want.

Trapit’s AI-driven approach goes completely counter to the dominant trend in news curation today, which emphasizes the power of social networking and collaborative filtering. News aggregation apps like Flipboard, Pulse, AOL’s Editions, CNN’s Zite, Yahoo’s Livestand, and Google’s forthcoming Propeller platform may seem to provide personalized news feeds, but … Next Page »

Comments (13) | Reprints | Share:          

Big news out of Boston and the Bay Area today. Oracle, the Silicon Valley database and business software giant, said it has agreed to acquire Endeca Technologies, the Cambridge, MA-based enterprise search and e-commerce tech company. Terms have not been announced, but the deal is expected to close before the end of the year.

Endeca started in 1999, led by Steve Papa and Peter Bell, and has been an icon of the Boston-area tech scene for the past decade. Its investors have included Bessemer Venture Partners, Venrock Associates, Ampersand Ventures, Intel Capital, DN Capital, and SAP Ventures. As of early last year, the company had raised about $75 million in venture capital. No word yet on any restructuring or employee moves.

Perhaps it’s not surprising that Endeca would get scooped up by a West Coast giant, given that many Silicon Valley firms have been interested in the company for years. But it’s yet another Boston-built tech company that’s giving up local control (see ATG, Novell, ITA, Quattro, Where, Vertica…), if not heading West outright.

Redwood Shores, CA-based Oracle (NASDAQ: ORCL), for its part, says that together with Endeca it plans to “create a comprehensive technology platform to process, store, manage, search and analyze structured and unstructured information together enabling businesses to make stronger and more profitable decisions.” The press release singles out Endeca’s InFront Web commerce and Latitude software products as being especially important to integrate with Oracle’s existing packages.

Comments (1) | Reprints | Share:          

Ladies and gentlemen, your attention please. Ahem.

[A giant “6x6” fills the screen.]

Macho narrator voice:
Star Wars had The Empire Strikes Back (Vader: “I am your father”)
The Godfather had The Godfather Part II (Pacino: “You broke my heart”)
Mad Max had The Road Warrior…well, you get the idea.

Now Xconomy presents 6×6, the long-awaited sequel to its riveting, change-the-world program from last December, 5×5. (It was “5×5” because we weren’t in New York City yet. Now we are.)

Yes, for the second straight year, Xconomy has canvassed its national network to find the most original, sensational, and transformative tech ideas out there—across software, hardware, digital media, social technologies, robotics, and more. We are inviting a select few speakers to Boston for an afternoon of mind-bending presentations and business networking.

It’s all taking place on the afternoon of Thursday, December 1, from 1:30-5:30 pm, at the Fidelity Center for Applied Technology in downtown Boston. One featured speaker will be on hand from each of our six cities: Boston, New York, Detroit, San Francisco, Seattle, and San Diego. We hope “6×6: Six Cities, Six Big Tech Ideas” will be a rallying point for the Boston tech community to get together and discuss the future of their fields with our out-of-town guests.

The concept is to highlight some of the BIGGEST tech ideas out there—as well as the nuts and bolts of how founders are building successful businesses around these ideas. This event is about truly changing the world, so we’ve asked everyone to please check their daily deals, social network plug-ins, and run-of-the-mill mobile apps at the door.

Who better to set the table than Stephen Wolfram? We’ve invited the renowned scientist, inventor, and business leader to give the opening keynote. Wolfram is the founder and CEO of Wolfram Research and the creator of Mathematica and Wolfram|Alpha. So he knows a little bit about changing the world from a software, computing, and business perspective. Oh, and he also spent 10-plus years working to reinvent the entire landscape of modern scientific thought with his book project, A New Kind of Science. I’ll stop there, but you can read an interview I did with Wolfram around the beginning of last year. (As for what he’ll talk about on Dec. 1, it’s safe to say I have no freaking idea—but I’ll keep you posted.)

Here’s a quick rundown on who’s representing our six cities at 6×6:

—From the hometown of Boston, we have Dave Icke, the CEO of Cambridge-based MC10, a pioneer in developing flexible electronics and sensors for consumer, healthcare, and energy markets.

—New York City is sending Jason Baptiste, the CEO of OnSwipe, a scrappy young startup that is trying to invent the future of media through a new tablet publishing platform.

—Representing Detroit is Nathaniel Borenstein, the Chief Scientist of Mimecast, an e-mail management company headquartered in the U.K. Borenstein, who’s based in Michigan, is one of the fathers of modern e-mail systems, and will talk about the future of communication in the cloud.

—From San Francisco comes Adam Goldstein, the CEO of Hipmunk, an online travel search company that’s been making waves with its novel visual interface for finding flights and hotels.

—Seattle will be repped by Kabir Shahani, the CEO of Appature, a fast-growing startup specializing in social and relationship marketing technologies for the healthcare industry.

—And from the sunny climes of San Diego comes Bill Walker, Chief of Global Hawk Business Development at Northrop Grumman, the aerospace and defense tech giant. Walker will talk about high-altitude unmanned aerial vehicles (UAVs).

We also have a few intriguing “burst” bonus talks to highlight, from some of the most exciting startups around Boston. They will include Affectiva (CEO Dave Berman), an MIT Media Lab spinout that’s commercializing software to make your computer or smartphone understand your emotional state (talk about a big idea); Krush (CEO Gina Ashe), an ambitious startup focused on social commerce and marketing for apparel and lifestyle brands; and GrabCAD (CEO Hardi Meybaum), a company that’s connecting engineers with people who need stuff built, via an online community and marketplace.

We are really looking forward to 6×6, and we hope to see you there on Dec. 1. You can register at the special super saver rate here.

Comments | Reprints | Share: