The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation

12/21/09Follow @gthuang

When it comes to product and business strategy, Google is one of the most secretive companies around. So I was pleased when the Internet giant agreed to grant me a series of interviews last week, most of them with senior product managers from its Mountain View, CA, headquarters.

I touched on elements of Google’s technology and engineering strategy in a Q&A I did last week with Google senior vice president Alan Eustace. Here I’m drilling down into topics that should be of great interest to readers in all tech sectors and geographies—including those who follow key Google [NASDAQ: GOOG) competitors like Microsoft, Amazon, Apple, EMC, and Akamai, not to mention the legion of startups around the world who need to account for the giant’s every step in a particular area.

In four separate interviews, Google delved into some of the most important topics of the day, from its advances in real-time and local search to cloud computing and a “data liberation” effort to help consumers export their files and digital information from Google products (should they want to do so).

And here’s the real secret: Google managers are pretty boring. And they’d like to keep it that way, thank you very much (at least when flanked by their PR folks). They don’t talk about competitors. They don’t make predictions about the industry. And they don’t say exactly who works on what where, within Google. They just focus on making their own products better and more user-friendly. Shouldn’t that be enough?

Perhaps Jack Menzel, a senior product manager, said it best when he joked about the “thankless job” of working on Web search at Google: “You demo [a new search feature] for people, and they say, ‘Yeah, so it works. So what?” (How quickly we all forget what it was like to search for information online just a few years ago.)

So without further ado, here are my highlights from various corners of the Google machine:

Real-time search. This is the increasing effort to update search results to include new information on the Web—including data from social media like Twitter and Facebook—almost as fast as it’s published.

Menzel, an ex-Microsoftie who studied computer science at the University of Washington, leads a product group on this front. He says Google has been working on indexing and ranking content more frequently for years—first, it was once a month, then it became once a day to keep up with news and blogs, and then in the past year as Twitter grew popular, updates on the order of minutes and now seconds have become crucial. “We’ve been looking at getting faster and faster for quite a long time,” Menzel says. “We already had a trajectory to rank content faster and faster. Every step in that process had lot of challenges. Getting fresh information has been one of the tenets of what we believe makes Google successful.” (The other tenets of search would be relevance, speed, and comprehensiveness of content.)

Menzel says the biggest challenge is not just the speed—it’s the relevance to the user. “It’s very, very challenging to take short-form content and rank it against a New York Times article or a blog post,” and show the results “only when you want it.” Especially when that content didn’t exist just a few seconds ago, and nobody’s linking to it yet, so Google can’t use PageRank, its classic technology.

Instead, they “heavily rely on what we’ve learned in the past 10 years,” Menzel says. That includes things like how to parse out content that is likely to be irrelevant or spam, in a more general way. And coming up with “brand new signals”—he mentions “new language models” to understand which updates are relevant and which might be some oceanographic scientist’s data beacon, for example, as well as factoring in “how reputable the author is.”

As for the future, Menzel echoed what everyone seems to be saying about search these days: it’s early. “We’ve really just started in on this problem. We still have a long way to go,” he says. Within five years, he says, he hopes Google will make search “much more personal than it is today.” That means more than knowing that you like to follow soccer, for instance, and that you call it “football,” he says—it means understanding who you’re connected to, where you are, and organizing all the information around you.

“Search is still a very unsolved problem,” he says. “There are still a lot of things that are very hard to find on the Internet.”

Local search. This is all about Web search queries that include geographic information—such as “Hong Kong hotels” or “Seattle restaurants”—or are done from mobile devices to find nearby locations, products, or establishments.

Carter Maslan, director of product management for local search at Google, calls it “organizing the world’s information geographically,” or creating a fast, simple guide to the “geo-Web.” The main challenge, he says, is “mapping all these different ways of expressing a query onto a very large corpus of local information. And returning the right answer as quickly as possible.”

Maslan, another former Microsoftie, says Google draws on a vast number of search queries to analyze the way people search for information locally and the way things are referenced on the Web. The ultimate goal, he says, is that “ideally it becomes effortless to find and discover places around you.” Some familiar examples would be searching for directions on your mobile phone after landing at an airport, or being out and about in a neighborhood of New York City, looking for bars.

Which sounds like it should figure heavily into Google’s broader mobile strategy. “Your phone knows a lot,” Maslan says. “It knows where you are, it can determine which direction you’re heading. It’s not based solely on keywords in a search box. We want to bring [that mobile information] to the forefront.” A current example of this is Google Goggles, which lets you take a picture of a logo, landmark, or location, and get information about it instantly.

Maslan says Google’s key differentiator in local search is comprehensiveness—being “open to all sources,” which he says is quite difficult technically. It includes being “actively global” and not just indexing information from key metro areas, for instance. “The scale that Google has in mapping assets and geo-coding assets and Web understanding are the core things,” he says.

I took this to mean we’ll soon see things like Google maps and listings in many more locations and languages around the world, and better accuracy in keeping up with local businesses opening, closing, or moving. “We’re very clear on where we’re doing well,” Maslan says. “We have small teams fanatical about doing it right.”

Cloud-based applications. This is about using software that’s running on remote servers, rather than sitting on your desktop, for everyday tasks like e-mail, scheduling, and managing documents. It’s part of Google’s broader strategy in cloud computing—the buzz phrase that essentially means consumers, businesses, and organizations renting computer power and data storage over the Internet because it’s cheaper and more efficient for certain applications.

Ken Norton, a Google senior product manager (and Boston University alum and former entrepreneur), talked with me about Google Apps and the company’s cloud computing strategy. Norton’s team works specifically on Google Calendar, but Google Apps also includes Gmail, Google Talk, Google Docs, and Google Sites. “The Web has won in terms of how applications will be consumed,” he says.

The key advantage for Google on this front is scale and infrastructure. “We have so many servers and data centers around the world, we can run them cheaply and efficiently,” Norton says. And that kind of advantage filters down to individual devices, he says, because it “opens up possibilities” for consumers to use Web software from any type of device, be it a smartphone, netbook, or conventional laptop.

Google’s cloud computing efforts are focused on two levels: the first is software like Google Apps that is marketed directly to end users (corporations and consumers); the second is App Engine, a cloud platform for software developers to build their Web-based products efficiently. (You can read more about App Engine in this Xconomy feature from last spring.)

I asked Norton what to expect on the cloud front next year. “We’re constantly doing improvements,” he says. In 2009, there were more than 100 major feature launches across Google Apps—things like video chat and Gmail offline. He says Google will “continue to emphasize the communication offerings.” Besides advancing Gmail and Calendar, that includes polishing up Google Docs and rounding out its suite of features. Norton says Google is also looking to expand its collaborative software offerings, including products for large businesses in the area of interoperability with security systems for authentication.

Bottom line: it sounds like Google is making the transition from focusing on free consumer cloud products to putting more emphasis on paid cloud offerings for enterprises.

Data liberation. This is a growing effort within the company to allow consumers to more easily export their data from Google products like Blogger, Google Maps, Google Docs, Chrome, and App Engine (developers’ user data). It might sound like PR-friendly stuff, but there’s a deeper and more interesting innovation strategy here.

Brian Fitzpatrick, an open-source software vet, heads up the two-year-old effort from Google’s Chicago offices. The basic idea is to help users get their files and other data out of Google’s cloud so they can switch them to other systems if they want to. “Most people don’t think about data liberation until it’s too late,” Fitzpatrick says. “We hope if you leave us for one product today, you’ll try another product tomorrow.”

Beyond “doing the right thing for users,” he says, there’s another motivation. “We as a company work hard on things like search. If users are locked into your products, you get more complacent. If it’s easier for someone to leave, you’ll be motivated to bust your butt, and make your products better.”

So there you have it. Google thinks this sort of openness about customers’ data will make the company work harder to keep them. Fitzpatrick says he doesn’t know of other companies that have a published initiative to do this.

He says his biggest challenge now isn’t the engineering, it’s raising awareness. “It’s hard to get people to think about why this is important,” he says. But it fits into the notion of how consumers and businesses will take care of all their data as more of it migrates to the cloud—and how Google wants to be in charge of organizing the world’s information, every step of the way.

Gregory T. Huang is Xconomy's Deputy Editor, National IT Editor, and the Editor of Xconomy Boston. You can e-mail him at gthuang@xconomy.com or call him at 617-252-7323. Follow @gthuang

By posting a comment, you agree to our terms and conditions.