Microsoft Research’s Jennifer Chayes: 5 Projects for the Future of Computing
When people talk about the future of technology, they tend to mention the same big players (what I might call the four horsemen of the consumer apocalypse): Apple, Amazon, Facebook, and Google. And if they want to get more cutting-edge, maybe they’ll throw in a viral newcomer or two like Dropbox or Pinterest. Who they don’t tend to mention these days is Microsoft (the Lumia 900 notwithstanding).
And that’s just fine with the venerable old software firm (NASDAQ: MSFT). Microsoft seems content to chip away at consumer markets while still making most of its money with business-related products—and continuing to invest boatloads of cash into basic research, which is quite unlike other tech companies of its time. Last month, Microsoft ran its annual TechFest R&D show in Redmond, WA, where the usual array of dazzling demos were on display: image and video processing tricks, machine translation, search data visualization, and human-computer interfaces. But what impact will these kinds of projects have on Microsoft’s business—and on the future of computing?
These thoughts were in my head as I sat down with Jennifer Chayes, the managing director of Microsoft Research New England. Every techie within a five-mile radius of Kendall Square knows about Microsoft NERD (New England Research and Development Center) from all the events it hosts. But few people seem to know about all the research going on there. That’s starting to change.
The last time I spoke with Chayes in depth was July, when Microsoft Research New England turned three years old. Chayes told me about the first Microsoft product to come out of the lab—a piece of software called Readmissions Manager that is part of Amalga, a healthcare package used by hospitals. She also talked about some of the lab’s research in empirical economics, computational biology, and social networks.
A couple weeks ago, she fleshed out some of the newer projects emerging from the center, as well as giving me a broader update on the state of the lab, which has grown to 15 full-time researchers, 13 postdocs, half a dozen research assistants and software developers—and far too many students, interns, and visiting researchers to count. “We hire slowly, but we are hiring,” Chayes says.
On how Microsoft will impact the future of technology, Chayes points to a few key themes. “I think the defining questions of this era of science and technology are going to be the ‘big data’ questions. And a natural user interface for how to interpret this data and act on it,” she says. “The themes of big data and natural user interface run all through our products. They’re under the hood of all of our products in a way that makes them be able to personalize to you, to figure out what you actually want and how to get it to you in a better way.”
So, I wondered, how does she see computing and information flow in daily life continuing to evolve? In other words, what’s after search engines and social networks, say, five years from now?
“There is going to become a seamless personalization that will take in the social, and also take in what we do and what other human beings do, and it is going to understand what we want and how to present it. And each person’s experience with their devices is going to be unique to that person. We don’t have that now. But I think five to 10 years from now, with machine learning, we will have that,” Chayes says. Computers “will understand where to pull in these various threads” so the right information is delivered to us and “we really don’t have to think about it.”
As for Microsoft’s not getting as much media love as other tech darlings, Chayes downplays the competition and focuses in-house. “We have been getting a lot of attention around Xbox and Kinect,” she says. “That’s our new cool thing, our new shiny toy, which is a lot more than a toy. I think it will be in every aspect of our enterprise [business] as well as our consumer and the living room and all that. But, beyond that, machine learning is becoming such a big part of what we do.”
And with that, let’s take a look at five projects from Microsoft Research New England that exemplify what Chayes is talking about—and could lead to some interesting new products (and possibly help shape the future of computing):
1. Machine learning for the cloud. This is a project led by postdoc Ohad Shamir together with researchers in Redmond. The basic idea is to use machine learning algorithms to help startups and other organizations bid properly for cloud-computing resources. “They can either do spot pricing or they can be buying upfront at higher prices—how do they optimize that?” Chayes says. “For us, on the cloud provider side [with Microsoft Azure], it would give us an analysis of what the flow was of the kinds of requests coming in, so that we could time things better and use our energy resources better. There will be opportunities for data markets in the cloud that will be absolutely huge.”
This may not sound very sexy, but neither did Amazon Web Services back in 2006. “If you make it cheaper and easier for a startup to get the cloud resources it needs, that’s not your shiny object, but to a startup business, it’s everything,” Chayes says.
2. Machine learning for people and tasks. This one is more along the lines of what people have been talking about as “machine learning” for the past 20 years. Actually it’s two projects. The first has to do with categorizing which images are similar to one another—something people are great at, but machines stink at. Lab members Adam Kalai, Ce Liu, Ohad Shamir, and their collaborators used crowdsourcing through Amazon’s Mechanical Turk to teach a machine how to decide whether image A—a floor tile, national flag, or human face, say—is more similar to image B or image C. The science has to do with understanding how humans perceive similarities, and incorporating those judgments into a machine. The applications could include e-retailers displaying things like home furnishings or apparel in a way that lets you drill down to styles you like by clicking on images, rather than sorting items just by their color or other blunt tags.
The second project has to do with “programming by example.” Led by Kalai, Microsoft technical fellow Butler Lampson, and senior researcher Sumit Gulwani, this one involves a computer learning a sequence of steps by watching how a person goes through a repetitive task such as formatting bibliographic references or sorting addresses for invitations or greeting cards. The idea is the computer can learn to do the formatting automatically. (You might imagine something like this being incorporated into a future version of Office or Excel.)
“Butler said he thought about doing this 15 or 20 years ago, and he convinced himself it was theoretically impossible,” Chayes says. “They found a language which is rich enough to encompass what most people are doing and yet small enough that it’s not this outrageous exponential search. It’s technically a phenomenally difficult problem.”
3. How information flows in social networks and offline. This is a major area of research for the lab. Senior researcher Markus Möbius, an empirical economist, is doing a study in which his team tracks the flow of digital coupons tweeted out by mobile food carts. The food carts tweet their locations and menus along with a discount code, and the researchers analyze which coupons get used. “They can see where one coupon code spreads out beyond just who it got tweeted to,” Chayes says. “Think of Groupon. There are lots of online businesses based on social media. How word of mouth is supplementing the effects of social media is a big business question.”
A somewhat related project in the lab is led by postdoc Andrés Monroy-Hernández (who’s moving to Redmond as a staff researcher). He and his collaborators are trying to understand the social and behavioral principles behind how news spreads through alternative channels such as Twitter, by analyzing micro-blogging reports from the deadly Mexican drug war.
4. Social effects in health and wellness. Another study led by Möbius is tracking the impact of incentivizing kids to eat more healthy foods such as fruits and vegetables in school cafeterias. The study compares how kids behave when they take part in the program by themselves, when they’re paired with a friend, and when they’re paired with a random partner. Healthcare is an important application area of the “economics of information flow and how information sticks in social networks,” says Chayes. “That’s a really big area here.” (As Microsoft gets more active in healthcare, we’ll see if there are new product areas for promoting health and wellness online.)
5. Video de-blurring in real time. And last but not least, researcher Ce Liu, a former intern and associate researcher from Microsoft’s Beijing research lab, working with MIT professor Bill Freeman, has developed software for taking live video that’s out of focus and de-blurring it, using machine learning and other techniques. This is a longstanding problem in computer vision and image processing. “It’s actually stunning if you look at it. He’s been talking to a lot of product groups now,” Chayes says. “You can imagine you might want that in Skype, or you might want that on your phone.”
Liu is an example of the New England lab’s research going where its talent lies. Ultimately, this approach could be key in helping it push the frontiers of computing. “We weren’t even going to do vision research,” Chayes says. “We made [Liu] a postdoc, and he was doing so much, we made him a permanent researcher.”