The Future of Microsoft Research: One on One With New Boss Peter Lee

How do you help invigorate the Microsoft machine? Well, for Peter Lee, the answer lies in the frontiers of computer science. Lee is the new managing director of Microsoft Research, the company’s global research arm. His job is to help his parent company, not always quick on the uptake with emerging technology, understand the latest advances in computer science and better integrate them into both new and existing products—in other words, to put some fresh life in Microsoft’s innovation step.

Microsoft itself represents a bit of a conundrum. In July, CEO Steve Ballmer engineered a dramatic reorganization that included shedding the long-standing business division structure (Xbox, Windows, etc.) for one based on function (Engineering, Advanced Strategy and Research, Finance, etc.). The move was designed to help Microsoft transition away from its traditional pillar of desktop computing and innovate faster in the new world of tablets, mobile phones, cloud computing, storage, and more (what Microsoft calls its devices and services strategy).

A big problem for Microsoft is that it has rarely seemed to lead in emerging arenas, and then when it finally catches on to the new thing, it fails to capture the public with its own products and services—be it a tablet, a phone, or a search engine. And yet, and yet… last month the company surprised analysts with much better-than-expected first quarter results. So it must be doing something right in its transition—just not the something that brands it as an innovation leader.

That, in a big sense, is where Microsoft Research and Peter Lee figure in. Like its parent, MSR is not without its doubters. With seven main research labs and six satellite arms that employ some 1,100 scientists and engineers, it has long been hailed as a haven for computer science research. But it has been taken to task by many over the years for allegedly not producing much for Microsoft’s bottom line.

Peter Lee, the new managing director of Microsoft Research

Peter Lee, the new managing director of Microsoft Research

Lee stands firmly behind MSR’s contributions: like other Microsoft leaders, including Ballmer and Bill Gates, he asserts that the organization has contributed to virtually every product and service Microsoft has—and I know from repeated visits over the years, as well as this document that led to the founding of MSR, this has clearly been a focus from day one. At the same time, it is clear Lee feels he has to make the connections to the business side stronger and perhaps more visible.

The former head of the computer science department at Carnegie Mellon, and also a former office director for DARPA, Lee came to Microsoft Research in mid-2010 to run its flagship lab in Redmond, WA. He took the reins of MSR this past July, when his predecessor Rick Rashid moved to a new role as part of the reorganization. Rashid had been at the helm of Microsoft Research ever since its 1991 founding. So Lee’s challenge is to at once preserve a culture of leading-edge research and stability provided by a visionary leader, while increasing ties to the business and the payoff for Microsoft itself. It is a tall order.

I spoke with Lee on the phone in September and then sat down with him for an hour when he was in Cambridge, MA, last month for the fifth anniversary of Microsoft’s New England research lab.

We covered a lot of ground, and I won’t try to capture everything. But below are some core issues, ideas, and comments—ranging from the pithy to the profound—that stood out for me. I think they are important to understanding MSR’s evolving role within Microsoft—as well as its future direction in terms of where computer science is headed and, more broadly, what that could mean for consumers. So read on for more about MSR’s enhanced focus on helping the company, a new openness from the labs in terms of sharing its work, and Lee’s thoughts on emerging fields like deep neural nets, telepresence, and more.

Of Microsoft Research and Artwork (The Big Picture, so to speak)

People don’t generally appreciate that MSR is only about 1 percent of Microsoft’s overall budget, Lee says. That’s “roughly on par with the cost of the art work and the lawn upkeep of the world organization,” he says. “MSR is an incredible bargain.”

The Challenge: Change Is in the Air

“First and foremost, the thing that I find remarkable about MSR is that it’s a place more than any other place that I know about, including universities, that really believes in research—really believes that if you bring enough really smart people, get them fully resourced, and then just stay the heck out of the way, then great things will happen,” Lee says. “That kind of fundamental belief in that concept is just so strong and pervasive here that it is really remarkable.”

”Having said that, there are lots and lots of changes in the air,” he says. Some are internal, given the company’s reorganization. Others are external. The IT industry, Lee says, “is going through many, many changes and is ripe for disruptions.” At the same time, basic research in computer science is peaking “beyond anything that I have seen in my career. There’s so many amazing things coming out. The convergence of all of the changes in the IT industry with this surge of optimism and excitement in the basic research community is something that I think will really define the next decade.”

The bottom line for his organization, Lee says, is that the rest of Microsoft “more than ever is really depending on research to come through.”

A New Balance

So how does Lee intend for Microsoft Research to pull this off? The idea has always been to leverage MSR’s relatively small position in a huge organization by looking out farther into the future than other parts of the company can afford to do, preparing executives for what is coming down the pike, and blazing new trails in computer science that might be beneficial to Microsoft. That includes taking some “wacky” (Lee’s word) chances that aren’t possible elsewhere in the company but may lead to disruptive new products or even new business lines.

But at the same time, Lee says, MSR must also feed innovations more directly into product groups, helping the company’s existing and soon-to-be-released products and services keep up with, and hopefully ahead of, what competitors are doing.

Maintaining the balance between helping with the here and now, while also taking chances on things farther down the road, has always been the challenge for Microsoft Research (and any other corporate research organization). But now that balance is changing. Lee didn’t quite say it outright, but that almost assuredly means shifting more to the shorter-term and away from riskier, long-term efforts. “In the new Microsoft, MSR is being called on to lead technical developments much more,” is one way he put it.

This, by the way, is not unique to MSR. Research arms at IBM, Xerox, HP, you name it, have all shifted this balance in recent years, and continue to do so. Nor is it the first time Microsoft Research has shifted. My sense, though, is that MSR is undergoing a more profound rebalancing than in the past. The next three sections provide more on Lee’s approach to striking this new alignment:

The Quadrants of Research 

Lee describes research as consisting of four basic elements that he graphs out as quadrants on an X-Y scale. The X axis represents time, from short-term research on the left to long-term on the far right. The Y axis is based on how directed research is. It runs from what Lee calls reactive problems, meaning they are directly in reaction to business needs or problems, all the way up to open-ended. The upper right quadrant, therefore, represents blue sky research, less tied to any particular business need, less directed, and with the longest time horizons. The lower left is the most close in and closely coupled to business needs. Lee calls this “mission-focused” research. On the upper left is disruptive research. This is shorter-term than blue sky efforts but can be just as open-ended: it is potentially disruptive and therefore different than mission-focused work on current products. The lower right, with a longer time horizon than the mission-focused or disruptive quadrants, is called sustaining research. This typically means continuous, iterative improvement in what has already been done, taking things to the next level, so to speak.

That is my summary, anyway. Here is Lee’s own chart, and you can find a description of these quadrants in his own words.


Some key points Lee made about this matrix in our interviews:

—Every manager inside MSR must have a strategy for all four quadrants, and “give me something to brag about in each quadrant.”

—No individual researcher can work only in one quadrant. This guards against researchers being too narrowly focused, and therefore missing key opportunities, like, say, when longer-term work might provide a breakthrough or key ingredient for shorter-term work—and vice versa. Plus, Lee says, “Being too mission-focused ends up limiting people’s imagination.”

—“My assessment is that without any direction from senior management, MSR has put more resources into the bottom left quadrant, the mission-focused quadrant.”

—“From my perspective it’s not a bad thing for MSR to be very important in mission-focused activities for the company.” However, “it would be wrong if MSR were to go 100 percent mission-focused. In fact, if that were to happen, there would be no reason to have MSR at all.”

Putting those last two bullet points together, I concluded that there is definitely a directed shift to more mission-focused research. Now what are some of the ideas that have Lee most excited?

Deep Neural Nets

First on his list was the field of deep neural nets, which can be thought of as a kind of unifying theory of artificial intelligence—in Lee’s words, “to get computers to see and hear and reason like humans, or even better.”

Lee says he first encountered the term back in the late 1980s. But there were lots of early problems and the technology didn’t work as hoped. About six or seven years ago, the field picked up again, and it has exploded in more recent times, he says. Beginning in 2009, one of the original pioneers of the field, Geoffrey Hinton of the University of Toronto (he now works part-time for Google), collaborated with MSR to apply deep neural nets, first and foremost, to speech recognition. Lee says MSR researchers advanced this work and in 2011 showed it dramatically improved the potential of real-time, speaker-independent, automatic speech recognition: rather than having one word in 4 or 5 incorrect, the error rate is now one word in 7 or 8. “Within two years, with changes in big data and cloud computing, plus steady algorithmic improvements, this stuff actually works and works way better than anything we have seen before,” Lee says. Microsoft has since advanced it as the core of its speech processing, in products from Xbox to Bing to Windows Phone.

What’s more, the change goes far beyond Microsoft, Lee says. Nuance and Google have altered their speech product direction to be based on deep neural nets, he says. “The entire industry just changed on the dime through this.”


One of the biggest MSR efforts, Lee says, is a broad investigation into the technologies and algorithms around telepresence. There are lots of problems to address, such as how to convey things like body language over long distances. But it is important for researchers to step back and ask why they should be doing this, Lee says. If they are too mission-focused, the answer they come up with will be things like enabling better business meetings and saving travel costs. Those may well be commercially viable but not that imaginative—and maybe not that interesting over the long haul, he says. So Lee wants researchers to ask questions such as, “Would it be possible for musicians to collaborate through time and space?’ and “What would we learn by enabling children to really engage at a distance?”

“That kind of playfulness in what we’re trying to accomplish” is more likely to lead to surprising things that might pay off in much broader or bigger ways, he says.

Computational Social Sciences

Lee believes that the social sciences can be utterly transformed by computing in much the same way that the physical sciences and the life sciences already have been—enabling game-changing advances in everything from analyzing and predicting human behavior to forecasting the behavior of companies and even society at large. Lee sees some big commercial applications of all this. “Because unlike understanding the cosmos and the big bang, which is important,” he says, “really deeply understanding how society works and from that inferring lots of things about people can have really profound consequences for economics, public policy, psychology, and so on in the social sciences, but also for how Microsoft and other companies do their business.”

One specific way this might bear fruit is what he calls a new twist on machine learning. Typically machine learning is about finding statistically relevant correlations between data. “It gives you correlation, not causation,” he says. But with advanced techniques and big data sets, you can begin to find causes.

Ubiquitous Computing

“The technologies for sensing and embedding sensors not just on our bodies but everything around us, I think, will be pretty significant,” Lee says. This has long been a kind of holy prediction in computer sciences, but he says, “It actually looks like it finally will become real.”

A New Openness?

When I met a few years ago with Craig Mundie (at the time, Microsoft’s chief research and strategy officer and now senior advisor to the CEO), he had clearly asked MSR to be less open about its research, worried that others were developing business intelligence or advantage from what MSR sometimes shared about its work.

Lee seems to have a different view. “The importance of openness is really underrated,” he says. “If there’s one thing I worry about, not just for MSR but for the IT industry, it’s that secrecy seems to have become cool. I think that this is something that is a dangerous trend really for the whole industry.”

He says secrecy has even “crept into academic research.” In the field of deep neural nets, for instance, “I’m aware of eight or nine academic research groups that are just very hush-hush about what they are planning to show there.” Such secrecy, he says, is at the level where be believes it can be “a drag on the flow of ideas.”

“For MSR, one of the things that we’ll be working very hard to do and try to show some leadership in is not to just maintain our openness but grow it.”

An example of this is some recent work on Kinect, which employs gesture and speech interfaces, among other technologies being studied at MSR. Researchers don’t reveal specifically how they are applying their work to future products or what future applications of their ideas are envisioned. However, rather than clamming up on it all, “We have been publishing all the foundational work,” Lee says. “It’s a partnership. We depend a lot on the advances in the academic research community, and we need to participate as openly as possible.”

Closing Perspective: Watch for Some Big Payoffs from MSR 

Lee has studied the history of industrial research and he spoke somewhat passionately about the long time it sometimes takes researchers to deliver seminal discoveries or inventions.

“I think MSR has the people now to do something really remarkable,” Lee says. “Can it do something as monumental as IBM Research did in its heyday, or Bell Labs? I do think that all the opportunities are there.”

Bob is Xconomy's founder and editor in chief. You can e-mail him at, call him at 617.500.5926. Follow @bbuderi

Trending on Xconomy

By posting a comment, you agree to our terms and conditions.

  • ja

    This guys will fail.