Adchemy Aims to Overhaul Search Engine Marketing By Killing the Keyword

7/19/11Follow @wroush

Google is broken.

Not in a fundamental way, of course. Given that Google’s search and advertising services are the glue holding the whole Web economy together, you’d know it if something were truly malfunctioning. But Google’s advertising program has a basic flaw, at least according to the executives at Foster City, CA-based Adchemy. It’s a problem that causes huge headaches for search engine marketers, says CEO Murthy Nukala, and one that keeps them from getting the most bang for the mucho bucks they hand over to Google for search advertising (more than $6 billion in the second quarter of 2011 alone).

Nukala traces the problem to a conceptual difficulty in AdWords, the keyword-based system that Google developed in 2000 for selling ad space on search result pages. Adchemy, naturally, thinks it has come up with a fix for that flaw, involving some fancy natural language processing and statistical machine learning software. Now, at right about this moment, you’re probably asking yourself “Do I want to read a whole article about the mechanics of search engine marketing?” It’s true that the only people who would be interested in paying for Adchemy’s technology are advertising managers at companies who win a lot of customers through AdWords ads. But keep in mind that SEM is a $30 billion business—counting the ad spending alone, not the sales the ads generate—and if you read on, I guarantee you’ll learn something about how it works.

First, a mini crash course in AdWords, the program that determines which pay-per-click text ads show up in the right-hand column of a Google search result page. AdWords is keyword-driven, meaning it only shows ads related to words in a user’s search query. It’s also an auction platform; it lets advertisers specify which keywords they think are most relevant to their ad campaign, then bid for a high position on search result pages by specifying how much they’d be willing to for a click. Google determines the actual placement of the ads using a complex formula that takes into account the keywords themselves, the amount of the advertisers’ bids, a quality score based on the ads’ past performance and the relevance of the “landing pages” they link to, and other factors. Because AdWords ads are often closely related to the user’s intent, at least as expressed in his search query, the click-through rates for the ads are pretty respectable, which is why companies advertise on Google in the first place.

Murthy Nukala

The problem is this: the bigger your company, the more products you probably want to advertise on Google, and the more keywords you’ll have to think up to cover all the possible ways people might search for them. Also, the more ads you’ll have to write to make sure they’re customized to the keywords, and the more landing pages you’ll have to tend, to maintain your quality score. For truly large companies—think e-retailers with thousands of products in their catalogs—this problem can get out of hand. If you had to think up a separate set of relevant keywords for every product, you’d end up managing millions of keywords on AdWords—and that’s a task that not even the largest Excel spreadsheet can help you with. “We know of large retailers who sell three million products who are only advertising for 10,000 of them,” says Nukala. “The reason is it’s too hard and too expensive to figure out all the keywords.”

The problem, Nukala argues, is with the keyword itself. He says it’s time to move beyond keyword-based ad auctions to and give advertisers a way to target potential customers based directly on their intent, not the specific wording of their search query. But since it doesn’t appear that Google is about to rebuild AdWords from the ground up, Adchemy is offering a workaround. “We want to break the current model and change the keyword as the unit of sale,” Nukala says.

In a nutshell, Adchemy helps search engine marketers generate large numbers of appropriate keywords automatically, using “intent map” technology that it’s been developing since 2004. Nukala hatched the original idea with co-founder Rajeev Motwani, the Stanford professor who advised Sergey Brin and Larry Page and co-authored the PageRank paper that catapulted Google to fame and fortune. (Motwani perished in 2009 in a drowning accident.) But for a long time, the technology was a solution in search of a problem: the company’s first idea, after spending about three years in stealth mode, was to see if it could help companies find the right audience segments for display ads and other forms of brand marketing. It called the practice “audience relationship management,” but it never really took off. More recently, Adchemy has focused on using intent maps to make SEM more effective.

The fundamental insight behind intent maps is this: while people are endlessly inventive when it comes to phrasing search queries, their questions can be boiled down to a much smaller number of basic intents. So what was needed to simplify the SEM problem was a way to save advertisers from having to divine millions of keywords and write the copy for millions of separate ads; a way, in Nukala’s words, to “productize search.”

And as the former senior vice president of enterprise products at Shopping.com, Nukala knows something about productization. “Back in the days of the first comparison shopping sites, you would crawl five sites and bring back five deals,” he says. “The hard problem was knowing that all of the deals were about one product.” That problem is called productization—and intent maps are just a way of productizing search queries. “The keyword space is unbounded and therefore mind-numbing in its complexity,” says Nukala. “Intent is finite and bounded, which is what makes it manageable.”

Adchemy’s patented WordMap software—its name for intent maps as applied to SEM—is essentially a new front end for AdWords, or, put another way, a giant filter that identifies intents and then translates them back into something AdWords can work with (that is, keywords). It works like this: A new Adchemy client feeds its entire catalog into the system, along with its website, its query logs (the record of searches people have used to find its pages in the past), its past AdWords keywords—essentially, all the data it has relating to its products. Adchemy’s natural language processing software then goes to work, picking out categories and themes, identifying the relationships between them, and generally reducing the mess to something manageable.

“We’ve seen there is approximately a 300- to 1000-fold reduction in the number of artifacts you have to manage when you go from keywords to intent,” says Nukala. “Three million keywords goes down to 3,000 intents.”

WordMap can then go on to automatically generate thousands or millions of fresh keywords specifically related to the intents. It can also create the copy for thousands of distinct ads by dropping the new keywords into templates written by the user, sort those ads into groups according to intent, and assign the appropriate keywords to each ad group. In the end, the client has a giant set of optimized ads and keywords ready to be dumped into its bid management system, which in turn dumps them into AdWords. Oh, I almost forgot to mention that WordMap can also generate landing pages corresponding to all the new ads. So you still have lots of keywords and landing pages to manage, but at least Adchemy is generating and managing them for you, turning SEM back into human-scale problem.

Adchemy, which has collected about $58 million in venture funding from August Capital, Mayfield Fund, Accenture, and a variety of individual investors, has two ways of profiting from its intent map technology. One is called Adchemy Actions; it’s a “performance marketing” business in which the company uses its own system to create and place ads designed to generate sales leads for its clients, which include auto insurance companies, online universities, and mortgage lenders.

Nukala says Adchemy Actions gives the company a way to “eat our own dog food and make sure the software works before we sell it.” More recently, Adchemy has opened up the intent platform itself to outside companies as a Web-based service. Nukala says the target customers for this product are the largest 4,000 e-commerce companies, including retailers, insurers, and financial services companies—any company with a complex set of products to advertise.

Does Google know or care about the keyword problem that created an opening for Adchemy? In fact, it does: to spare advertisers from having to anticipate every keyword a search user might use to describe a concept, it offers a partial solution called “broad match.” When AdWords customers activate broad match, the AdWords system will automatically run ads for keywords that are variants of an advertisers’ original keywords—a seller of tulips, for example, might also see his ad show up on search result pages for the keyword “flowers.”

But as you might expect, Nukala isn’t too excited about broad match. “We think broad match is a bad thing,” he says; it’s a “symptom of a problem,” meaning the difficulty of manually composing effective keywords. In the long run, he says, even Google will be forced to switch over from keyword-based ad auctions to intent-based advertising—mainly because people keep coming up with new ways to ask for things.

“In any given month, 20 percent of search queries are unique queries that have never been seen in the history of search,” says Nukala. “If consumers are constantly changing how they express keywords, how many would you have to bid on so that you would never miss a keyword? The answer is it’s infinite—which is a silly model. We think it’s time for a change.”

Wade Roush is a contributing editor at Xconomy. Follow @wroush

By posting a comment, you agree to our terms and conditions.