With uTest, U Find Software Bugs, U Save

8/20/08Follow @wroush

There’s no such thing as a flawless software application: the only question is how many bugs its developers had time to catch and fix before release, and how many will be discovered by customers. And with software being written today for so many different platforms and operating systems, from servers to desktops to mobile devices to Web browsers, it’s becoming harder and harder for software companies’ in-house testers to try out their creations in every context where businesspeople or consumers are likely to encounter them.

That’s why more software makers are outsourcing all or part of their quality assurance (QA) process to specialized testing firms in places like India and China. And it’s why a new Boston firm called uTest thinks it can make it big by building a global community of thousands of freelance QA testers, available to test software at a moment’s notice in any environment where a program is likely to run.

Need to see whether your new Web service works on the Safari browser for Mac users in Germany? No problem—uTest, which has been serving a couple of dozen pilot customers since April and opened its doors for general business yesterday—has testers there who can try it out. In fact, the company has assembled a network of more than 8,000 testers worldwide—some 10 percent of the world’s entire population of professional software QA testers, according to uTest CEO and co-founder Doron Reuveni.

“QA is a big business,” says Reuveni. “Applications today need to operate in complicated combinations of platforms and networks, and hiring a staff and building an infrastructure for doing software testing internally is not always cost effective. The fact that we have access to such a large community of professional testers has allowed us to take that big business and provide it on demand.”

An “on-demand” approach gets around several of the inefficiencies built into the traditional software testing business, in Reuveni’s view. For one thing, most QA outsourcing deals require a long-term contract. But by doling out bug-finding work through uTests’s software-as-a-service platform, software makers only have to pay testers when they’re actually working toward a new release.

And QA often suffers from its own lack of QA: software companies usually have to pay up on their outsourcing contracts whether or not they’re getting back a lot of high-quality, actionable bug reports. But on uTest’s platform, clients accept or reject each report manually, and are only charged for those they accept. “We like our customers to pay solely for performance,” says Reuveni.

Reuveni and co-founder Roy Solomon started uTest last year to prove that this new, more flexible model could save software companies money and time. They’ve collected $2.2 million in venture funding from the Massachusetts Technology Development Corporation and Connecticut-based Mesco Ltd., a private financial advisory boutique that previously funded Greenfield Online, an online community of paid citizen panelists who provide feedback on consumer products. And they recruited their own community of QA testers by putting the word out through Facebook, LinkedIn, and key blogs frequented by QA professionals.

Testers who put in a couple of hours of work each day can make $250 to $300 a week, Reuveni says. UTest makes money by taking a slice of its clients’ per-bug payments. The percentage kept by uTest—like the per-bug fees themselves—varies according to the number of testers assigned to each software release, the type of application being tested, and other factors.

You might think that the pay-per-bug model would give some uTest clients a way to find out about important problems and then evade paying for the bug reports simply by rejecting them. But during uTest’s pilot period over the last few months, the model has worked well, with a very low percentage of reports being rejected, according to Reuveni. “The testers want to report good quality bugs and have them appproved, so they can get compensated and then get invited back for the next release,” he says. “And the companies have an incentive to approve the good bugs, because software testing is a never-ending process, and they want the good testers coming back for the next release. Nobody is looking at this as a one-off—they look at it as a relationship.”

All 24 of the companies that worked with uTest during its pilot period—including a few Boston-area firms like MocoSpace and Second Rotation—have become ongoing clients and have agreed to serve as references, Reuveni says (see the whole list here). Some of these companies are using uTest as an extension of their QA teams, while for others, uTest testers are their QA team.

Reuveni says the on-demand model works especially well for companies that use the so-called “agile” software development model, which is designed to minimize the risk that big, extended software projects will end in failure by putting applications through an entire development cycle—including planning, requirements analysis, design, coding, and testing—every week or two. “They can test with us over the weekend, and then when they come in on Monday they have a list of bugs from 30 or 40 testers, all prioritized and ready for them to work on,” says Reuveni.

Historically, 25 to 50 percent of the total budget for a software development project goes to QA testing, says Reuveni. “That’s just not an appropriate way to do things today,” Reuveni says. By outsourcing QA to uTest, companies can spend less on the process, and software engineers can focus on what they’re best at—design and coding—while leaving the dirty work to a large community of experts looking to put their software skills to work from the convenience of their homes. With uTest’s help, software companies might just get to a point where they—and not their customers—are the ones finding most bugs.

Wade Roush is a contributing editor at Xconomy. Follow @wroush

By posting a comment, you agree to our terms and conditions.