Yale Spinoff Hadapt Hops Out of Stealth, Looks to Help Big Companies Handle Big Data

3/23/11Follow @gthuang

If you don’t know Hadoop from a hole in the wall, please meet Justin Borgman. He’ll break it all down for you.

Borgman is the co-founder and CEO of Hadapt, a New Haven, CT-based startup that is coming out of stealth today at GigaOm’s Structure Big Data conference in New York City. Hadapt calls its product “the adaptive analytical platform for big data,” but trust me, it isn’t that boring.

The company is trying to merge the best features of advanced databases with a software platform called Hadoop to enable business customers to analyze enormous amounts of data, both structured (e.g., spreadsheets) and unstructured (e.g., free text), in a super-fast and efficient way.

OK, first things first. Hadoop is an open-source version of a software platform originally built by Google to store and analyze data generated from search indexes. But Hadoop is primarily used for unstructured data, and mostly by technical geeks. So, what Hadapt is trying to do is broaden the market for Hadoop by opening it up to all types of data and queries, by linking it to what’s called a relational database (which handles structured data).

“We’ve built a true hybrid,” says Borgman, a Yale MBA with Boston-area engineering roots. “You can have both structured and unstructured data in one platform, and do analytics across both.”

If you’re wondering how it’s different from the dozens of other companies trying to harness Hadoop, or otherwise help companies crunch huge amounts of data faster, well, it gets a bit technical. For starters, take Cloudera, a heavily-funded Silicon Valley startup that is also trying to bring Hadoop to the masses. Borgman says Hadapt is actually complementary to Cloudera: Cloudera focuses on tools and services to help ordinary IT staffs run Hadoop, he says, whereas Hadapt is more fundamentally “trying to make Hadoop better.”

But then it gets more directly competitive. Other systems like Hive, an open-source platform built on top of Hadoop, similarly lets users query big data, but because Hive was driven by Facebook, it isn’t suited for business intelligence applications, Borgman says. For certain types of queries, Hadapt is about 50 times faster, he says. Another system that works with Hadoop is called HBase, and Borgman claims his company’s technology is 600 times faster than that. (The details of why involve shifting workloads to faster computing nodes, and the fact that the data is replicated three times in the system.)

Hadapt is based on Yale professor (and Hadapt co-founder) Daniel Abadi’s research on merging Hadoop with databases. Yale PhD student Kamil Bajda-Pawlikowski is also a co-founder. The company, which officially started last July, has filed three patents around the technology and has seven full-time employees. Hadapt got its start through a summer program at the Yale Entrepreneurial Institute and has since moved to CTech, an incubator sponsored in part by Yale, LaunchCapital, and Connecticut Innovations. The company has raised an initial financing round (primarily angel capital), but Borgman declined to give any more specifics.

The startup has a long way to go, of course—it’s still refining its revenue model—but one intriguing future has it competing with Netezza (now owned by IBM), EMC, Oracle, HP, and other data warehouse combatants of the business-intelligence world. “We won’t replace them today, but that’s the five, 10-year plan,” Borgman says.

Gregory T. Huang is Xconomy's Deputy Editor, National IT Editor, and the Editor of Xconomy Boston. You can e-mail him at gthuang@xconomy.com or call him at 617-252-7323. Follow @gthuang

By posting a comment, you agree to our terms and conditions.

  • Andrew Koyfman

    Greg,

    I don’t think you can say that Hadoop is built by Google. It is actually Yahoo who has been the biggest (and I think earliest) contributor to the project.

  • http://www.xconomy.com/author/ghuang/ Gregory T. Huang

    Andrew,
    I didn’t mean to imply Hadoop was built by Google. I meant that it is an open-source implementation of a platform (MapReduce, etc) originally built by Google. Though there is some question of whether it was just “inspired” by MapReduce.

    Greg

  • http://bit.ly/gM1FrM Justin

    Analytics in Financial Services: TDWI + NY Tech Council Present First of a New Series on Practical Industry Analysis http://bit.ly/gM1FrM

    Thanks for help spreading the word!

  • Pingback: Big-Data Startup Hadapt Backed by $6.75M from Atlas | Xconomy