Engaged Learning, Backed by Gates Foundation, Adapts to Each Student
When University of Washington computer science professor Zoran Popović describes his effort to dramatically change the way kids learn math, you can pick up echoes of personalized medicine, targeted online advertisements, and even choose-your-own adventure books.
As in virtually every other sphere of business and life, the technology-driven transformation of education from a data-poor to a data-rich field is creating the potential for innovation. And data is at the heart of what Engaged Learning, a nonprofit founded by Popović and backed by the Bill & Melinda Gates Foundation, is trying to do.
The Engaged Learning system is designed to monitor a student’s answer to each new question, and adjust the next question in real-time—though, more realistically, on a daily basis—to address that student’s specific challenges, level of engagement, and readiness to advance toward mastery of a subject. Each new data point gathered on a student informs what the system does next.
This automated, adaptive curriculum should, in Popović’s words, understand “exactly what the kid is thinking about, and exactly the kind of new light bulb that’s about to illuminate in their heads.”
The 11-person Seattle organization is building what it describes as a platform capable of turning an ordinary curriculum into one that adapts to each individual student’s pace of learning, automatically measures and propagates the best teachers’ best practices across K-12 classrooms and schools, and generates reams of new content to ensure that struggling students can get the amount of practice they need to master a subject.
That might sound like a big, audacious promise, but Engaged Learning has years of research behind it and says it is gaining traction within its target market. The nonprofit is planning 10-week trial implementations with Washington schools this fall and is in discussions with seven large educational publishers, which would use its software to add capabilities to their curricula, Popović says. Assessment companies are interested in the technology, as well. “They need a way to automatically generate a huge amount of content that they curate towards assessments. With the emergence of Common Core standards, this becomes more important,” he says.
The company was created a little over a year ago to translate to a broader K-12 curriculum the novel approach taken in a highly successful algebraic math game developed by the UW Center for Game Science, which Popović directs. By adapting to the way each individual player is learning, the game helped more than 93 percent of players master arbitrary complex linear equations in about 90 minutes. (Mastery here means they were able to complete three problems—such as solve for x: a*x+b=c+d—with no errors or extra steps.) He says this approach could apply not just in math—though that’s the natural place to start—but also subjects like reading comprehension and science.
Popović says the Gates Foundation urged him to found a company to build and commercialize technology that expands on this adaptive model, personalizing and optimizing existing K-12 curricula. “I don’t know if it would have happened without their help initially,” he says. “I had venture capitalists and many other people contact me, but I didn’t want to sacrifice the actual outcome, which I think requires a scientific approach and a huge amount of iteration. If I did it as a [for-profit] startup, I would have had to have an outcome in two years.”
Engaged Learning declined to disclose how much funding it has received from the Gates Foundation.
Popović has a lengthy resume. Engaged Learning’s technology, which he’s been working on for nearly five years, was inspired by his earlier work helping create another video game that allowed non-scientists to solve puzzles that advance science. The game, called Foldit, built on the Rosetta@Home protein structure calculation project of David Baker and has helped inform new approaches to treating cancer and other diseases.
“That was the first thing that pointed to the ability to create experts from relative novices,” Popović says. “From there, I basically realized that the same kind of principles should be applied to education at large. If we can create experts, why can’t we create mastery at schools?”
Popović says he focuses on games because they foster engagement and mastery of a subject at the same time. Also, he says, games can easily contain lots of automated variation in the problems students are asked to solve, quickly amassing data on the best ways to present material to optimize learning.
With funding from the Gates Foundation and U.S. Defense Advanced Research Projects Agency (DARPA), Popović and his team of about 45 people at the University of Washington Center for Game Science have developed games that have been played by more than 7 million kids around the world.
Games are one thing. Marketing new products and services to K-12 education is another.
“There’s no shortage of better mousetraps for education,” says Engaged Learning CEO John Mullin, who previously co-founded and led Seattle-based Teach First, a professional development company focused on helping teachers identify effective practices for challenging students.
And the procurement process in many school districts is “irrational,” he says, with decisions made by everyone from state superintendents and chief academic officers to district school boards to school principals to individual teachers. Budgets to try new products and services are typically limited to a single school year, he says, which often isn’t enough time to show whether a new technology or approach is effective.
The market is dominated by big textbook publishers—Pearson, McGraw-Hill Education, Houghton Mifflin Harcourt, and Scholastic—“because they have the financial wherewithal to live inside of school districts” and state departments of education, Mullin says. “It’s very challenging for a startup, as great as their idea might be, to sustain” that kind of sales and marketing, he adds.
Challenges aside, there is a sense that “education is having its Internet moment,” Mullin says. Venture capitalists are taking note, pumping $1.25 billion into 378 edtech deals globally last year, according to CB Insights, breaking the billion-dollar mark for the second straight year.
“The ability to actually use data in a meaningful, objective, and real-time way, to be able to take that data and translate it into something that can help a teacher know what to do right now for that specific student who’s struggling—who [the teacher] might not have realized is struggling to begin with without this data—is hugely powerful, if it can be done at scale,” Mullin says.
Indeed, Engaged Learning makes some tantalizing promises about a complex piece of the education puzzle: How do teachers, faced with larger, more diverse classes, tune their instruction to help individual students? It’s long been an intractable problem, Popović says.
The company’s approach is enabled by the availability of computers in classrooms, as well as more advanced tech that is transforming many other fields: real-time data capture and analysis, machine learning, and cloud computing.
Popović distinguishes Engaged Learning’s idea of adaptive curriculum from other approaches that have used the term. Existing “adaptive” curricula may simply mean that after taking a test, students either move forward to new material, or repeat the old material, depending on their performance, he says. Or the system may gather data and a team updates the curriculum every six months based on student performance. (If you want to impress your pedagogically inclined friends with some jargon, this is sometimes referred to as a gated adaptive curriculum, as opposed to Engaged Learning’s “generative” adaptive curriculum.)
He sees several problems with the gated approach, each of which Engaged Learning is designed to solve: Test outcomes don’t tell teachers the best next set of problems to give to individual struggling students to help them advance, he says. Repeating the same material can be demoralizing, causing students to disengage and fall further behind. The approach is also inefficient.
Popović, a computer scientist, says Engaged Learning melds several concepts from his field to address these problems. It applies a “thought-process language” to encode everything a student needs to learn for mastery of a subject. Algebra, for example, is broken down into small conceptual parts: variables, say, or equations. Questions that cover those parts are identified.
The system applies something akin to software verification techniques to figure out all the possible paths—or sequences of problems—a student can use to achieve mastery. “If you want somebody to definitively learn something, you want them to have visited every part of the thought process,” he says.
In this way, the system can identify and present problems that introduce the concept a student is ready for, based on having correctly solved earlier problems. Likewise, it can surface a problem that revisits the specific concept a student may be struggling with—even if it’s from a few lessons back—in a simpler form. “We can find just the right remediation piece, generated specifically for that kid,” Popović says.
And, as the company’s name indicates, Popović places as much emphasis on keeping students engaged as on helping them achieve mastery. Say a student is repeatedly struggling to advance to the next concept. “The best thing to do is to try something tangential for a little bit to give them a sense of accomplishment, and then merge that with the key thing they were struggling with,” Popović says.
In a 17,000-student trial using a learning game that employs the underlying approach, kids not only completed 30 percent more problems correctly than a control group, they also worked longer on a culminating problem designed to be well above their current level of knowledge. Popović says the system essentially makes students struggle just enough to feel challenged, but not frustrated. They feel that with each struggle, they improve, building up persistence, he says.
“This has implications in school, but also in life,” Popović says.
What’s more, as data is accumulated through students’ work on individual problems and exams, the Engaged Learning system determines the most effective sequences of problems. In this way, individual elements of the curriculum—including non-technological aspects like a teacher’s examples or a lesson at the chalkboard—could be evaluated and adapted not only for individual kids, but also to entire classrooms, schools, districts, and beyond.
For example, I had a math teacher, Craig Holt, who taught the distributive property by having pairs of kids stand as though we were inside of parentheses, holding ice cream cones. Holt would “distribute” a scoop of ice cream on each student’s cone. It definitely caught the attention of sixth and seventh graders.
Engaged Learning could measure whether students were suddenly mastering the distributive property after Holt’s ice cream lesson. If they were, the system would ask him to input his notes on the lesson. “You have best practices in non-technology ways [of teaching] collected and immediately spread to all the other teachers for further experimentation,” Popović says. “Is this something specific that only this teacher can do, or is this a gem that everybody else would benefit from?”
A system like Engaged Learning obviously requires computers in the classroom, ideally one for each student, Popović says. That’s not the reality today for most U.S. classrooms, let alone for the developing world—and Popović does aspire to take Engaged Learning to a global scale—but it will be soon enough, he says, thanks to cheaper smartphones and tablets.
Until one-to-one computing in classrooms is a reality, the system can still suggest the best set of activities for classrooms with limited resources, Popović says. It can also help teachers determine which students could benefit most from using a computer at a particular point in the class. “Is it kids that need remediation, or kids that need to zoom off because they’re way ahead of everybody else, so that the middle can be focused on by the teacher?” he says.
Even if there is a computer for every student, Popović believes the teacher will always have an integral role. (He does not have some Ender’s Game vision of the future of education, even though people often mention the sci-fi classic, recently a major movie, when he describes what he’s working on.) “What we’re trying to find out is the way in which technology can maximize the outcomes of all the other things a teacher can offer, rather than in any way replace it, because that would be a big mistake,” he says.