A Physics Rebel Shakes Up the Video Game World, Literally
What’s the connection between hardcore, chest-pounding video game action and Niels Bohr’s interpretation of wave-particle duality? It’s an Iranian-American physicist-turned-entrepreneur named Shahriar Afshar. Five years after Afshar announced the results of one of the most controversial experiments in the recent history of physics—one suggesting that it is possible, contrary to Bohr’s long-accepted theory, to observe light behaving as both particles and waves at the same time—the Cambridge, MA-based startup he founded, Immerz, is about to launch an “acousto-haptic” device that lets gamers both hear and feel gaming action at the same time.
Immerz’s product, called Kor-fx, is essentially a pair of woofers for your chest cavity, designed to enhance the sense of being immersed in a game (or a movie or a song)—hence the company’s name. Immerz showed off the device for the first time last week at the i-stage competition in Phoenix, AZ, where the Consumer Electronics Association—the same organization that runs the giant CES convention in Las Vegas every January—chose it as one of the 11 most innovative consumer technology products shipping next year. The company plans to bring the product to market in the first quarter of 2010, focusing first on PC gamers, and later on console players.
Afshar’s switch from experimental physics to gaming may sound like a strange change of direction—and it is. But there’s some logic to it, just as there is beneath the perverse and often baffling world of quantum mechanics. “My mission in life, ever since I have been mature enough to have a sense of a goal in life, has been to reveal realities that are right in front of our eyes but we missed,” Afshar says. “It excites me that there are so many hidden realities out there that we can unravel”—including the hidden monster who may be sneaking up behind you in a video game.
The Kor-fx device consists of a pair of vibrating transducers attached to a yoke that holds them snugly against a gamer’s chest. They translate the same audio signal going to a user’s speakers or headphones into a shaking sensation that is literally visceral—the vibrations echo through the user’s chest cavity and vastly heighten the sense of immersion when playing an action-heavy PC game, watching a movie, or listening to music.
Because the transducers vibrate in stereo, and because the human tactile system is pretty good at translating vibrations into directional information, it’s actually possible for someone wearing the device to sense which direction gunshots are coming from in a first-person-shooter game like Half Life, and even to feel events occurring “behind” them in the virtual world. Afshar calls this the “seventh sense.” (I’m not just repeating public-relations verbiage here—I’m one of the first journalists who has had a chance to try out the device, which adds an almost frightening level of you-are-there realism to both video games and action movies.)
Immerz, a two-person company based at the Cambridge Innovation Center, has applied for patents on the transducers. It has what Afshar calls “big name” angel investors, though he won’t identify them yet. But it’s also seeking venture-level financing so that it can start to produce the Kor-fx units in mass quantities (the company outsources a lot of its design and manufacturing work). The next big public showing for the technology will be at Pepcom Digital Experience, an event for journalists, analysts, and industry insiders preceding the CES trade show in January.
Afshar obviously isn’t your typical game-industry entrepreneur; my interview with him yesterday started off with a 15-minute discussion of quantum mechanics. In 2004, I learned, the Harvard-trained physicist presented the results of a groundbreaking experiment he’d carried out three years earlier at the Institute for Radiation-Induced Mass Studies, a privately funded research center in Boston. As Afshar explained, it’s in part due to the physics community’s incredulous response to the experiment that he is now taking a detour into acousto-haptics.
The experiment was a variation on the famous double-slit experiment familiar to any freshman physics student. Light passing through a pair of pinholes in a screen forms an interference pattern that’s most easily explained by thinking of light as a wave. Quantum mechanics, however, says that light travels in packets called photons.
In the classic double-slit experiment, if you try to measure which slit a particular photon passed through in the experiment, the interference pattern disappears; the upshot of this “principle of complementarity,” first defined by Bohr in the 1920s, seems to be that experimenters can measure the wave-like properties of light or the particle-like properties, but not both at the same time. Afshar’s test upset this conventional notion. By inserting a lens and an array of vertical wires into the double-slit apparatus, he says he was able to detect individual photons and wave patterns simultaneously. (For the details see this New Scientist article; subscription required.)
No less a thinker than Albert Einstein was doubtful about Bohr’s complementarity principle, but it’s become the accepted dogma in physics over the past 70 years, and Afshar’s results have met with widespread resistance. Afshar says he was prepared for this. “When you do an experiment that you know is going to change the very foundations of what everybody believes, you have to have a long-term plan—that is the reality of the world of science,” Afshar says. “Every time somebody does something like this they should expect at least five to ten years of complete isolation.”
Afshar’s isolation has not been quite complete: he has a position as a visiting research professor in the department of physics and astronomy at Rowan University in Glassboro, NJ. But he says he’s ready to wait for up to a decade on the outskirts of academia while the physics community assimilates his paradigm-busting research results, and while he cogitates about a new, even more convincing experiment. And he says that it was his stay in university housing at Rowan a couple of years ago—in the same dorm with video game-addicted undergraduates—that gave him the idea for a project to fill the gap.
“I don’t want to embarrass the kids there—I think they are typical of college kids all over the place—but I would from time to time tell them that they shouldn’t be playing their video games so loud,” says Afshar. “You could feel the whole place shaking. After harassing these kids a bunch of times, I realized, first of all, that my nagging was ineffective, and secondly, that there was something missing [from standard video game setups]. They really needed this pumping bass sound. So I said, instead of bothering everybody, why don’t I think about how to deliver the same experience on an individual level without having to make everything shake.”
Afshar says he spent the next two years studying the neuroscience and ergonomics of what might be called extra-auditory tactile perception—the way the brain uses information about vibrations below the level of sound to supplement its picture of the environment. Unlike visual information, which the eyes feed directly to the cortex, tactile information flows first to the limbic system, which is in charge of emotions and also helps to assemble incoming sensory experience into a basic sense of self, Afshar says.
Experiments with a series of prototypes helped him to come up with a design for a pair of transducers that would be convenient to wear and yet would be ideally placed to feed vibrations into the chest cavity, where they’d send information back to the brain along the same pathways that sense the vibrations of one’s own voice. “We are activating the same parts of the brain that are typically activated when you talk, which is strongly associated with a sense of self and also emotions,” says Afshar. “What this ends up doing is giving you a sense that you are not experiencing something external—you immediately internalize it.”
When gamers put on the Kor-fx yoke, Afshar says, it enhances the illusion of being inside a world filled with gunfire, explosions, or crashing vehicles. “It’s almost like unlocking a sense that you always had but you were not aware of,” says Afshar. “Once people use it, it’s so natural that not having it afterward is like missing all the action.” (The video below, produced for Immerz by Boston-based advertising firm Hill Holliday, is pretty good at portraying gamers’ emotional reaction to the experience.)
The Kor-fx device comes with a splitter that players can use to adjust the intensity of both the vibrations and the sound going to their conventional speakers or headphones; it plugs into the audio jack of any desktop or laptop computer. A forthcoming version of the device will work with consoles like Microsoft’s Xbox and Sony’s PlayStation. But Afshar’s vision for the technology goes well beyond gaming.
“Because it’s platform- independent and works on audio input, we have access to all the entertainment media devices that exist, from PCs to laptops to DVD players to iPods and even cell phones,” he says. “We’ve been told—and this is a direct quote from a user—-that watching an iPod movie with this device is equivalent to having an Imax experience.”
Speaking of Imax, it wouldn’t be surprising to see Kor-fx devices turning up in movie theatres or amusement-park attractions. And after the entertainment market, acousto-haptic technology could even have applications in real-time sensing—it’s easy to imagine how the directional sense that the transducers convey might assist jet fighter pilots or people with hearing or vision impairments.
But all of that is in the future. The most logical way for Immerz to bootstrap itself, says Afshar, is to start with PC gamers, who are already accustomed to the idea of buying peripherals like force-feedback joysticks to enhance their gaming experiences.
Afshar says that if there’s one thing he’s learned from his time as an experimental physicist, it’s “not to regard any event or observation, as mundane as it seems at the time, as irrelevant. Once you have an observation, you think about its application and you run with it.”
That approach hasn’t won him many fans in the quantum mechanics community. But it might just bring him a few among college dorm proctors.