Where are all the holograms?

Everybody knows what a hologram is. It's Princess Leia in Star Wars: "Help me, Obi-Wan Kenobi. You're my only hope."

It's Captain Jean-Luc Picard in Star Trek: The Next Generation moving through an imaginary 1940s San Francisco as private eye Dixon Hill. It's 40-foot tall ballerinas dancing in LA streets in Blade Runner: 2049.

Except that it's not. "What people think of as holography that they see in the movies in general isn't," says Daniel Smalley, associate professor in the Electro-Holography Lab at Brigham Young University. "The academic definition is that to have a holographic display, you must have light diffracting from some modulation surface. That means that you have to be looking at a screen to see the image."

Most of what the public thinks of as holography, and what's been portrayed as such in much science fiction, is actually "a different type of display technology called a volumetric display," Smalley explains.

In essence, a hologram is just a still image of the interference pattern of different light waves diffracting from an object. Unlike the living, moving images of science fiction, they "move" only when we change our viewing angle, which is why we're not (yet) watching holographic movies.

Although the principles of holography were first discovered in the late 1940s by Hungarian scientist Dennis Gabor, the concept didn't become well-known outside a relatively small circle of specialists until after the invention of the laser in 1960. Gabor was looking for ways to improve electron microscopy techniques, not something the average citizen is much concerned with. But the laser brought holography out of the microscopic realm and into the world of visible light, making it accessible to anyone with eyesight.

Optical trap display (OTD) image of the Earth above a finger tip. Credit: Dan Smalley lab, Brigham Young University

When optical holograms were created for the first time in 1962, nearly simultaneously by Yuri Denisyuk in the Soviet Union and Emmett Leith and Juris Upatinieks at the University of Michigan, the possibilities seemed limitless.

"The technical community responded with enthusiasm," writes optical engineer Karl A. Stetson in a memoir. "Here was a process that gave truly three-dimensional images, complete with parallax. Many ideas were put forward for applications, 3D television, 3D movies, dazzling displays in store windows....Little of this came to pass."James D. Trolinger, co-founder of MetroLaser, Inc., recalled it as "an explosion in the field of optics," writing in an SPIE conference proceedings that "during the next 50 years many applications of holography have both thrived and died...some, oversold, failed to meet expectations, while some were too complicated for general use. Others could not compete, never got accepted, or were impractical for general use." And yet, Trolinger notes, "In many cases, as methods approached maturity or abandonment, new technology revitalized them, and this cycle continues today."

Obviously, we still don't live in a science-fictional world surrounded by gorgeous 3D images. The ongoing pandemic and lockdowns didn't give rise to a brave new world of remote conferencing and working in simulated 3D spaces, only endless gazing at tiles of sullen, poorly lit faces in interminable Zoom meetings. Why? What happened to all the grand holographic dreams of yore? Is holography another obsolete technology of the past like laser-disc players and 8-track tapes, or can it still fulfill its original promise?

The answer to those questions involves both perceptions and technology. As Trolinger observes, the breakthroughs of the early '60s "excited many people with promises of fame and glory fostering many diverse holography communities in a short time. Much like a gold rush, hundreds of people around the world became holographic gold diggers, making holograms, discovering new possibilities and phenomena...by 1970, holography was being pitched as a savior in almost every existing industry, and engineering and research centers around the world looked to holography for new capabilities and solutions."

Some of those capabilities and solutions, such as holographic metrology and interferometry, have proven extremely valuable to scientists and engineers, even if not broadly commercialized or known to the public. Others are ubiquitous if largely unheralded, such as the holographic security images you likely carry around on your credit cards and driver's license.

Schematic of the Smalley Lab Optical Trap Display setup. Dan Smalley lab, Brigham Young University

Javid Khan, founder and director of Holoxica Ltd., noted in a presentation for the 2015 International Year of Light that even if holography hasn't yet lived up to the overblown promises of earlier decades, it never really went away. "Holography went ‘underground' in the 1990s with the research being performed by just a handful of small companies, the military, and academic organizations," he writes. That relatively small community is where most of the holographic action remains.To a public used to the notion that Princess Leia is a hologram, applications such as metrology, security, or data storage might be useful, but aren't very sexy. "As far as public perception is concerned, holography was never what the public wanted," Smalley says. It was a word attached to an image, he says, that holography was never going to be able to deliver.

Such expectations were already unrealistic for the analog holographic techniques of earlier decades that used photographic emulsions to record images, but remain a problem even for digital holography. As with any other data-intensive computational task, bandwidth and data capacity are huge hurdles.

"Holography in general is a huge computational load to large displays," explains Smalley. "If you want to make a bigger TV, you can make the pixels bigger, so the resolution doesn't necessarily go up. But the pixel in a holographic display is bound to the size of a wavelength of light, so they're inextricably connected. If you want to make a display look bigger, you have to add more pixels. My advisor at MIT was fond of saying that you could make almost any display a holographic display, but you need a million times more pixels. So that's just been the real issue."

Increasing color variations and frame rate add to the demands. "So, we just don't have the computational power sufficient to run these displays," Smalley adds. "And when we gear up to create that hardware, we have to sit back and ask ourselves, ‘Gosh, is it really worth it' given the amount of redundancy in a holographic image? The imagery from one view to the next is so similar, there's so much redundancy in that pattern, you can't help but think to yourself, is it really necessary to reproduce all this information? And so we have computational schemes that try to cut corners or make the sorts of tradeoffs to alleviate this bandwidth requirement rather than doing a brute force CPU strategy."

Another reason that we're still not seeing the ambitious 3D holograms of science fiction, explains Smalley, is that other imaging solutions can work almost as well for most needs. "My guess is that if we could find a really good killer application for holographic video display that couldn't be done by a lesser technology, I think we would see this problem solved pretty quickly, if there were hundreds of millions of dollars to resolve the problem," he says. "Absent a really compelling killer application, it's going to be a hard slog."

Some alternatives to holography have been around for more than 150 years. Perhaps the most enduring is "Pepper's ghost," popularized by Englishman John Henry Pepper beginning in the mid-19th century. For this illusion, a brightly lit figure out of the audience's view is reflected off a clear pane of glass to create a ghostly illusion, used both for theatrical performances and by charlatans summoning "spirits from beyond." Today, this trick of light is used in concerts, museums, amusement parks, and most mundanely, in the teleprompter beloved by every speechifying politician.

Pepper's ghost and similar tricks, however, are only "fauxlography," not holography, says Khan. "Any vague illusion of an image floating in space is labeled a hologram, whether it's celebrities, politicians, or a cardboard cutout in an airport. So, beware of fake holographics—remember, if there's no diffraction, then it's not a hologram."

Aside from Hollywood movie magic, what will finally give us the 3D images of our dreams remains undetermined. It is likely to be a variety of different techniques, depending on particular requirements and available technical resources.

Optical Trap Display (OTD) image of a Pokemon character. Image shares space with a 3D-printed open ball. Credit: Dan Smalley Lab, Brigham Young University

"It's the volumetric displays that are going to make possible the kind of images from movies," Smalley says. "Avatar, Star Wars, Blade Runner—all these images are going to be accomplished with things like tiny drones with lights on them, particles of dust that are being illuminated, or plasma balls. These are the technologies that are going to make those images possible."

It's true, however, that some science-fictional displays are closer to actual holograms, such as the holodeck from Star Trek: The Next Generation. "That's in my mind, at least in part, a good target for holographic technology," says Smalley. "Because everywhere you're looking in the holodeck, you're looking at a surface, you're looking at a screen. There could be a diffraction pattern."

Smalley notes that volumetric display technology already offers compelling applications, beginning with telepresence. "Given COVID and remote work, the idea of being physically present in another space is really appealing. The thing about volumetric displays is they're not just light displays: they are a combination of light and atoms suspended in the air in a particular way, so you literally can be physically somewhere else through these displays. You could imagine having a disembodied head in the room that can turn and look at each of the individuals in the room in turn. That can't be achieved with a screen-based display, that kind of a sense of presence is just impossible to achieve. There's real, tangible value that people could viscerally feel, and that's going to provide motivation to scale and develop this technology."

Such 3D imaging capabilities could be valuable in any task dependent on accurate, real-time spatial understanding, Smalley notes. Vascular surgery, air-traffic control, and satellite tracking are just a few examples in which strict separation of physical objects and spatial awareness are crucial.

"Just imagine if the people performing these jobs had a volumetric display in front of them to see the complicated orbits of satellites, and they could just know viscerally that these two satellites are going to smash into each other," Smalley says. With air traffic, he says, "if you could represent all of those aircraft as blobs in the air, they could be a physical part of the world around you, their positions, how they're moving. I think that volumetric displays, in particular, have the potential to fundamentally change the way we interact with our digital data by making it a physical part of the world around us, make it a part of our environment. Our bodies and our minds are built to interact with the 3D world."

Smalley's lab is working to make such volumetric 3D images possible through an approach called optical trap technology—or, as the lab staff like to call it, the Princess Leia project. "We take a laser beam, focus it through an aberrated lens to create essentially a turbulent focus, a focus that has lots of holes in it. And we can put particles in those holes and then drag them around, like dragging around an object with a tractor beam. And then we illuminate that particle with a red, green, and blue light. We can create images in the air, volumetric images, make these sci-fi style geometries." At the moment, such images are only about a cubic centimeter in size, but Smalley and his team are striving to scale that up to more practical dimensions—at least the eight to 10 inches of the Princess Leia image of the movie.

Other researchers are exploring digital approaches that can create and manipulate diffraction patterns in a computer—rather than on film as in analog holography—and then create images that can be illuminated not simply by a laser but any sufficiently monochromatic light source. "Digital holography, the ultimate marriage of optics and electronics, offers great promise in essentially all applications of holography," writes Trolinger.While we may not yet inhabit the wondrous holographic worlds of our science-fictional dreams, it's also true that the notion that holography is a failed, obsolete technology is as much of an illusion as, well, a hologram itself.

Mark Wolverton is a freelance science writer and author based in Philadelphia.