Impossible Experiences: Experiments in Augmented Reality
by Kirsten Nelson
Richard Humann used Membit to create his location-based augmented reality multimedia art installation, Ascension, which placed 12 newly imagined constellations over the city of Venice, Italy.
The promise of augmented reality is as seemingly unlimited as the technology itself, adding new dimensions to our experience while also breaking down physical barriers to see beyond what was previously opaque. As more augmented reality glasses enter the marketplace, and mobile-based apps provide users with this new way of experiencing the world, we’re just beginning to see how augmented reality enables experiences that were impossible until recently.
Already, the worlds of art, education, corporate training and robotics are shifting with the presence of augmented reality. Where once there were barriers to the forms of communication and information sharing in these spaces, these formerly impossible experiences are happening now.
Adding Artwork to the Sky
For artists working in realms of creative expression that can be difficult to realize in tangible form, augmented reality is a valuable medium. And as the technology continues to evolve, new tools can provide a more direct translation of an artist’s vision, observes Karina Mitchell, Director of Community and Partnerships for Membit: “Now people are seeing things very much in a way that creatives are already seeing the world around them.”
Originally conceived as a geolocative photo sharing app that allows pictures to be placed and viewed in the exact location they were captured, the Membit platform has become a gateway for users to share creative works as well. Seeing the potential for creative expression, the team at Membit is collaborating with artists to bring new work to the world.
One such artist is Richard Humann, who worked with Membit to create augmented reality works reflecting one of the earliest visual storytelling methods. Harking back to how the Greeks extrapolated their myths across constellations of stars in the sky, Humann is using augmented reality to add new drama to the skies above. The first such endeavor was Ascension, a location-based augmented reality multimedia installation comprised of 12 newly imagined constellations suspended over the city of Venice, Italy and made viewable from anywhere in the world via the Membit app.
Humann worked with Membit to create The Tiger Horse, an augmented reality
piece inside the DMZ between North and South Korea.
From there, Humann went even more site-specific and impossible with his augmented reality works. In 2018, he installed The Tiger Horse augmented reality piece inside the Civilian Control Line at the Demilitarized Zone between North and South Korea during his exhibition at the Hangang Art Park in Seoul.
The pairing of augmented reality with specific locations adds a layer of “cultural density” to an artistic venue, Mitchell notes. “That’s why it’s important to be thoughtful about how we pair experiences with location. Once people see the augmented art, it’s then part of their memory, their experience and their feeling about the place. Even though it’s not a physical thing, it’s very much part of their memory.”
Bringing Archives to Life
Now that nearly everything is searchable online, it’s easy to forget that some of the most important historical source materials are still in physical form and stored in special collections. For the archivists and librarians who work to ensure that these artifacts survive and continue to provide insights for future generations, augmented reality is providing an avenue to share their wonders with more people.
Dr. Heather Calloway’s students at Washington College used augmented
reality to open up rare books and share their contents with a wider audience.
It was with the hope of bringing new life to these artifacts, combined with a need to engage the digital natives enrolled in her family history course, that prompted one archivist and special collections librarian to discover new potential in augmented reality. “Although I take care of all these things, I don’t want them to just sit on the shelf or in the closet or in a box,” says Dr. Heather Calloway, Executive Director of University Collections at Indiana University Bloomington.
In her previous post at Washington College, Calloway and her students produced a variety of expanding uses of augmented reality. Beginning with family history exhibits that augmented artifacts with links to videos and other student-created content, the freedom and flexibility of the technology eventually led to an opportunity to connect students with rare artifacts that are usually hidden away in storage because of concerns about their fragility. Opening the rare book room for their use, Calloway asked the students to choose a historical work and produce an augmented experience that would allow its interior contents to be viewed more widely by those who visited the college library.
The result was not only beneficial for students, but also members of the local community who visited the library and found themselves entranced by the augmented reality apps provided on iPads near the front entrance. There was also another ripple effect from the project. Calloway noticed that students returned to the archives and special collections for more projects, frequently referencing the materials for other coursework. “So, it’s giving new life to archives and special collections again,” she says.
Washington College students augmented artifacts and artwork
to bring new life to special collections materials that would
normally be locked away in archives.
Now, as Calloway continues her work with using augmented reality to connect more students with primary source materials available in archives, her attention has turned to a concern for the future. Augmented reality tools and platforms are still very much in a state of change, with obsolescence and new iterations often rendering previously digitized content useless. To solve that issue, Calloway and her colleagues Raven Bishop and Sarah Graff started creating their own augmented reality apps.
To perpetuate their knowledge and augmented reality experience with more students and educators, they created the PocketMuseum app, a virtual museum of augmented artifacts. The app is still in the prototype phase while the team seeks funding to get it out into the world. “Our goal is to teach other people how to do this,” Calloway says. “We want to share this.”
Training for Reality
The idea of a shared viewpoint on work tasks, for training purposes or day-to-day assistance from a network of experts, presents some intriguing opportunities for augmented reality, according to Cortney Harding, Founder and CEO of Friends with Holograms. Utilizing AR glasses, team members can share knowledge easily in a real-time and clear fashion.
When a worker is in the field and needs an assist from a remote specialist, the two can view the same scene together through the glasses. This works well for complicated technical tasks or even those requiring soft skills in social work, healthcare or customer services realms, where users need to share intelligence on how to manage various interactions.
Cortney Harding of Friends with
Holograms dons a pair of Focals,
customized augmented reality glasses
made by North.
This scenario holds additional appeal in fields where workers are aging out of field roles, but still wish to contribute to the workforce. “It’s a good way to keep people engaged in the economy when they can’t physically do work,” Harding observes, “and it shortens the training cycles for newer workers as well. They won’t have to spend as much time in the classroom, they can be out in the field more, because they’ll have someone who can see what they’re doing and walk them through it.”
Even without the glasses, AR can help people learn. Mobile-based apps provide the impossible experience of seeing the moving parts of devices and cars and even animals (in a somewhat “creepy,” application, Harding noted). “If you hold your phone up over your coffee maker, it will show you what to do, step by step.”
Seeing the World through AR
The ultimate impossible experience might be robots seeing the world as we do, but augmented reality is helping there, too. As we move toward a future where the digital and physical worlds merge, and autonomous robots share the planet with us, the next big problem to solve is how augmented reality characters and machines will navigate actual physical environments.
That’s where a company called Fantasmo is focused. They’re building the framework for indoor and outdoor 3D maps and camera-based positioning, which means they’re finding ways to capture and interpret environments for negotiation by machine intelligence.
Essentially, there is a lot of complexity inherent to solving the problem of how Pokemon can walk through a door and sit on the couch next to you. That’s the half-serious, half-humorous example that Fantasmo Co-Founder and CEO Jameson Detweiler gave when explaining his company’s work. “In order to do that, we need to build a design language that parses spaces,” he explains, “How do you define what a couch is, what is the surface that you’re supposed to be seated on versus lean back against?” All of these elements that we naturally perceive must be solved for how a machine will see.
Right now, Fantasmo is mostly working with clients on an enterprise level, but Detweiler also sees a near-term future where 3D spatial mapping will have a consumer-facing purpose. Starting in retail, for example, where in-store apps will help customers to find products and information. That involves a lot more cataloging and tracking of goods than most retailers currently employ, but again, as digital and physical worlds merge, stores will become more like intelligent machines. Essentially, Detweiler says, “it’s less about how do you bring a robot into a space, than how about we make the space a robot?”
