Sheffield ’16: New dimensions in storytelling

A project showcased at Sheffield Doc/Fest will allow people to pose questions to a 3D video of a Holocaust survivor, which then answers in real time.
June 10, 2016

A hologram-like technology developed by a Holocaust education institute in California is creating a potential new platform for filmmakers to store and present archival material.

The USC Shoah Foundation’s New Dimensions In Testimony uses a mix of multi-dimensional video and audio responsive software to allow members of the public to interact with a projected image of a Holocaust survivor.

The project, which will be showcased as part of Sheffield Doc/Fest’s Alternate Realities Summit this weekend, allows people to pose questions to a 3D video of survivor Pinchas Gutter who then answers in real-time.

The genesis of the project began in 2011 when USC Shoah partnered with Conscience Display and the University of Southern California’s Institute of Creative Technologies (ICT), which built the body-scanning technology Light Stage 6 that director James Cameron used to scan actors and turn them into animated creatures for his 2009 blockbuster Avatar.

USC Shoah used ICT’s facilities to film Gutter over a week in 2014. They asked him 1,250 questions and put his answers into a databank. Researchers then use natural language processing to search that databank for a corresponding answer when someone asks the projected video of Gutter a question.

The organization has since partnered with the the Illinois Holocaust Museum to install a temporary working model of Gutter and has raised money to interview 12 more Holocaust survivors for future New Dimensions projects.

For archivists and researchers, New Dimensions in Technology interactivity is a great hook to encourage museum goers or students to explore first-person testimony but the project is not just about a flashy display of technology.

“This is all about content,” explains Stephen Smith, USC Shoah’s executive director. “Technology and visualization are by-products of decisions that we’ve made to enable different audiences to interact with this content in different settings – now and in future.”

The fast pace of technological change is not only impacting the ways archival footage is captured and stored by foundations and footage firms, but how that material can then be accessed by researchers and viewers.

According to USC Shoah’s internal research, average session times that educators spend in its archives reviewing first-person testimonials is around 45 minutes while members of the public spend roughly 27 minutes viewing that content online.

“Video testimony is very sticky,” says Smith. “People stick with it longer and come back to it often.”

Moreover, New Dimensions’ long-durational interview format means USC Shoah amasses a deeper archive. Gutter’s interview resulted in 25 hours of additional material compared with the two-and-a-half-hour interview he gave the foundation in 1998.

Given the youngest Holocaust survivor working with USC Shoah is 82 and the oldest is 92, the researchers have created two interview formats to take into account the survivors’ needs but so far they have been keen to sign on.

“Interestingly, the survivors have had the lowest barriers to wanting to do this,” says Smith. “They’ve seen the march of time and how being a part of new media is an ongoing part of telling their story. We’ve had virtually zero barriers to entry from that generation.”

New Dimensions In Technology has not been developed as an alternative to USC Shoah’s archives on the Holocaust and genocide but it could be a future application for documentary filmmakers whose work focuses on those subjects.

Although Smith will be at Sheffield to talk New Dimensions, he is also touting a handful of initiatives to rescue and archive collections in danger of vanishing – including first-person testimonies collected by filmmakers.

Docmakers typically end up with much more interview footage than can fit in film so USC Shoah is starting to work with directors on oral history archives. The first collection under the filmmaker program is that of the late J. Michael Hagopian, founder of the Armenian Film Foundation and director of several docs on the Armenian genocide.

During the 1960s, ’70s and ’80s, he traveled throughout the Middle East and Europe interviewing members of the Armenian diaspora as a side project to his wildlife and natural history commissions. In all, he amassed 400 survivor interviews plus b-roll and log books.

It’s too early to say if USC Shoah will work with a docmaker on an original New Dimensions In Technology project, but Smith says the foundation is open to suggestions. It is currently raising money to film the only survivor of the Nanking massacre who is capable of flying to Los Angeles and sitting for an intensive interview.

The project is part of USC Shoah’s ongoing effort to interview the 100 remaining survivors of the World War II-era atrocity, in which an estimated Chinese civilians were massacred by Japanese troops.

Throughout the development phase of New Dimensions in Testimony, researchers constantly held each technological decision against ethical ones. Was creating a virtual image of a survivor appropriate? Would it be a misrepresentation? Would people feel tricked?

The decision to use video rather than digital techniques to morph between questions and answers was ultimately an ethical one. Even though the system of capture would have allowed USC Shoah to create a hologram-like character, it was determined that people would feel like they had more agency over video than a digital character.

The rise of virtual reality is also posing new ethical questions around visual rendering of the Holocaust. USC Shoah is developing a VR experience called Landscapes Of Memory that will allow users to tour the Auschwitz concentration camp and view original photographs and documents, and is still grappling with what material to include and how.

“I often put virtual and reality in inverted commas with a question mark,” says Smith. “I don’t know what virtual is and I don’t know what real is. We’re still figuring that out. The blurring between the two is where the ethical problems or misrepresentation could lie so we have to be super careful in that environment.”

About The Author