Depth Kit – Exploring What’s Possible

At the ICONs Festival in Belfast we had the opportunity to meet and hear from James George – the director of Specular, a New York based creative studio “exploring the application of photography and interactivity towards immersive experiences.”

@obviousjim showing us that filmmaking, fine art & programming make for a pretty incredible combo

A photo posted by Mary Stewart (@marygstewart) on

He spoke to us about his experimentation as a programmer as well as an artist, and how he loved the blurry middle ground between fine art and computer science. Over time, he and a group of people with similar passions began to experiment with how to combine the fields, resulting in a collection of incredible pieces which range from documentary film to collaborative interactive art. One of the pieces of technology that Specular developed was the DepthKit; a tool that allows the recording of video footage in 3D.

This tool fed into their experimentation in the world of film and documentary film in virtual reality. The depth kit was an integral piece of technology in their development of Blackout, a virtual reality interactive documentary based on the New York Subway. As a viewer, you can place yourself in the minds of the different characters on the train. This new take on documentary film making blew my mind when I first saw it. Not just because of the impressive visuals or concept, but because of the potential that it has.

When we saw Blackout and the DepthKit last semester we were dying to look into what it could do. I love the idea of creating a piece where we combine mediums to contribute to an overarching story; the depthKit would allow us to combine our 3D knowledge with filmmaking and virtual reality.  James George and the Specular team have come at the virtual reality space with an artistic and computer science approach. Their results are incredible and highly experimental. With our backgrounds in digital art, I think we could have an interesting and original contribution to this space; a space where anythig goes because it’s so new. 

After a little research I came across a few other projects which use the technology to create artifacts or documentary film.

In Love Child, they use an experimental set of visuals which sit on the border between documentary film making and fine art. The imagery pairs the intimate setting of the bedroom with vast natural landscapes, layered over each other to add depth, while the depth kit video records the interaction of a couple. What I like about this piece is in their interpretation of the depth kit data; they’ve used soft colours and interesting 3D textures to compose the scenes, challenging the Matrix-like tech-inspired visuals which are all too common in the emerging world VR.

Evolution of the idea(s)

After a lot of research into the world of virtual reality, we started to brainstorm again to nail down the right idea. We realised that with this new platform, we had to approach our stories in a new way; treating them as experiential rather than a purely linear narrative as we would approach film. While linear narratives may still exist within the experience, we have started to consider the 360 experience as a more integral element of our piece. This may seem like stating the obvious, but it’s very easy to forget after years of creating linear narratives!

The future! #oculus

A photo posted by Mary Stewart (@marygstewart) on

Collection of Dreams VR Idea

One of the first ideas we rounded out was a VR experience that places the user in a dreamscape and allows them to explore the hopes and dreams of others. We would record interviews on a depth kit, capturing the hopes and aspirations that people had throughout their lives. The user could travel through the space, and select the stories that they wanted to explore.

The concept evolved from the idea that while not all dreams become reality, nearly everyone has dreams for their lives. A child may dream of becoming batman, an adult may dream of a specific career, or specific lifestyle, or to achieve certain things. Dreams are often not shared for many reasons and I would argue that one of these reasons is because by sharing your dreams, you expose a little piece of yourself that matters to you making yourself vulnerable. People may poke holes in your dreams, infusing doubt or negativity, which may dishearten you from ever pursuing them. This piece would provide a keyhole look into the minds of strangers; the viewer could explore a space where they normally are not permitted, gaining a little insight into a vulnerable place in a stranger’s mind. There’s something captivating about that. 

Transition to Exploring Happiness

While we all liked the idea of exploring “dreams”, we didn’t feel that the story was quite solid or specific enough to run with. We had another brainstorm and as we grouped our ideas, and themes of happiness and personal wellness rose to the surface.

We had thought of creating a social experiment with the concept being shaped with data which we would gather throughout the project. This made me think of ‘The Happy Show’ by Sagmeister and Walsh, a data driven project which is essentially a collection of visually rich infographics exploring the science behind what makes us happy. What I loved about their show was that there were a range of mediums to demonstrate their concepts, using both digital and physical interaction design to communicate the messages.

On the back of this Nicole introduced the film Hector and the Search For Happiness, a movie about a psychologist’s pursuit of a happy life, after realising he was drifting through each day in auto-pilot.

These references sparked a load of ideas, and we began to see that the theme of happiness or personal wellness was a thread that ran through many of our previous brainstorms; mindfulness is about find peace within yourself, dreaming is based around the heart’s desires, and even the topic of leaving home is about a pursuit of finding your place in the world. We thought that this could potentially be a strong theme to explore for our VR experience, as it is:

  1. Something that we are all passionate about
  2. A universal topic, allowing for a piece which would be of interest to many people
  3. A positive emotion – which could allow for a playful approach

IMG_9876.JPG

We all went off and did a range of research on the topic of happiness (LINK TO OTHER POST), and then came together again to share and refine the idea with these new findings and insights. We ideated a range of possibilities and imagined how the piece could connect together, thinking both about the theme and stories as well as the visuals.

This slideshow requires JavaScript.

After learning the hard way last semester, we were determined to focus and clarify our ideas to make sure we were pursuing the right goals. Even though the specifics of the idea are still very open, we thought it would be helpful to create a short statement to define the current idea:

‘Happiness is…’ is an interactive piece which collects moments where people have felt happiness in their lives. By exploring an environment the viewer will see and hear the stories of others; of their happy moments, big and small. The idea is to remove the viewer from the busyness of their world and into a space where they can take a moment to be still and reflect on their own happy moments.

Overcoming Ideation Obstacles

We came back to this statement the next day and found it hard to know the next steps to take. We knew that we needed to refine the concept further, but were not quite sure where our weak links in the idea where. I really wanted to nail down what it was that made the idea interesting (or if it was interesting at all) as without understanding this, we may overlook integral elements of the idea in the development process. We sought feedback from the class and from Matthew Morrison (great guy!) who was working nearby. -side note: co-working spaces make these feedback sessions effortless- We asked him about what happiness means to him. He shared a few of his happiest moments and happy places, but also noted that he felt that the most moving happy stories featured the journey to get people there-what they had to overcome to reach that happy place. This was a great point, and we began to think about the journey of happiness, rather than it featuring as a ‘big event’.

We challenged a range of things, specifically the purpose of the project raising questions like:

  • Why it would be interesting to a user?
  • How do we want the user to feel?
  • What is the overarching story we’re telling?
  • How would the user move through the space?
  • Liner vs open approach?
  • How to combine artistic styles and types of content?
  • How do we keep interviews interesting?

After raising these questions we refined the idea again and start to think about a more focused subject area. Instead of a broad idea of a pursuit of happiness, we thought it could be interesting to think about this pursuit in one or a number of individuals lives. We thought about focusing on a small group of people such as service workers, celebrities, mums, graduates…

IMG_9881.jpg

After a discussion, we were all drawn to the idea of interviewing graduates and exploring how their journeys unfolded after they left the art college. Exploring another artist’s journey is an interesting way to blend a range of the ideas we’ve been studying over the past week – a pursuit of happiness and the chasing of dreams and aspirations. What was particularly poignant about this idea was the fact that this piece will be showcased for the first time at the end of year degree show; a time where we, and many other graduates, will be thinking about our futures and what’s to come next. Each of us hopes for the best and for success in the future – by exploring another artist’s journey to artistic success, it could be a hopeful and insightful way to end our time at the art college. This is obviously still in development, but I’m hopeful for this one!

Visual Inspiration

Ben Black

Ben Black’s 30 Second Story uses light and tone to create rich visuals which leave the viewer guessing what’s going on. The mixture of textures in the setting of an empty space gives an immediate surrealist feeling, with objects that don’t quite seem to belong in the same place. I think the reason this works is the monochrome tones connecting everything together. I’m not sure if monochrome would be the way forward for us, but it will be useful to keep exploring ways to blend different styles to create a cohesive piece.

Chris Trappineirs

Chris Trappineirs is a paper artist based in Belgium, who creates delicate portraits by intricately cutting out paper stencils and finishing them with spray paint. Above are examples of his work in both paper and ink studies. The half finished and abstracted portraits leave lines that tail off into the empty space. Thinking about using people’s stories as part of our piece, this ‘unfinished’ illustration style could potentially led itself to our subject area; like the lines that flow freely in Trappineirs’s pieces, the stories of individuals are ever-evolving and always moving in ways in which you don’t expect.

Henrietta Harris

Henrietta Harris is an artist from New Zealand who specialises in surrealist portraiture. I was drawn to these images of people painted in a fairly traditional way, but then twisted or abstracted from their normal form. The themes we are exploring with our piece examines people looking back on past memories. This type of surrealist style could be an interesting way to represent the removal of reality, and the passing of time as people reflect on what once was and what could have been.

Typography & Multimedia

This piece by Pokras Lampas and Noobusdeer caught my eye because of the unusual pairing of mediums. They use both physical painted calligraphy as well as a digital element to connect the panels together. When thinking about our piece in the 3D space, we may have the challenge of finding a way to connect elements recorded on the depth kit to the rest of the piece. This could potentially be done by fading transitions but I think it could be more interesting to have a visual connections, so that they seamlessly like to each other.

 

Story Ideas – Pt 1

Over the weekend I took some time to get my head around our ideas from the previous week. We had spent a lot of time thinking about the possibilities of virtual reality as well as how we would approach it. We also had a number of themes we were interested in exploring such as the power of nature, stillness and mindfulness, literature and words and cultural themes connected to home.

After an in depth look at storytelling in my recent blog post, I decided to think more about the specific story or message we wanted to convey in our work. It’s important to distinguish between a story and a theme, and we had been missing the story side in our ideation. I recently read in Fast Company’s article on 6 Rules For Great Storytelling that the stories are about how you felt, rather than just about what transpired. Zax writes,

Stories can be about very small stuff, so long as the emotions involved are big.

This slideshow requires JavaScript.

On this theme of emotional storytelling and on the advice of Stanton (2012), I began to dig into my personal memories to see what story threads I could come up with. Here are a few:

  • I thought about times I’d lost things I really wanted.
  • I thought about heartbreak.
  • I thought about the transitions I’ve made through stages of my life.
  • I thought about home, and all that has shaped me.
  • I thought about feeling trapped in a place I didn’t feel part of, and on the other side, times where I felt completely at home.
  • I thought about the funniest moments of my life.
  • I thought about the dreams I have had through my life and how they have changed.

I started drawing up some story ideas on the back of this brainstorm, and with the decision to use virtual reality and my research on storytelling within that medium in mind, I started to see a few potential stories emerging. The ideas with a positive tone are appealing to me, as I like the thought of the viewer leaving the experience feeling uplifted and inspired. Coming to the end of a five year journey at art college, I would love for my work to reflect the feelings I will have (or should have) by the time I get there; ready to move on to something new, oblivious to what the future may hold, but excited nonetheless.

Over the next few days, as a group we are going to solidify the ideas we have together. Watch this space.

Stanton, A. The clues to a great story. Accessed on: January 31, 2016, fromhttps://www.ted.com/talks/andrew_stanton_the_clues_to_a_great_story?language=en#t-1129627

Storytelling in Virtual Reality

We have started to look more seriously into designing a virtual reality experience. VR is such an exciting space to explore for many reasons. Firstly, it is very new, which means that there is room for creating something that hasn’t been done before. As a final year project, this is an exciting prospect! Secondly, it’s potentially the most experiential form of film that exists to date. The viewer is immersed in the setting you create for them – if done well it feels like you are part of the experience, instead of just a viewer of it. While these aspects of VR are so exciting, they bring new considerations that differ to traditional storytelling. This article on VR Dribble (2015) highlights some of the specific considerations of storytelling in VR in a very succinct way. Below are a few that stand out:

  • Consider your Viewer

It’s important to think about how the viewer is experiencing the piece. How do they move? Do they move? Are there cuts? How does this feel to the user? In filmmaking, these problems have been solved over and over but it is different in virtual reality, and each decision will have an impact on the tone of the piece. This is something we want to keep in mind through the whole process.

  • Presence

“Audience participation within virtual reality is an idea recently referred to as ‘presence’. Because our audiences are no longer passive, they are inherently a part of the scene.” – White (2015)

With this fully immersive experience in mind, I have been trying to get my head around how to construct a story within it. There are so many opportunities for trying something different, and the VR experiments that others have tried already are testament to that. There are 360 films being made, interactive VR games, interactive films and more. With each year that passes, new and inspiring pieces roll out that push the limits of the technology and challenge it’s potential. The Oculus Story Studio are one of the studios who have recently been experimenting with a range of styles and have some projects in the pipeline which use VR in ways that it has not yet been used.

What Oculus Story Studio highlights is VR’s ability to evoke empathy in the user. Of course empathy is not new to filmmaking, but virtual reality is unique in its ability to bring the viewer into the scene, which in turn, allows the viewer to connect with the experience in a different way. It is an experiential medium and the viewer feels present in the world they’re inside; they stand with the cast in their environment, seeing what they see and hearing what they hear. This new role that the viewer takes on in virtual reality is something I’ve been thinking a lot about while constructing the story; could this be an opportunity to create a deeply emotional, inspiring or thought provoking piece? I am hoping the answer to this is yes!

  • Using Theater as a Tool and Guiding Viewer Attention

White poses the questions, “How do your characters interact with that environment? Where do they enter/exit from? What lighting would best aid you in creating the feel of the world you’re developing?”

The 360 world of virtual reality gives us a lot to think about. In Chris Milk’s Ted talk below, he describes film as “a sequence of rectangles played in sequence” where as VR is completely different to that. We are going to have to train ourselves to think in this new medium, rather than what we’re used to. Not everyone believes that the VR open environment is necessarily a good thing, as Ed Catmull (2015) of Pixar commented,

“Linear narrative is an artfully-directed telling of a story, where the lighting and the sound is all for a very clear purpose. You’re not just wandering around in the world.”

Catmull brings up a fair point, as the constraints of the screen gives the artist complete control of the story. In a sense, virtual reality is the release of control from the artist – we’re saying to the viewer, ‘go and take the wheel on this one – choose your own adventure’. It is going to be a learning curve in our story writing to cease the right opportunities to take advantage of the digital space that virtual reality brings, but it is an opportunity to explore environment creation in a new way. Bring on the challenge!

 

  • Experimentation

Basically, break the rules. Well, not exactly, but this is a new field and no one really knows what works yet so experimentation is important. One guy whose experimentation has led to some of the most impactful virtual reality projects to date is Chris Milk.

While there are a number of guidelines and suggestions that people have shared openly online, there is so much yet to be explored. For this reason, it will be important to conduct our technical experiments from the very start to begin to see what will work and what won’t work. While launching ourselves into a new and developing field is a daunting thought at times, I am seriously excited about getting stuck into this project. If university isn’t the time for taking risks, then when is?

 

Citations:

Catmull, E, Dredge, S. (2015) Pixar co-founder warns virtual-reality moviemakers: ‘It’s not storytelling’. Available from: http://www.theguardian.com/technology/2015/dec/03/pixar-virtual-reality-storytelling-ed-catmull

White, J. (2015) Storytelling in Virtual Reality: A Starter’s Guide. VR Dribble. Available from: http://www.vrdribble.com/allthingsvr/2015/11/4/storytelling-in-virtual-reality-a-starters-guide [Accessed 31/01/2016].