Portrait of The Artist explores story of internationally award winning painter, Colin Davidson in virtual reality. Through 3D video with animation and an interactive gallery, participants are invited to immerse themselves in the virtual space and experience documentary film in a new way. This post highlights a brief overview of some of our decisions and insights, that have been informed by research and testing every step of the way.
An Experiment In Virtual Reality
Who doesn’t love fresh powder?
VR gave us an opportunity to play with something new. It’s not every day you get to work with tools that are in their prototype stages, and a medium that does not yet have “rules”. This project explores the potential of VR in documentary storytelling, but it also opens the can of worms that asks, what else? I don’t know about you, but I want to be a part of the community shaping the future of storytelling – and it is clear that virtual reality has a very interesting role to play.
Storytelling that’s immersive.
Documentary film has always been something that’s difficult to define; John Grierson describes it as the “creative treatment of actuality”. Although there have been a number of experimental documentary films in the history of the medium, documentary films traditionally follow a linear narrative, where the director choices shape the story.
This project challenges that tradition, transitioning the audience’s role of a spectator to a participator. Virtual reality helps us create a more immersive experience.
Oculus Story Studio (Brown, 2015) highlights VR’s ability to evoke empathy in the user. Of course empathy is not new to film making, but virtual reality is unique in its ability to bring the viewer into the scene, which in turn, allows the viewer to connect with the experience in a different way. It is an experiential medium and the viewer feels present in the world they’re inside.
In the interview section, we used the Depth Kit, a tool that captures 3D volumetric video. This allowed us to create a space that felt like Davidson was sitting across from the participant, as though he is only a meter away. And then Colin shares the details of his work, the values behind them – it’s incredibly personal.
We then used a mixture between particles, animation and an ambient audio soundtrack with a noise cancelling headphones to create “presence”.
“Audience participation within virtual reality is an idea recently referred to as ‘presence’. Because our audiences are no longer passive, they are inherently a part of the scene.” – White (2015)
In Portrait Of The Artist, the participant stands with Colin Davidson, as his world is built up around them. Sound floods the space to match the tone of Colin’s words. There is something very powerful about that.
Rethinking The Archives in Documentary Film.
Inspired by progressive documentaries like, The Universe Within, be began to dream up ways to present archival information.
With the explorable gallery, the participant has, unlike screen based meduims, the freedom to explore a space and the control to engage with whatever information they choose. The archive is active and engaging; Colin narrates the participant through the space, with thoughtfully curated audio clips explaining the thought process behind the work. The participant can walk up to the paintings and move millimeters away from them, viewing the details of each paint stroke or sketch.
This area of the piece is to accompany the interview which acts as the main body of our documentary. If we were to have more time, I would love to further develop the user interactions with Davidson’s work. As this gallery stands, there is still a delightful user experience. Imagine if it were enhanced and developed to engage the user more?
The Challenges Of Writing Your Own Rules.
With virtual reality as an emerging medium, there are no standardised principles when it comes to filmmaking, leaving the space open for anyone to define. There is so much that has not been figured out yet, both on a technical and storytelling level. We found that we were not only designing the work within the experience, we were designing the experience of building for VR too.
At every stage through our production process we have had to get our heads together to design with intent for this experience; from thinking up new ways to create storyboards, to rethinking video cuts and transitions, to designing ways to teach the participant how to use the experience correctly.
This was never easy, and always iterative. We learnt to test absolutely everything. As Newton et al (2016) writes,
To even scratch the surface of these questions, we need to better understand the audience’s experience in VR — not just their experience of the technology, but the way that they understand story and their role within it.
With so much to figure out, we learnt a lot. We learnt to start small, and build up. We learnt to think fast, and fail faster. We learnt that you have to be okay with killing your ideas. We found that we got much better at iterating on our work as we progressed through the weeks. We almost expected failure of our tests, which was a good thing. We developed a mindset of improvement, always asking, “is this achieving its goal?” upon the completion of each element.
I have always loved storytelling; it’s rooted in my culture, the way I was brought up, and now in my education. Over the past five years of university, one of the most important lessons that I have learnt is to lead with a strong narrative, and let everything else follow.
We began first year of our Animation degree with no technology – just post its and whiteboards, being told to create great stories before you get near the screen. I spent my summer with interaction design firm MU/DAI who taught me to first, get the right idea, and then get the idea right. I believe that the reason Portrait Of The Artist worked, was because we applied the same story-centered approach that we have been taught all these years. I am really proud of what we achieved in this light.
Of course the second part of “getting the idea right” involved a lot of technology, including the Oculus, The DepthKit, DSLR cameras, Unity Game Engine, Maya and more. There is a natural tension between the artist and the technologist; a desire to push forward the art form, or a desire to push forward the technology (Corby, 2016). For our project, we used existing or developing technologies to tell our story. The vision from the story shaped what we did in the tech, and we wrestled with it every day to achieve that vision.
What surprises many is that there are no programmers in our team. With the help of Adventure Creator, a node based plug in for Unity, we were able to create interactions anyway. What we achieved as creatives on a highly technical project, taught us that with determination and a lot of support, it is possible. We could not have conquered our technological battles or have had access to this tech without the generous support of a wide community of people.
Portrait Of The Artist is not just a story about Colin Davidson; it is an experiment in what is possible, and it is one approach that explores how virtual reality can add to the way we experience storytelling. As I graduate in the next month or so, I will be leaving university proud of what I’ve achieved, and excited about what the future holds. I can only hope that as I build on my career, Portrait Of The Artist acts as a springboard to opportunities where I can apply what I’ve learnt from this new, and unexplored world of story telling.
Peronay, N. (2009) John Grierson (1898-1972). Leeds, UK: University of Leeds. Available from: http://www.griersontrust.org/assets/files/articles/john-grierson-n-pronay.pdf [Accessed 08/05/16].
White, J. (2015) Storytelling in Virtual Reality: A Starter’s Guide. VR Dribble. Available from: http://www.vrdribble.com/allthingsvr/2015/11/4/storytelling-in-virtual-reality-a-starters-guide [Accessed 31/01/2016].
On the back of creating my website to reflect my work, I wanted to have a couple of business cards to display at the end of year show. This was something that I wanted to be really consistent with the project boards and team design; the whole space was designed with minimalism in mind, so I wanted to carry this through on my business cards.
I used the same image as my website cover image, which will be displayed during the week of the end of year show. Like with my landing page on the website, these business cards are designed as a conversation starter to chat about my work and provide contact info; the use of the visualise capture ties into the POTA project which opens ups range of conversations in VR, design and technology. I tested this out in London with MPC, and the head of legating and the head of Matte Painting said “she has better business cards that me!” I feel that this little touch ties all the work together in a professional and classic way.
This display is a culmination of all the research into exhibition spaces, gallery design, content curation and portfolio displays that we’ve explored throughout Portrait Of The Artist. We set out aiming to create a clean and crisp design, which focuses the viewer on carefully curated information about Portrait Of The Artist.
The front side is treated like a movie poster to draw the viewer in from across the room. Each key element of the piece is hinted upon, with a small segment of information to accompany the piece.
As the viewer comes to the other side, they are greeted with the team; each member has a breakdown of their roles, differentiated form one another with a set of illustrated icons in the style of the typography within the piece.
I am really proud to be at this point with our work, and feel that this display showcases our skills and our intent in a really clear way. The final stage of this will be showing it off to the general public – I can not wait to see their reactions!
In planning the making of video, I spent time thinking about what story we wanted to tell and what our angle should be. Over the last 14 weeks we have done so much, so there are many different approaches we could have taken.
To help myself think through this, I decided to write out a script. I find that when I write something down, I am forced to clarify my thoughts and think through exactly what the purpose of the task is. My research from gallery writing was applicable again, encouraging simple explanations of terms and meanings to connect with the audience. With this, I began commenting on a number of key areas. They included:
- Depth Kit Technology
- Logic without programming
- What documentary film brings to virtual reality
- Davidson’s style transferred to animation
- The purpose of the piece
- Personal and team learning
I tweaked and edited these sections, until I could read through the whole thing and feel like it flowed. I commented on what I expected the accompanying footage to be, even though we hadn’t recorded everything yet. This helped gauge what sections we should focus on.
At this stage I knew I had way too much content, but I wanted to capture much more video than I needed; it’s always a good idea to have extra options for editing.
Footage capture for B-Roll
With the knowledge that I would be interviewing the girls and using clips from our piece, I wanted to make sure there was enough interesting footage to match what we would like to say. I reached out to Rachel Hynds, a cinematographer from our class, to take some shots of us all working. I complied a shot list for her before we began and briefed the girls on what we were going to be doing.
We then took a bit of time setting up scenes in the computer and setting up the DepthKit to recreate our working process. Rachel did an amazing job of capturing the energy of the team and the selection of work we choose to display. As user testing has been a big part of the process for us, we also got Lauren, Peter and Andrew to test out the Oculus, and recorded their reactions as they came out of the space.
Footage Capture For Interviews
With the script at hand, I planned on using this to base my questions upon. I used it as an outline to direct the interviews, but tried not to put words in the girls mouths. With Nicole, I didn’t show her the script to try and capture a more natural interview – she did brilliantly! Then for Fiona, because I had to capture a few specific comments about the art direction of the piece , I showed her the outline of the script to help her think of the answers. Fiona then interviewed me doing much the same. This was a much more time consuming process than I had anticipated, but it was worth putting the time in for ease in the edit.
Editing The Making Of Video
This was a challenging job, partially because there was a lot of brain power involved in cutting a great story together, but also because I felt a lot of pressure to make this video great. This making of video is how we are presenting our piece to anyone who does not wear the headset. It’s our hook to get people interested. We wanted to tell a great story and we wanted to comment on VR and I was responsible for making that happen.
Choosing The Best Takes
I began by sorting through all of the footage, listening to the audio clips and noting the best versions of each. I had got all three of us to repeat a number of sections so that in the editing, I had the flexibility to choose who said what and when. This meant that I would not have to sacrifice the best takes, just because I needed to cut to another individual.
Editing A Story – TK01
I took my first pass at the video and got Fiona to watch it and give me feedback. Her overall reaction was great, but she suggested that a middle section would be switched up a bit, as it was commenting on a very specific thing for a very long time. When I looked back at this, I knew she was right. This section I had struggled to find the right clips to introduce different sections of the piece, so it ended up being a lot of information on one area, which wasn’t what we needed.
To combat this, I collected all the clips that were potentially suitable, wrote down what I wanted to be saying, and then started to play around. This is a messy process of trying and testing, and anyone with OCD would not be a fan of what my screen looks like in those moments. After a good while of chopping clips up to manipulate sentences, I managed to cut something together that felt natural. I maintained the essence of what was there before, but created room for more clips of other work to filter in.
Overcoming this hurdle allowed me to refine the edit and then start to add the B-Roll and clips of our work. I tried to match footage of the individual or the topic, with the interviewee. As well as adding a visual richness to the video, the clips allowed me to hide the cuts in the interview footage.
I then synced the audio which we recorded externally from the camera audio, editing some of the clips to match the levels. I figured out how to eliminate noise from the clips, but the levels did not match each other. At this point Fiona came to help out and got me a higher quality audio track.
Challenge In Screen Record Quality
The B-Roll that Rachel had captured was fantastic, but I noticed that the quality of the screen recordings were not as good as I had hoped. We had always struggled getting good quality with the Oculus, but we had thought that we had cracked it when Nicole made a build for MAC and recorded the footage on her laptop. Unfortunately this was still not as good as we had hoped.
What was puzzling was that I had done a screen record from within Unity with the exact same assets, and the quality was much higher. As Nicole’s time was freed up with the build made, I asked if she would be able to have an explore around the issue. She was able to put the project on the machine I had previously used and then record the work. This made such a huge difference to the finish of the piece, replacing the clips with higher quality versions. I was even able to layer a few of the clips to crete more interesting compositions.
Polish And Finish
After replacing and editing slips I added a new soundtrack, and got the rhythm right to match the beat of the music. Finally, I added the title sequences and credits to finish.
This project has been a long, tough journey. We’ve had so many unknowns and so many challenges along the way. It has also been one of the most fulfilling and exciting projects that I’ve ever worked on. I am delighted to say that I feel that I captured this excitement in this video, and I can not wait to show this at the degree show.
In addition to a Making Of Video, we wanted to create a teaser trailer to give people a taste of what is to come, and encourage people to view it at the end of year show. After collecting the footage from Rachel and creating numerous screen recordings from our work, I was able to begin working on this.
The goal of this piece was to showcase our work, not necessarily showcase the process or the team. For this reason, I kept the edit short and snappy with hints of all the areas of our piece through it.
Bloom by Odesza sets the pacing of the piece, and adds to an energetic and exciting edit. View the final cut below:
We got our first batch of prints back and the colours were not exactly as we had expected. You can see in the image below that the colour on the first icon on the right panel is hard to see. We tweaked these a little to make them more readable and sent them off again.
Consistency Is Key – Personal Panels
We all worked under the same template for the personal panel development. Fiona had designed a few layout options for us to populate with our own personal work; this allowed us to add a personal touch to the work but maintain the consistent theme we have been developing for the overall show display.
Research into exhibitions and displays shaped our decisions to keep a clean, simple and minimalist style for Portrait Of The Artist’s degree show layout. In exhibitions, I have learnt that less is more, and it is actually easier to take in more information, when there is less presented. Speaking about exhibition text, Trench writes,
These word limits don’t restrict the amount of information that most visitors absorb. Instead, they increase it. In a gallery or exhibition, less really is more.
She goes on to highlight that there is a real difference between the complexity of information that can be gained through an exhibition and that which is suitable for a book. For this reason, we chose to limit the amount of content going on our boards and space.
For my personal panel, I choose to highlight my core roles in the project, which overlap into my strengths as a designer: storytelling, art and design, and filmmaking. Using the template below, I filled in images to represent the copy I was adding into the space.
I took my time to develop the copywriting to accompany the images in the piece. I wanted to truly reflect my key roles and represent myself in a clear way.
In the storytelling section, I wanted to reflect my role as a writer and storyteller on this project, which came through on multiple levels of the production process. My role was not just writing content for the gallery, it was also in forming the story of the interview section through my editing ability, as well as writing and editing the final making of video.
This section was to reflect my role as a key asset creator along with Fiona. Together we dreamed up, designed, created and refined anything that you can see in the POTA scenes.
I felt that it was important to communicate how my filmmaking skills and technical ability overlapped in this project. We were working with a Beta version of the DepthKit, which brought massive technical challenges. Persisting with this technology was a major part of the process. I spent weeks figuring out how to get this to work, and days after figuring out the variables that allowed us to get the best shots in DepthKit Capture.
We made sure all our panels matched and lined up together and then sent them off to be printed.
For my images, I was finding it hard to get a good screen grab of the scenes so I took them into Photoshop and edited a few scenes together to show the artwork in its best light.
They all turned out really well – we will be setting them up on the next couple of days.
Trench, L. (2013) Gallery Text At The V&A. London: V&A Museum. Available from: https://www.vam.ac.uk/__data/assets/pdf_file/0009/238077/Gallery-Text-at-the-V-and-A-Ten-Point-Guide-Aug-2013.pdf [Accessed 08/05/2016].
After helping Fiona set up the general display of the team layout, she began to put together some designs for the piece in the formats that we had discussed. I loved how the stripped back layout was working, and the ideas were really coming together!
Copywriting For The Layout
Once we were happier with the layout, Fiona then a started to write some of the content of the piece. There were a number of elements that we were not too sure about, so I sat down with Fiona and together, we bashed out some new iterations on the copywriting.
Having done a large portion of the writing for this piece in the explorable gallery and in the video script, I had an idea of the tone that works well for this type of display. My research into the V&A gallery writing (Trench 2013) also came into practice, as we thought about how to create the shortest and simplest sentences, to communicate our ideas.
Below are a range of our iterations, as we wrestled with words and phrases to get the right ideas across.
Finalised Project Boards
Our main challenges was in the simplification of complex systems and processes; trying to find a way to explain to the general public about the technology we were using in one line is not easy! We managed to come round to what I feel is a real strong piece of work, that both highlights the best of Portrait Of The Artist, but also comments on the medium of VR in general.
Trench, L. (2013) Gallery Text At The V&A. London: V&A Museum. Available from: https://www.vam.ac.uk/__data/assets/pdf_file/0009/238077/Gallery-Text-at-the-V-and-A-Ten-Point-Guide-Aug-2013.pdf [Accessed 08/05/2016].