The Voices of VR Podcast
Hello
My name is Kent Bye, and welcome to the Voices of VR podcast. So this is the last of a brief series looking at dreams, dream logic, and symbolic storytelling language of virtual reality. Today’s interview is with Punchdrunk’s Kath Dougan. She’s an art director there, and there’s a VR experience that Punchdrunk did in collaboration with Samsung.
So for people who don’t know, Punchdrunk does the immersive theater show of Sleep No More, which opened up in 2011. Punchdrunk’s been around for about 20 years now, so they’ve been exploring immersive theater, putting the audience in the center of the experience. And so they collaborated with Samsung to create this VR experience. Samsung originally had it at Cannes back in 2016, it was in Montreal for a while at the PHI Gallery, and they had it in Venice. It was in a gallery that had a lot of different other stuff that was curated by the PHI Gallery. So we’ll be not really talking too specifically about the experience, but more about the broader experiential design principles. But if it does come through your town in some way, keep an eye out for Believe in Your Eyes by Punchdrunk.
Exploring Experiential Design
So we’ll be exploring the experiential design process, this evolution of immersive storytelling and virtual reality with Punchdrunk, and exploring different aspects of dream, dream logic, and the spatial storytelling language of immersive experiences. So that’s what we’re covering on today’s episode of The Voices of VR podcast.
So this interview with Kath happened during the Venice International Film Festival. It was not at the Venice Festival; the Venice Festival is happening mostly on the VR island off of Lido. This was actually in Venice. The PHI Gallery had a whole location where they had a number of different experiences by contemporary artists that are pretty well-known, as well as this piece by Punchdrunk called Believe in Your Eyes. And I did this interview with Kath on Saturday, August 31st, 2019. So with that, let’s go ahead and dive right in.
Kath Dougan’s Background
My name’s Kath Duggan, and I am an associate director at Punchdrunk, which is a theater company based in London. And maybe it’s good to tell you about the theater company, actually, then we can put the VR in context to that. So we have been going about 20 years now. So it’s a while, and it was driven by the desire to put the audience’s experience at the center of a theatrical experience. And rather than have them sit back passively observing something, to be up on their feet close to the action. And to have a more visceral experience rather than sitting back and letting the brain process what they see.
Seeing the kind of work that’s been happening over the last 20 years, it’s starting to shift slightly now, but a lot of it has been what we call our masked shows. So we take over a building, an empty building, and transform the inside with like a cinematic level of detail and transform it into another world— but a realistic one— so touch real. So if there’s a table in there, you can touch it. If we’re transforming it to 1950s America, you really feel like you’re in 1950s America. And brilliant performers inhabit that world, and the audience wear masks and are free to roam and free to feel.
Immersive Experiences
So what it’s like if a performer, a character, kind of whisks past you suddenly? How does that feel? How does it feel to sit in the corner and observe this world pass you by? How does it feel to rifle through a cabinet that you’re looking at, the diaries of one of these characters? So yeah, they’ve been the masked shows, and then more recently we’ve been looking at ways to blend the reality and fiction and bleed them together. So if you were putting that world out into the real world, and those rooms were scattered across a city, how could you not know the difference between a performer and a member of the public? So how suddenly can your perception of the world around you shift, and you not know what’s fiction?
The Evolution of Punchdrunk
So that’s been the way it’s been driving a little recently, and with the use of new technologies, experimenting with new technologies to try and find that point where fiction and reality blur. Yeah, and I first saw Sleep No More on October 14, 2011. And so happened to be in New York City, and it really completely blew my mind. In fact, I saw it, and then I went home and stayed up all night writing about my experience because I had to find the language to try to describe what I just experienced. And I had noticed that people were going through it, and they were like, “Oh, this is impossible to describe. You just have to go see it.” So I feel like there’s a lot of word-of-mouth and people really saying, “Well, I can’t describe what I experienced, but you have to go experience it yourself.” Then we can have this similar point to be able to connect about what our experience was in that.
Influences Prior to Punchdrunk
So maybe you could catch me up briefly into like the evolution of that. And maybe even like what came before— like influences that were kind of feeding in before Punchdrunk started doing it. I’m just trying to get a sense of the history, or at least the evolution of what we have today with immersive theater, which has really taken off since a lot of the work that Punchdrunk has done.
Yeah, I mean, oh, placing that in context. Yeah, because, of course, we weren’t the first to do that. It was, you know, there’s a whole world of theater, of happenings, of events that people would be swept up within and things that would happen within the real world. Art exhibits that you didn’t expect you would come across, you would stumble across and would happen. So I feel like I can only talk about it from our perspective and how we’ve shifted through that.
Felix Barrett, who’s our artistic director, so he was at university at Exeter, and it was there that he started to kind of try to pinpoint how to engage an audience, how to startle them, how to take them along the side of a motorway service station and leave them there and see what would happen and then get the next bit of performance in. My background is in contemporary dance, and I started with them as a performer for a show called Mask of the Red Death down at Battersea Arts Centre in London. And a lot of the language within the show is a physical language rather than spoken text. So I joined the company in 2007, and I continued with them performing for a while and then started to do workshops and master classes and kind of the learnings and the techniques of performance that we found within Punchdrunk doing that within an educational setting.
Shifting to Direction
And I worked a lot with what was then Punchdrunk Enrichment and lead projects within schools—immersive experiences within schools that would pop up that they wouldn’t know was theater and would kind of blow their tiny little minds. And then I kind of started to make the shift more onto the directing side of things, working as a resident director on the show and starting to create the more we kind of had partnerships with different brands at the time, and it was more of the experimentation side of things that we were doing. So it was the work with technology.
So we were doing a project called Silverpoint that was a collaboration with Absolut Vodka to try and find a way to make a mobile app game become real around the city. And by playing a simple game of a saccharine, almost like Candy Crush kind of game, and collecting points and then narrative bleeding out of that and then those characters coming off out into the real world for live performance interaction. So I’ve just kind of progressed with Punchdrunk in that way, starting to develop the ideas for shows and then directing them and bringing them to life with a team of other people.
Documenting Experiences
Yeah, and I know that recently Punchdrunk had released like a whole encyclopedia. Were you involved with that project at all? I was involved with that project only in being asked some questions about my time with Punchdrunk and how that was and on an anecdotal level of things that I remember. But it’s so great to read through things and read other people’s experiences of it all, the performers’ experiences of it especially, because it’s people that I’ve worked with and just looking at their memories from those shows and those times. Yeah, really enjoyed reading it.
Yeah, because it seems like these performances, these VR experiences, can be so ephemeral. Sometimes they’ll show up and go away. And so just to kind of help document the oral history and evolution. And so I guess some of the big questions that are facing both VR and immersive theater is how you combine interactivity and agency with a story. And with VR, it’s particularly challenging because you can’t really locomote around in a way that you can, like, in Sleep No More. So I’m just curious how you think about this narrative and interactivity and sort of the inherent tension that goes between those two.
Creating the VR Piece
I think it was particularly interesting with creating the VR piece because it’s almost thinking about what the restraints of that medium are to then be able to create something within it rather than thinking about we want to make a piece of theater and shoehorning in the advances in technology, but really looking at if we’re making a piece of VR, what does that offer us? And what are the parameters that that gives us to create within?
So one of the first things when we were creating the VR piece was how the act of putting the headset on actually, it’s not necessarily about what you see instantly, it’s about the fact that you’re not seeing everything else, that you’re no longer in the world around you. Rather than focusing on what the film will be, what the world that they see is, it’s using the framework that that platform provides.
The Immersive Theater Experience
Yeah, when I was at the immersive design summit and talking to Colin Nightingale, his comment was that doing immersive theater, you know, you have a certain bar that you have in terms of an experience. And that, I think initially, it felt like Punchdrunk was a little resistant to jump in and do a VR experience because, you know, this piece was released in 2016, and VR was sort of on the scene by 2013. So you have 2 or 3 years of development and experimentation before the first project was created. So maybe you could talk a bit about that time period from what you had already been doing from Punchdrunk opening up in New York City 2011 and then this whole new technology coming up in 2013 and then how this project actually came about.
Yeah, well, it came about we had a two-year partnership with Samsung America. And I think it was through these collaborations that, well, firstly actually they showed us the technology and asked us to create something within that field. So that was great, but it also provided us with the means to do it and the reason to look at it, I suppose. And I think just before that, it was those brand collaborations that I think did encourage us to look at the technology because it can be expensive ways to create theatrical experiences if you’re using technology. And so I think as a theater company when you’re looking at the economics of creating work, it’s not necessarily the natural way to go because of how much these things cost to create or can cost to create, which I think is shifting nicely to make it cheaper.
Cannes and Beyond
But so we created Samsung’s piece that was going to be in Cannes for the Lions Festival in 2016, and they asked us to create a piece of VR for a house that they were going to host lots of people in and lots of different experiences in. So they asked us to create something for that specifically. So I think it was about a six-week turnaround from being asked to do it to the event being on and live. And then since then, this piece has kind of traveled to different people who have wanted it on. We’ve been, Samsung took it to New York, we took it to Art Basel in Miami, we took it to several different places, and then the PHI Centre in Montreal had it on there last year or the year before and the brilliant team there have brought it to Venice this year to bring it back to life again, which is wonderful for us to have it back up and on again.
Yeah, so that’s the journey of it, and actually, we did one version originally, and then we kind of reshot it and transformed it for something that could be seen by many more people within a day. And so that same version is the one that’s been around, and it’s a piece that it’s a mixture of live action film and physical performer interaction. So we come and find a local cast in the area and work with them to rebuild the show.
Navigating the World of Immersive Experiences
Yeah, and so I think there’s this challenge I’ve found with, you know, I know Noah Nelson from No Proscenium. He covers immersive theater and how I cover these VR experiences. With VR, it’s a lot easier because a lot of the VR can get out there and people can see it. So I would usually say, “Go see this experience,” and then we’ll dive into the spoilers, but there’s other experiences where I kind of have to keep my cards close to my chest as to what my actual experience was just because it could potentially ruin it for other people. And so I feel like when I was reading about Sleep No More back in 2011, I was curious enough to see that some people felt like on their second experience was better. And so that as well, maybe if I do these things, I can get a really rich experience the first time, but then I’m robbed of that sort of mystery, you know? So I would never know what my experience would have been had I not known that. And I don’t regret it, but at the same time, I feel like this is like a personal choice in this immersive world. The challenge of marketing to get the word out about something like this is good, experience it versus, like, well, don’t give out too much information as to ruin certain aspects.
Finding the Balance
And so how does Punchdrunk navigate this immersive world and trying to find that threshold of secrecy? I think it’s a really tricky line to balance. I mean, talking about this VR piece now, there is the frustration of not just being able to go and then describe the experience and then being able to unpick it with you and talk about the details of it all. So it can be a frustrating thing from within. And always that weird thing of how much information you can give somebody and not because you don’t want to give— you don’t want to, there’s a tricky thing recently with a show we made with Cabey Roy where you kind of can’t tell everyone what happens within, but you have to be sure that they’re ready or you’ve given warnings of things that might happen. But by giving warning, you’re raising expectation that we really don’t want to raise, and also these things might not impact these people in this way.
So you can say something as like, you don’t want to say something would be— you don’t want to say don’t enter this experience if you have a heart condition because you imply that it will be terrifying. But at the same time, we have to be cautious and warn people in the right way. So it’s a really tricky balance to tell people what the experience is without giving away the spoilers. But like you say, it’s the word-of-mouth that really helps us. It’s the individual audience experience that’s at the center of all of our projects that that individual audience will tell another individual potential audience, and that’s how the word of the show can spread through that.
The Journey Through Immersive Experiences
When I see what’s happening in this immersive space, I’ve been covering it since May 19, 2014. But my origin point, I really put in that October of 2011 where I had saw all these people who had had the experience, and they’re like, “I cannot describe this in words. Just go see it.” And my reaction was that’s not good enough. And I was like, I’m going to try to find a way to find the words. And oddly enough, as I’ve been doing the Voices of VR podcast and talking to creators, this happens to be still a pretty huge open problem, which is like how to have a direct phenomenological experience and then to be able to describe what your personal experience is, but also the experiential design components of what we’re able to actually modulate your consciousness in a consistent way. Like a consistent experience across many different people.
So it feels like as we go through these immersive theater experiences, there’s like a language that’s being cultivated that’s almost like this embodied poetry. Like you have a metaphor that, oh, it’s like Sleep No More. And then you understand, oh, it’s like all these different elements that you understand once you see it, and it’s like movies got to that point where it’s like, oh, this movie is going to be like this movie plus this movie, and then that becomes a shorthand. So you have these references and symbols that you can get through these embodied experiences. But still at the same time, just within the context of an individual experience, I think it’s still a challenge to essentially break down all the different components of the human experience to be able to talk about what your experience was. So it feels like there’s this language that’s being cultivated.
Cultivating Language Within the Company
As creators, I’m imagining you have to cultivate your own language within your company. So I’m just curious to hear a little bit more about that process of talking about phenomenological experiences and experiential design.
Yeah, I think it’s actually can be quite prohibitive for an audience member coming in when they already think that there’s like a word and they understand that word in a particular way. So like immersive and like what is an immersive experience? And it being such a broad category now, that everything is put within that, it kind of drags an audience towards— well, I think at least it kind of drags an audience towards thinking it’s going to be a certain thing before they even experience it. And so the word itself is dictating how they feel about an experience.
And how we talk about it a lot internally is through filmic language. The shows, it’s a lot about close-up, wide shot. Often if we think about an audience’s journey, it’s almost like a tracking shot through space and how the performers relate to that. How we can encourage them at this particular point to have a wide shot and how we can encourage them to see something in close-up, see the details. So we do talk a lot about filmic references. I mean, we’re always discussing from the audience perspective, it’s always about them and what they see and how they feel, and quite often, if we’re working on a new project, it’ll be about what an individual audience member might see and feel and experience within a space. So it’s often from the one person.
Performers and Their Language
And then in terms of performers within this work, there’s a whole other bubble of language in the way that they communicate together. And of course, it’s people from physical backgrounds and physical practices, so it’s often not well-constructed sentences that they can communicate in. It’s the sounds, it’s the views, the feelings, and the way they’ll use their hands to describe something. It’s how meaty something is and how dreamy, ethereal, and light it might be. And it’s words like that that the performers would use within something.
The Audience’s Perspective
Well, it’s interesting to hear that you think of the audience member as a camera that’s moving through space because in both Unity and in real engine, real-time game engine flavor, as opposed to the 360 video, you’re literally a camera that’s moving through space. So it’s quite a literal translation. But I sort of see at a high level that there’s yang and ying sort of metaphors. So yang being ways that you can express your agency, make choices, take action, kind of like a video game. And then the yen is more of the passive receptive, a modulation of building and releasing narrative tension or with music. And also, there’s a certain level of embodiment that you’re receiving through your sensory experience.
And so because in Sleep No More you’re a ghost and you have no real agency that impacts the story, you have the freedom to be able to have a very controlled narrative that isn’t being influenced by the audience members in any sort of interactive way. I feel like that’s a good first step. And where we’re going feels like the live-action role play where people have the ability to speak and be a character, and the whole Westworld version of where things are going. I think there’s a lot of competencies that the audience has to cultivate to even get to that point, to really understand the dimensions of narrative, the narrative tension. But I see it as this kind of tension between like to what degree does the audience member, are they a ghost or are they a character? And also how much that you as a person being there is impacting.
Combining Worlds
So that’s at least how I start to think about it in ways of in VR, seeing how still trying to combine these two realms of like the video game world with the film world as well, was like putting a sense of embodiment and the sort of more contemplative embodied experience into the whole mix as well.
Yeah, it’s interesting because lately, even thinking about who the audience are at the center of something. So like you say in Sleep No More, for example, they are ghost-like creatures that inhabit the space and they observe, but they don’t have an impact upon the story. And the difference between those audience being observers to other shows that we’ve done where the audience don’t have masks and they are placed as the lead character within that journey and people are addressing them as if they are that lead character, and they’re led through the journey as if they are not that they have an impact on the way that things will turn out, but they even then feel like they are more active within the narrative because they have been addressed. And I think because our work is a lot about the detail and the controlling of the environment and controlling what the audience will see in the stuff so far, I think that’s the reason to not have them impacting the show.
Complete Freedom vs. Control
So like it’s been quite within the piece and how you could make them feel as if they had complete freedom, but eventually by dictating where they look and what happens that you’re kind of shutting that field of vision down so that we can control exactly where they’re looking at the time, although they feel as if there is the complete freedom. So it’s through choice that we do that so that we can control every moment that they are seeing rather than let them impact upon that, you know.
First Impressions of VR
Yeah, I’m wondering if you could talk a bit about your first experiences with VR as a medium.
Yeah, I’d watched very little. So I made this show with another associate director at Punchdrunk, Hector The Harkness, and both of us actually had had very little experience of watching VR, of experiencing or watching it. And a lot of the pieces we’d seen were quite fantastical things like doing the impossible almost, you know, flying through the sky, soaring through the sky like you’re a bird and you’re looking down on the world or well, it’s not impossible, but like being on a roller coaster. And I think that was one of the things that kind of drove us towards trying to do something that was more realistic and more intimate.
And like I said earlier, it was a lot about noticing the things that for us felt wrong. About like felt like the things that were pulling me back to reality when I was meant to be immersed in a VR experience. So the fact that I would look down and it’s a different body really, I found it quite disturbing somehow that it doesn’t look like me or that I would perhaps be watching something that I didn’t realize was made to be viewed standing. And I was sat down, and I’ d move my legs and they’d go through a chest of drawers, and that I found like not being solid within an alternate world kind of, I found quite difficult to make it feel like I was really there or inhabiting this world of VR.
Vulnerability in VR
Yeah, so it was weirdly these, and I think it was also the strange thing of putting it on. I’m so used to working in live theater and being present with people in the room that you’re working with, either be it on stage or the audience able to see them. And it was quite a strange and actually quite vulnerable sensation of putting the VR mask on and feeling exposed to the room in a different way that I felt like, you know, you kind of feel like people could easily prod you or be laughing at you or, you know, a whole other world going on in reality in the room that you can’t see. And it felt quite— the act of putting the VR headset on felt made us feel quite both of us quite vulnerable in the space.
The Nature of Immersive Experiences
And I think that was one of the key driving factors for us was that feeling of the audience in a space with a headset on, how that makes you feel. And I think when you’re doing VR, you get really used to that. You get really used to putting it on and being in another world and everybody else going about their business around you, and it feels quite normal. But I think when you do and Sundance and South by Southwest, there’s a number of themes that I see emerging in terms of like trying to give, like, metaphors for what immersive entertainment is, whether it’s immersive theater or virtual augmented reality experiences. And people compare it to memories or a psychedelic journey or a dream or like an initiatory ritual, a rite of passage.
Dream Logic in Storytelling
So you have in film, we have a certain language that’s developed, which is a cinematic visual storytelling language that’s taken over 100 years to develop. But I think of people like Carl Jung or Freud who are looking at the interpretation of dreams and seeing how there’s a whole certain level of dream logic that’s used that doesn’t necessarily fall under like rational thinking but is something that’s more akin to a dream. And something like Bodiless here at Venice this year explicitly uses certain aspects of dreamlike metaphoric mythological stories. The key director, Celine Tricart, is a lucid dreamer who uses dream logic as a metaphor. Jordan Peele’s doing a lot of metaphors in Us and Get Out, where it’s a lot of symbolism.
So I feel like there’s a sort of movement towards dream logic and the language of dreams and memories that is this overlap to immersive theater, and there’s this challenge between a personal symbol and a universal symbol and trying to translate a story into something that goes beyond just the personal that people can kind of project whatever they want onto it but not be so arcane as to not understand what it might mean. And there’s a mystery there, like a dream might be mysterious and may stick with you. But I’m just curious to hear a bit about how Punchdrunk thinks about this sort of grammar and storytelling language of spatial storytelling in immersive entertainment.
Nonlinear Experiences
Yeah, I think I can understand why people latch onto the dream landscape somehow because it really allows a fit with a nonlinear experience. You know, we all travel through life in our own linear journey, and I think it’s for a lot of people, dreams, I suppose, are the only experience that feel kind of cut up, feel fragmented, and feel quite disparate, but also it feels quite normal to them. They talk about it in a normal way. It’s not unusual.
They—I mean, of course, the way they describe the dreams and the that happen within the dreams really are unusual, but they aren’t necessarily perturbed by that. Whereas some people coming into an immersive experience, if it’s not linear, can feel perturbed. And I think it fits quite well to be discussing these things as dreamlike. Things that you can’t quite grasp. And I think for me, it kind of feels like it might help some audience members somehow because they’re not hunting for the logic and they’re not hunting for complete clarity of narrative and piecing everything together to gain a complete picture. If they understand that it’s their own perception of these things and how they piece these things together, then it can give them a bit more freedom.
That being said, it’s the difficult thing that we also really want to make them believe it as reality somehow because it then has a much higher impact. So if there’s a way to allow them to be happy with the fragmented journey and still be in reality, then it’d be a win.
Concepts of Time in Immersive Experiences
Yeah, it makes me think of how the Greeks had two words for time—Kronos time and Kairos time. And the Kronos time is that linear time. It’s like the unending fatedness of Saturn eating its children. And it’s like this fate that’s bestowed upon you that you can’t get out of. The Kronos is much more the quantification of time—like, time is it? Well, it’s 10:59. Or the Kairos time is much more the quality of the moment. So it’s the right moment to act, and that’s more nonlinear and cyclical.
So it’s like thinking as patterns of cycles that are repeating, and so you have returning to moments that are connecting you to memories and associations, and it’s also like when you’re on vacation, and you don’t have a plan and you just sort of go out there and you have these series of synchronicities where your internal state gets matched to the external reality of what you’re experiencing. And I feel like a lot of what I wrote about when I saw Sleep No More back in 2011, and as I said, it felt like this initiatory ritual that was like trying to cultivate this synchronicity of being able to make choices but to have things happen that you read deeper meaning into, and you have this deeper flow state of like feeling like you’re in the flow of the quality of the moment. So I feel like there’s this sort of Kronos and Kairos—the linearity and the nonlinearity that I feel like at the heart of it is almost like cultivating our skills to be able to really get into those Kairos moments.
Re-experiencing Reality
And it’s really difficult to be open to those things and how you’re feeling that day, how your day’s gone means you’re going to enter an experience with a very different mindset from one day to the next. You’re going to be open to seeing how the journey goes and on another day where you’re feeling more uptight, maybe you’re not going to be open to those experiences. But yeah, it’s a really nice way of seeing it with those two concepts of time. It seems like people’s perception of time changes when they’re in these experiences. They either think it’s a lot longer or shorter than it actually is.
Yeah, totally. And also, it feels like there’s something about taking moments of experiences out into the real world that can really shift that concept of time as well and shift the perception of how the real world is moving around you. And weirdly, when these interactions happen within the real world, it’s almost like the real world around you becomes more dreamlike. So it kind of somehow flips from within the show, and you start to look at people and the way they move, and I think time slows down slightly because you start to look with more clarity.
Experience and Time Dilation
Yeah, it feels like that’s a big part of the immersive theater experience, is sort of the modulation of your experience of time. And that’s a big consistent thing. I’ve been doing a lot of VR, and sometimes people will have an experience in VR where they play for 12 hours and they think it would have been like 2 hours. So you have this time dilation effect that happens. And so kind of studying what is at the heart of that, but I feel like that seems to be a pretty consistent experience of the immersive theater world.
Yeah, it is. And it’s that thing of like having eye contact with someone for a moment, you can’t tell how long that moment is. And maybe after you’ve seen a show for three hours, you’ve been in there for three hours, and the moment that feels the longest is that one blink where you had eye contact with one performer. So it really can stretch and play with time in that way. And I don’t know if that is—I need to experience some things and think about that more if that is a thing that is consistent with immersive theater or not.
Connection with Novel Experiences
Yeah, I need to think that if that is a definite connection. To me, I think the connection is if it’s a novel experience, you sort of enter into—we have category schemas where we have a predictive coding model of neuroscience where we have a repository of things where we project these models onto the world. And if it goes beyond our model then we have to do this error correction code in our brain which then releases endorphins. But it also, I think there’s a connection there. If you have a lot of novel experiences that you’ve never had before, then your brain is doing extra work and then it sort of loses track of time because it’s trying to create new models in its mind. And so you’re kind of putting people in this state of awe and wonder in a perpetual way that they can’t deny the reality of this experience, and they have to find some way to make sense of it. So I think through that process, it invokes all these senses of awe and wonder and time dilation, and yeah, just kind of modulating your experience of time.
Technological Advances in Theater
Yeah, I mean, for sure that’s true. And I suppose it depends as experiences become more and more immersive experiences and more commonplace within theater. It’s not always a novel experience for people. And I think that’s why time might not have that, but for sure, it’s the continual new experience of new things and new processes that time shifts dramatically. Your brain is working to understand, and oh, your body is— I feel like it’s like a visceral process somehow that you allow your body to understand things or they are dealing with things. Of course, it’s your brain really working away, but it kind of somehow feels like your whole body is processing these moments.
Open Problems in Immersive Theater
And so for you, what are some of the either biggest open questions you’re trying to answer or open problems you’re trying to solve?
One problem or thing like to solve is that as we use these technologies, how we can continue bleeding them, like cross-fertilizing them with theater. So that if we’re using a mobile phone, actually, how can we not get drawn into the mobile phone? But how can we force audiences’ eyes open to the real world as they’re doing that? So I suppose it’s as we keep progressing with the use of technology, I think it’s continually playing with these technological advances that we cannot get drawn into that and remember the skills that we have and how you can keep looking at the technology afresh by using the theatrical tools and skills we have rather than just getting drawn into what the technology is currently used for and does.
The Potential of Immersive Experiences
And finally, what do you think the ultimate potential of immersive experiences might be and what they might be able to enable?
I still kind of really feel like it’s allowing the audience to see the world differently, to be able to step out of an experience and look at the world around them and the people around them through different eyes even for a moment because I think by looking at people and the world around you through a different lens even for a moment, you’ll have a connection with that world and behave differently within it. So I think for me, it’s all about how an audience through those experiences can look at the world around them differently. And I think the potential of that—in the same way that I don’t think it’s exclusive to immersive experiences—but I think that they can really push people to do it in a quite sudden way somehow.
But I think by the experience of looking at the world around you differently will allow people to see what’s really happening in the world and interact with it differently. I know that sounds really vague and wide, but I think it can allow for different personal human interactions after having an immersive experience by seeing the world differently.
Final Thoughts
Yeah, it’s a common theme in VR to say like it’s blazing new neural pathways in, and so once you take the headset off, then you’re perceiving the world in a new way. So I think it’s very similar. Is there anything else that’s left inside that you’d like to say to the immersive community?
I’m just very curious to see and watch where it goes, especially VR and not knowing an awful lot about the world. I’m really curious to see what happens when artists that don’t know about VR and haven’t experienced a lot of it interact with the technology for the first time to create new things.
Thank You
Awesome. Great. Well, thank you so much for joining me today on the podcast. So thank you!
Yeah, pleasure. So that was Kath Duggan, she’s associate director of Punchdrunk, and we were talking about Believe Your Eyes. So I have a number of different takeaways about this interview is that, first of all, well, this was a very amazing experience. I wish I could just like tell you everything that happened, but I think it’s just better for you to try to see it yourself. But for me, I just felt like this state of awe and wonder in a number of different ways.
And you know, it’s just interesting to talk about the design process for Kath, because coming as a dancer working with an immersive theater troupe, they’re really trying to break down this fourth wall of this separation between the audience and the actual experiences. Starting with a lot of the immersive theater type of experiences of Sleep No More where you are a ghost with no impact and you’re in these masks with these masked shows walking around seeing these different immersive theater actors kind of run by you. It sounds like they use filmic language to be able to describe different aspects, and so you get like the wide shot to kind of establish new context and then the medium shot and close-up where the close-up is you really just digging in and being able to open up different drawers and read different letters, and for you to really maybe see an interaction between two characters and use those types of filmic metaphors to be able to describe the audience journey because they’re really thinking of the audience as this camera that’s kind of moving through this space.
Now I think some of the other experiments that they’ve been doing with some of the other immersive theater shows where they’ve started to actually put you out into the world, and then experiences that are really designed for one person, you start to have these interactions with people within the public where sounds like this vodka app where you’re walking around the locations of this place, but then these people are coming up and interacting with you and you don’t quite know who’s an actor and who’s not, and they start to blur the lines of that reality. And so we’re talking about these dreams and dream logic and how you kind of entered into this kind of altered state of paying attention. And usually when you’re walking around, you have like these category schemas, you know what to expect. And then in this type of blurring of a line of what is reality, you start to have this experience of you not quite knowing what is real. And it kind of starts to make you pay attention more is what she’s accounting here, which is you starting to just notice things you didn’t notice before. And she wonders if that’s kind of a universal experience within immersive theater.
As you go and have all these different novel experiences, then you start to dip into this liminal space where you don’t know quite what to expect and you feel like you’re walking through a dream. And I think they really like those dream metaphors, especially like my experience of Sleep No More very much felt like I was walking through a surrealistic dream where there’s all this embodied interactions and it has deep metaphoric meaning that you can get a sense of what it means. But it also maybe you have like a symbolic action that’s happening in front of you, and it’s actually really reflecting what you’re feeling inside. And you have this moment of alignment of what you’re feeling inside and what’s happening outside of you.
And that the deeper you get into these deeper flow states of immersive theater, then you have those experiences of those synchronicities and those different qualities of those experiences of time. And then she was also reporting that people were experiencing the opposite of the effect, which is that you’re out in the real world and when you start to have those different types of interactions that are perverting your expectations, then it also makes you look at the entire reality as if we’re living in a dream, which I think is an exercise that you can go through and start to interpret everything that you’re saying as a dream. And then really start to question like is your waking life the real life or is your dream life the real life and then your waking life is just the dream, which I think is fascinating to think about what’s more real—the dream or the reality.
And I think Punchdrunk is at that intersection of trying to really blur the lines and really make you question what’s more real. As I was talking to Celine Tricart, she said the brain is actually kind of experiencing things in a way within, you know, for her as a lucid dreamer where she can’t actually physically tell the difference as to what’s real or what not, and she has to do all these different types of reality checks. And so maybe that’s the process of immersive theater as we move forward is we’re all going to be kind of walking through the world as if it was a dream and then to start to have to do these reality checks to be able to check are we dreaming or not.
Future of Storytelling
So the final thought is just how in the future a lot of what you go through in the immersive theater world is on the rails. You can’t impact the unfolding of the story. Logistically, it’s very difficult to think about the branching narratives or the improv or what kind of structure you’d actually have in order to allow you to be a full character and you were getting into the realm of live-action role play and the cultivation of the skills of the audience. Know how they kind of fit as a character and who that character is and how that’s defined. And it’s much more easy logistically to be able to experiment with these live immersive theater shows where you have no impact, and they’re starting to play with different ways in which that you’re being addressed as an individual versus you being an anonymous masked ghost character where you’re just, it doesn’t really necessarily matter that you’re there or not; they’re pretending that you’re not there.
So the action is going to unfold whether or not you are witnessing or not, and that some of the directions that they’re going with some of this more bespoke individualistic way that you’re actually the center of the character addressed as an individual. And even though you may not be able to impact the outcome of the story, as we move forward in the technology, I think it’s actually going to be easier for artificial intelligence and other ways within the simulation of a game engine to be able to allow people to experiment a lot more. And maybe it’ll be this dialectic process to kind of figure out what those narrative structures are and then to find ways to feed that back into like a live immersive theater context.
You can do a lot with a small scale with just a few people and be able to track it. But once you get into like dozens or hundreds of people, then you know how could you reasonably expect that the audience is going to somehow shift the overall outcome of the narrative, especially when you have that asymmetry of the number of audience members versus the number of characters. So I think you could start to maybe play around with that a little bit within immersive simulations of virtual reality, but I think that’s kind of like the bleeding edge for how could you actually have a group participatory process where the agency of either an individual or a group could aggregate to be able to actually change the outcome of what actually happens within a narrative. That’s something that’s really on the bleeding edge. Maybe we’ll see that much more into the realm of artificial intelligence and different models that are doing that. And there are some other AI experiments that are happening both at Tribeca and South by Southwest and here at Venice. We’re starting to experiment with different levels of artificial intelligence. But that’s a whole other separate series to kind of dive into what’s happening in the realm of AI with this interface with VR.
Conclusion
So that’s all that I have for today, and thanks for joining me on this series of looking at dreams, dream logic, and the symbolic spatial storytelling language of virtual reality. This was a fun little short series to kind of dive into and themes that I think came up between what I was seeing at Sundance and South by Southwest and Tribeca and Venice, especially at Tribeca and Venice with these specific experiences that were very explicitly using different elements of dream logic or dreamlike qualities or you know symbolism within spatial storytelling and playing with that in different ways. So I will be heading off into Aquos Connect 6 and then, you know, some associated events after that. Be going to the SVR meetup that’s happening on Tuesday night and then I’ll be at the indie dev night I think on Thursday night. And then San Jose State University on Friday is having a whole day long virtual reality connect. I’ll be giving an opening keynote there. I come back home and then I fly out to Amsterdam to go to the View Source conference in Amsterdam where I’ll be recording some podcasts with web developers and doing a live recording of looking at the future of WebXR. So I’m excited to be out there next week at AquaSkynect 6. If you see me, track me down. Love to connect and see what you’re up to.
And, yeah, I’ll be back in about a week from now, kind of dive into the content that I capture there this coming week. And if you enjoy the podcast, there’s a couple things you can do just to support the podcast. One is just spread the word, tell your friends. I really rely upon word-of-mouth to help the word out about what I’m doing here at the Voices of VR. And this is a listener-supported podcast, and so I do rely upon donations from Patreon to continue to bring you this coverage. And if you’ve been enjoying these deep dives and series, then really consider becoming a member of the Patreon. I do rely upon donations from listeners like yourself in order to continue to bring you coverage. Just $5 a month is a great amount to give and just allows me a foundation and baseline to be able to continue to do a real-time oral history of the evolution of spatial computing and immersive storytelling. So you can become a member and donate today at patreon.com/wistavr. Thanks for listening.