The exhibition of Klimit's Magic Garden in Brussels has received the support of the Chancellery of the Prime Minister, Belgian Federal Public Service, Foreign Affairs, Foreign trade & Development Cooperation - Austrian Federal Chancellery, Directorate of Arts and Culture, in the framework of: The European Year of Cultural Heritage 2018 & the Austrian Presidency of the Council of the European Union.
Go to interactive wordcloud (you can choose the number of words and see how many times they occur).
Download full transcript in PDF (138.22 kB).
Host: Federica Bressan [Federica]
Guest: Frederick Baker [Frederick]
[Federica]: Welcome to a new episode of Technoculture. I'm your host, Federica Bressan, and today my guest is Dr. Frederick Baker, a film scholar and digital humanities researcher at the Wolfson College and the Centre for Film Studies at the University of Cambridge in the UK. Welcome, Frederick.
[Frederick]: Thank you.[Federica]: Thank you for being on Technoculture. So, you embody different skills. You wear different hats in your life, so I would like to ask you to keep introducing yourself beyond the labels, the keywords that I just gave, to give us an all-round profile, mostly combining your activity as academic scholar and creative artist.
[Frederick]: Well, I don't like squares. I don't like rectangles. So I've always been, after making about 15 years of film, I always got very bored with the image always being within that frame, and in general I've also become very bored with all of our information: the page, the writing, the lists always being in the frame in those rectangles. And I've spent about the last five years or no, actually, ten to fifteen years working on new ways to go beyond the frame, to break the frame, and to come up with new ways of organizing our experience and our vision of the world. And that's a roundabout way to say I suppose I've been doing expanded cinema, and I'm also doing digital humanities where I'm working with spherical forms of storytelling. So I've just invented a form that I call the story spheres, working with Arthur Schnitzler's work, and I've also been working, since about 10 years, over 10 years, with VJs, and I've been very inspired by club culture and club visuals. Although I don't like techno myself, I'm really a great admirer of what the VJs do visually, and I've, to put it intellectually, introduced that into my own praxis.[Federica]: I first learned about you when I was traveling in Austria a few months back. I was in Vienna visiting the Museum for Applied Art (the MAK), and I saw that there was a virtual reality installation on display there. And I'm a bit of a virtual reality freak, meaning that I'm very curious about it, although I don't have much experience with it, so since I think it's fair to say that it is becoming more and more common, we find it in museums but also in train stations, airports. So every time I have a chance, I want to try to have that experience, and so I did at the MAK. And in the few minutes' time that I was inside Klimt's Magic Garden (this was the name of the installation, which is your installation), how I see VR was changed forever. It was so different from any other experience, and I meditated why, once it was over, and I was still processing the beauty and just how my imagination was triggered while being in the virtual world. And so I took note of your name because I had this intuition that you had done something specific there. This was no chance that this installation was so magically beautiful; you made conscious use of some of the parameters, some specific traits of this technology, which is still relatively new. So I wanted to get in touch with you. (And thank you so much for, of course, being on Technoculture, again). To begin with, I would like to ask you to say something about this work. Introduce it for us, and especially for those who have not been fortunate enough like me to experience it firsthand.
[Frederick]: So about over a year ago, I met the director of the MAK, the Museum für angewandte Kunst, the Museum of Applied Arts in Vienna, and I showed him my first virtual reality film, which was called Pitoti Prometheus, and that used prehistoric rock art as the basis of a virtual reality experience. And we got talking, and he said, 'Are you interested in Klimt?' And I said, 'Yes, I know his work.' 'Well, we've got a Klimt.' He said, 'What could you do with that?' Cut a long story short, he gave me a very high-level scan that had been done of the painting, which is known as the cartoon or the sketch for the Stoclet Palais. And it's a piece of art that Klimt called going from expectation to fulfillment, and it depicts a huge tree, like a live tree, with lots of swirling branches. On those trees are birds, and there are flowers in the meadows and lots of plants, and it was meant for a dining room in the richest villa in Belgium that was built many years ago. And on that piece of art are three figures. One is a woman that he called Expectation (had little bells in her hat), another is a knight (very geometric), and the third figure is a couple in an embrace, and he called that Fulfillment. And what I did was that I took that to my colleagues at Leiss Postproduction in Vienna, and particularly to Markus Cermak, who's the 3D graphic artist there, and said, 'What can we do with this?' You know, and I said, 'I'd like to make a new artwork out of it.' So Marcus started by cutting up the pieces within the painting digitally, and I said, 'I want to create a Magic Garden out of this, you know. This is the kind of the logical thing.' And we started placing the trees, the elements, in space so that it would become a VR experience. And that became the first iteration of the Magic Garden, and each time after about another month's work, the director of the MAK, the Museum für angewandte Kunst, came, Christoph Thun-Hohenstein, and had a look at it. And we developed it further and further. As I say, Klimt had his hundredth year of his death in January, and it became the piece that was done to open that year of remembrance, and was with two headsets in the Museum of Applied Art. Now, what you see in Klimt's Magic Garden when you go in is, firstly, the work as you see it today. You're surrounded. That then leaves the ground and you are in a garden, you are in a space. And the figure of Expectation leaves the artwork and leads you into the garden muttering, and you're given then a choice of paths, and you're in a kind of a labyrinthine mountain-like environment which I designed because it's a kind of a physical embodiment of what it's like to expect something. When you're expecting something, you can't see round the corner. You can't see over things. It's difficult. It's labyrinthine. And so we have this kind of area. It's set in a kind of a landscape surrounded by lakes. This is important because Klimt spent his holidays, the Sommerfrische, as it was called, in the Attersee, in this Austrian lake district, and so it's similar to the Attersee, which inspired him, because when Klimt paints wet meadows, it's alpine meadows from the Attersee that he's seen and he means. And out of that area there, once you explore that, there is a space where you come to a castle, which is the figure of the knight, and the knight is always a figure who is, in a sense, a doorman, watchman, and if you trigger that, you... The walls become a bridge. You cross that bridge, and you come into the next, we can say, level. But we come to the next island, which is a huge meadow of fields, and you find your way through that until you come to a golden island, and on that golden island, you will trigger lots of certain things to happen. And that's where you will find fulfillment, literally the figure of the fulfillment, that is composed of its components and literally comes together to create a visual fulfillment. And that, very briefly, is the path through the work that's the drama, that's the, in a sense, the screenplay, the structure of it, but not all people do that, not all people take that path. And they find their own way through the work.[Federica]: Is this the first time you engage with virtual reality?
[Frederick]: Virtual reality. Well, I suppose we all engage with virtual reality every night because we dream, and I think that's the kind of the basic form of virtual reality, but no, I started my first virtual reality film with the headsets in about 2014 when I was working together with someone called Marcel Karnapke who was at the Bauhaus, and we were part of a big EU project. I was there from Cambridge, he was there from the Bauhaus, and we were working with 3D scans of prehistoric material from a valley in what's now Italy in Lombardi called Valcamonica. And out of that material, together with new material that we did in the Bauhaus Virtual Reality Lab, I created a film called Prometheus' Pitoti, which is or Prometheus Pitoti. 'Pitoti' is the local dialect word from Italy for these figures, and Prometheus was the story from Goethe of the Greek god who creates humanity by literally lifting us out of the rock, and since these figures were in the rock and we digitally lifted them out of the rock, it became a nice sort of metaphor. But Prometheus was also a kind of a metaphor or a symbol for what virtual reality is about, in a sense. It's about taking something like rock or clay and then molding it into something that's movable, that's visible. And as I say, that idea goes all back to the Greeks. In fact, it goes back beyond the Greeks. The word 'Adam' means 'from clay', you know, from the New Testament, from the Old Testament, from the Bible, that's how, you know, Adam was created. The Jewish words, Hebrew word adam 'from the clay', and of course, before that or around that time, we have the Babylonians as well. So a lot of these stories have been there, but they didn't have the technology, the technology that we have today, to make this such an illusion. And working with Prometheus and that story with Marcel together, we made a VR film which is 17 minutes long, so very long for that period. This was... We finished the film in 2015, very end of 2015, presented it for the first time here in Brussels, 2016, right at the beginning, in March. And once I'd seen that, look, I kind of got bitten by the bug of what one can do, but I was keen to go beyond VR film and obviously VR experience now with Klimt is really a step up in terms of the digital difference.[Federica]: VR is a technology new enough to still want to experiment with it mainly for the sake of experimentation, just to test its limits. I would like to ask you to talk a little bit about how you learnt about VR and what is possible today, and how you then found a balance between what was possible and just using everything, every effect possible, and actually implementing your idea and just letting this installation go in the direction you wanted it to.
[Frederick]: Well, if I divide that into two, the first part is, I really got interested in what we have to call expanded cinema, i.e., breaking the frame. About over ten years ago when I was making a film with someone called Thomas Kühne about The Third Man in Vienna, and I worked with projectors, and we projected sequences of The Third Man onto the surfaces where they come from, so the sewers on the big wheel, on all of those sorts of things. And I developed this, and I wrote a book about it called The Art of Projectionism, and it was a way of kind of getting pictures to be more interesting, particularly, you know, old films, black-and-white films, and to get more [pattern 00:14:00] into them. I think... I studied archaeology at university, so I've always been interested in kind of digging into things, and I think by projecting one image onto another surface that's not a clean, white, you know, cinema's wall, you suddenly get a dialogue between those two things, between the image and the surface, and it brings out something different (you know, people talk about dialectics and things like that). And so it brings out something different that goes beyond the original image. And ironically, by adding the light, you can kind of excavate into it and dig into it. So that's what I found. And Thomas Kühne, by chance, was also an inventor and a VJ, and he'd invented a system called the VMS, VideoMovingSystem, that's used in clubs. And basically, it's a mirror in front of a projector that's computer-directed that can shoot the image all around. And I once, when I went to a club called the Cafe Donau in Vienna, and all the patrons had gone and we had the system to ourselves, and we put in these image from my Third Man film, and suddenly all around the club, I had Harry Lime running, I had Alida Valli walking and crying, and suddenly I had a real epiphany. I was in a film, and it wasn't up there just on the wall. It was like all around me. I was immersed. That was my immersive story. And I looked at that and I thought, 'I want to make films like this. I don't see this club in terms, it's just, or it's an image and these are visuals, as they're called, to sort of get me in in a certain mood, but these are actually stories we can tell, and what if we can do this as well, with the same level of stringency, that we'd make films with?' And so that started my journey into something I called ambient cinema where we took the VMSes and we went into different locations and I took short film that I'd shot with them with a Bavarian star that I'd done and with an English writer called Bob [Hewis 00:16:07] and cameraman called Christian Mehoffer, and we split this film instead of just being for one projector into lots of channels, but we made it a round experience, and we looked for a place to do it, and the only place we could find in Vienna to do this was the planetarium because it was big and it was round. So we took in eight of these projectors, the four streams, and coordinated it, and this is the difference with movement, so the images weren't just around you, but they moved. Sometimes they all came together and overlapped, they fell apart, they danced, and that was amazing. And I wrote about that in my book Art of Projectionism, and that was kind of ambient cinema. I then, for a big show at the Biennale in Seville, got invited by Peter Weibel to do a piece about the Mediterranean, and I chose to do Homer's Odyssey, but done through the version that Jean-Luc Godard had done called Le Mépris, which is about the Odyssey. And I thought, 'What if we fill in the gaps?' Because in that film, you only have bits of the Odyssey shown. 'Why don't I shoot the rest?' So long story short, we developed a camera that we could shoot in four different directions simultaneously, and we shot a lot of material in Vienna, and it... Whereas you could do... You know, Joyce did the Odyssey in Dublin. He wrote it in Trieste, of course, but he wrote about Dublin in a day walking around, and I thought, 'Why don't we do the big street?' The busiest street in Vienna is called the Gürtel, the belt. 'Let's do a journey along there.' And so I did that, and we did that as an ambient cinema piece for the Biennale, and we had, within the 90-minute, 80 minutes, we had about almost a hundred projector movements, so we created this amazing sort of spectacle, and I'm still really very proud of it. But it was so much work. We were so exhausted, and Gary [unclear 00:18:00] who did the projection, was so amazing, Antonia Adelsberger, who cut it. We were all completely exhausted, and we thought, 'We can't carry on like this. We need a new technology.' And basically, I stopped until 2012, '13, suddenly this stuff coming up about there are these glasses, there's virtual reality, there are these GoPros, they're all this technology that allows you to capture spherical, round images or to create story worlds in which you then put a person inside. And that's when I started again. So that's a long answer, but it's been a kind of a long journey and it's been times where the content has gone forward and then the technology has offered something, we work to the limit of the technology, then we had to stop and wait for the technology to catch up with the ideas. And now this new technology has come along, and I'm really, really enjoying it, being able to do that work.[Federica]: VR is in fact recent enough also to expect, to reasonably expect, a feeling of surprise when you put your headset on, even if it's not the first time you put it on. It's so new. We're definitely not, you're just used to what it can do, that, I would say, regardless of the type of world you are being immersed in, you will always have a surprise moment. And I think that it just goes to your credit that clearly you have not used every possibility of this technology today for cheap effects like to evoke wonder and awe just because of that. And speaking of the experience that the audience has, have you heard from your audience? What do people say? Have you had some feedback?
[Frederick]: Well, the first thing I'd say about content is that when you're creating an experience or a film, it is when you're inside it, you're not an engineer; you're a human being. So you're responding to the light, the colour, the story, the emotion as a human being. Therefore, bit rates, render rates, RAM sizes, gigabytes don't matter. It's not what the experience is. Those all things are delivering the experiences. Now, they're very important, and without really great and talented engineers and technologists, it wouldn't be possible to do that, but there is a key difference between a tech spec demonstration piece and an actual film or an artwork, and that's the point that when you say, 'This is going to be [unclear 00:20:58],' and you really want the public to react, it then has to come generally into the hands of artists who demand this of the technology. Now, there are some people, brilliant people, who are engineers, can program and do all of this and there. I can't. But I have great respect for these people. But I have to ask those questions, and in the end, I have to make those demands. And sometimes, as you rightly say, it's the right decision to actually hold back, that, in a sense, something could be an amazing technical display, but it's actually not right for the piece. So with Klimt's Magic Garden, it's a hundred-year-old piece. It's abstract. It's a garden. It's quite abstract and subtle, and therefore you are not in a disco, you are not in a club, you are not in a horror movie, and you are not in a VR demonstration. You're in a garden. So it's, how can this technology deliver this, but in a magical way and in a different way? And this is about taste, it's about modesty, and it's about making those calls as to what can happen. It actually includes Magic Garden. We pushed it to the absolute limit of what the technology could. Markus Cermak, who was the 3D graphic artist on it, utterly brilliant, I wanted some more things to happen, and he said, 'Fred, if you do that, if we do that, we'll overload the system. I cannot guarantee that it will run reliably.' And I then had to pull back. So I had to make sort of decisions like that, and that's where the technology is important because it will allow us to do more things, but in the end, it's about those artistic choices that you make, and new pieces that I will do will be other sort of choices that I make. In terms of the public, well, it was very gratifying this morning when we had the press launch here in Brussels at the bazaar, and all I heard from everyone coming out was the word, 'Wow.' Now, as far as I know, that isn't Flemish and it isn't French. It just seems to be human being for astonishment, pleasure, exhilaration. Some people I asked and they said, 'Well, sorry, I'm speechless,' and they literally were. And that's very gratifying when that's a positive effect, because my philosophy I call slow VR, like slow food. It's something that you should allow VR to do its work and people to explore in their own time, and not to kind of overwhelm, you know? It's a technology that can be overwhelming. It's immersive. It's extremely powerful a technology, but then it's about, I would say, it's more about kind of seduction in a kind of a slow way, rather than coming all guns blazing. And that seemed to work with these people. And I can say I've seen that now for seven months in Vienna, where it's been showing since its premiere in January, and there's constantly this look of astonishment, and that gives great energy, and I love that. And the other thing that we've done in Vienna is that I thought, 'Okay, well, this is, you know, qualitative response. I've been there, I've seen it as well. That's a response. So what's the quantitative response?' So we've done some questionnaires. We've had over 500 replies from people from all over the world to see who's kind of looking at it, at VR, and what are their responses? And we gave people a five-star rating, from 1 star is not very good to five stars is excellent. And we got 70% 5-star ratings and 20% four-star ratings, so that means all the other stars were 10%. We had about a 3% of people who didn't like it, and that was mainly to do with some form of motion sickness or people's feelings of entrapment or whatever. Now, I can't... You're always going to get that within the spectrum of the population that there are some people who won't like VR, but, you know, I'm happy with just that, and it's very low. Extremely happy with the high ratings. When they were asked, 'Should the Museum of Applied Art do more VR?' 92% said, 'Yes.' So it's a great indictment that this is a success, that people like it and these are people where most of the people were actually older, and they were people who, a lot of them, hadn't played video games. Over 60% had university degrees. So it's a very educated public as well. So different from what you see in a VR arcade or you would assume for a VR arcade and what you would assume for video games. And that's, I think, quite important and interesting because a lot of these people are first-time users. I always say to most of the audiences, 'When you're going to see my piece, you're going to get a double premiere. You're going to get to see my piece for the first time, but you're also going to explore a whole new technology and a whole new form of art.' And that's often for people, further challenge. They have to get used to the technology and being in there before they can really get used to the art, but they learn pretty quickly, and we've had amazing, amazing, amazing results. So it's been great. The Museum for Applied Art has now bought the work, so it's in the permanent collection. I think it's the first definitely in Austria; I don't know where, anywhere, maybe other museums around the world or in Europe who have now started to collect VR art, but this is now part of that collection. So I think that's quite important. One other important thing about that is that initially, one has to say, when we were going through the iterations, there were people in the museum who were skeptical, and I know people in museums who are really against anything digital, but the Museum of Applied Arts was open, but there was a discussion, and we had to, with iteration after iteration, showing the work people became more and more convinced, and when the public at the opening, we had 1,700 people queuing to see two VR headsets, you know, it got a bit frightening. And then when people liked it, that was good, but the point about what happened there and it's also a kind of a first is that first we were in the cellars in the experimental space, you know, where you put the weird stuff where you're not quite sure, and within two weeks, the museum decided, Christoph Thun-Hohenstein decided, 'This has to be, go beyond its normal two months. We're going to go for seven months, and we're going to go upstairs, and you can be in the same gallery as the original Klimt painting.' And that is a great honor. So firstly, it's a great honor to be next to the work that we work next to, but secondly, to actually make that work. So there's a big balcony, and the VR sets are on the balcony above, below is the original artwork, and they live really happily next to each other. And so for me, if anybody says VR can't work in a museum and that it will destroy the experience of the analog works, I can say, 'Klimt's Magic Garden, Vienna, 2018, we made it worked. It worked successfully. We have not only the qualitative but the quantitative results to prove that that's right.' But for that to work, the VR work has to be an artwork as well.[Federica]: It must be particularly gratifying to receive such consistent positive feedback, considering that people have very different experiences in this magic garden. There is no task. And this is something so beautiful about this installation. It's not that you are on an adventure, there's a goal, and you need to follow instructions. You're actually just in the world, and you proceed at your own pace, you explore more if you want, you stay still if you want and just look around and contemplate, so the fact that people have very different experiences and yet come back with such positive feedback must be very gratifying. And you said that most people in the audience are older and have a higher education, but what about children? I believe children have also tried this installation. I know that there were children at MAK in Vienna when I was there. What can you say about children? How do they receive this type of installation?
[Frederick]: Well, I'm lucky enough to have a six-year-old daughter. She's been allowed to see the work. She calls them goggle films. And she's been allowed to see my goggle films since she was about five. Before, we didn't want that. And she's been in about twenty-five times. She adores it. And children are the best users, really the best users. She sits down on the floor when the flowers are there. She sits down and says, 'I'm having a picnic.' When the flowers come in front of her, she tries to bite them or to catch them. She runs through it, with it, so that the cable pulls her back sometimes, right at the beginning, when she wasn't used to it. We use HTC Vives. And she's very intuitive about that. It seems completely normal that you put those headsets on and you're in that kind of world, you take them off, and it's gone again. And the interesting thing is, she's actually discovered things about the work that I didn't think of. Well, she discovered things like my wife discovered as well. My wife wanted to go out and to climb the trees, so we hadn't, I hadn't thought about that, that we had to go up as well as across. And then she tried to crawl inside the trees and the objects, so to get behind this digital skin, and found the hollows and the vacuums inside, because they're very thin in digital space, and because you can move your body, you can find those hollows and vacuums. And my daughter discovered them as well, absolutely adores it, and she will sometimes shout to me, she's... I remember she was once in there and she shouted, she said, 'Daddy, daddy, I'm inside the expectation. Daddy, daddy, I'm inside the fulfillment. I'm right inside.' And so the children are actually fantastic in exploring the limits and the language and what those possibilities are in that interactivity because she's not just seeing it as a garden. She understands magic better. But grown-ups can, as well. As I say, my wife could as well. Maybe it's something about women. But to be honest, all users are like that, and it's very really interesting to see that there is not a normal user. There are people who can get in, they're digital natives, they charge around, they want to walk around. There are other people who literally just, at the opening when the original artwork lifts off and you're in the Magic Garden, they just stand there for five minutes and watch the light change. And I said to this one lady, I said, 'So, you know, you didn't go anywhere. You didn't travel. There's a lot more space.' She said, 'I was so overwhelmed by what I saw, I didn't want to. I didn't need to.' I said, 'Would you like to go in again?' She said, 'No, I can't. I need to digest this.' And a lot of people have that, actually, that need to digest what they've just seen, which I take as a kind of a compliment, because it reminds you how powerful this is, and how important it is to get it right. And that's why I wouldn't put it in with children before that sort of age before they've kind of got to the storyboard abstraction stage and they've probably seen a few videos and stuff like that. But there are other people who I've heard who sit down and meditate inside, like, you know, properly with their legs crossed. Some people just want to sit on steps, and some people just come back again and again and just want to spend their time within the space. And I was told by one, the guards at the Museum, the MAK in Vienna, recently, that they had one lady from Australia broke out in tears and took the goggles off and embraced her husband and kissed him, and she just, and he said ['What's los' 00:33:42], 'What's happening?' And she just said, 'It's just so beautiful.' And she was just kind of... It was an aesthetically overwhelming moment for her that the aesthetics of this had created joy. It was an epiphany in that sense, and that's really great. I mean, there have been people who've also come and said, 'Well, it's kind of different.' You know, and they're a bit gruff, and they're a bit sort of, and as I say, and we know statistically there are at least about 3% who didn't like it. I also last week had a man who was a bit unsteady on his legs and he felt a bit claustrophobic. No problem. You know, that's... He doesn't have to. He can take the goggles off. You know, you can't expect an art form to please everyone. You just can't, you know, and because some people don't like it is not a reason for not having it, so... [Federica]: Well, I can tell you that I liked it so much that if I had it at home, I would spend some time in it every day. This is how much I liked it. And I actually consider myself very fortunate because I had the chance to try twice. I tried it in Vienna first, and then today I tried it again here in Brussels, and I did it differently because after trying it in Vienna, I learned that there is so much more to explore than I had seen. I was one of those people who doesn't move much. I was mesmerised by so much beauty around me that it's not that I didn't move, but I didn't even go halfway through the map, so to speak. So today, I tried to move faster, and it's very funny, because by trying to move faster, at some point I clicked wrong and I basically threw myself off a cliff, and I got scared and I think I even vocalized that emotion, and I felt ashamed because it was like, 'Oh, come on. Don't be silly. This is not happening for real. This is VR.' But then at the same time, I was so excited. I was, 'This is so cool.' Speak of expanding the experiences with VR so realistically, you could try to actually throw yourself off a cliff and try that one, but knowing in the back of your mind, that you're safe. This brings me back to something you said about your daughter. You said you didn't want her to experience VR too early on. I would like you to elaborate a bit more on that, especially in the light of this. Do you think that the possibility of mixing virtual reality with actual reality is a thing or it's just some, for some reason, message that was put out, especially, I remember starting from the '80s, even with television and movies, like... It's not that I'm going to jump off a cliff in real life, but I grew up hearing this, like too much TV or too much video games then make you confused between fantasy and the real world. Is this one of the reasons why you didn't want your daughter to experience VR too early on, and do you think that this is a thing at all or not?
[Frederick]: Well, I think it's virtual reality, and, as I say, in one sense we've always been doing it because we dream every night, and dreams have a structure and a pattern which is, you know, very fitting to kind of virtual reality, and it's very digital because you can jump from one thing to other and all sorts of things happen. You can fly. You can do whatever your dreams are. So the mind can do these things, and we kind of do it anyway. The point with virtual reality is that your mind is doing that in the pace of whatever your body needs. That's why we dream, and it's driving that. With virtual reality, I could overwhelm people with surprises. I could play tricks on them, and it's maybe not what the body itself is needing. So for me, the difference is, a dream is coming from inside, whereas whatever happening with virtual reality is coming from the outside. Now, this is a really important point, I think, because when I make a film, a normal film, or even the virtual reality film Prometheus, I controlled what people saw, generally. With a normal film, that's why we have a frame. I control 100% of what you're generally going to see. If I make it a VR film, 360, okay, depending on what direction you look, you will see different things but in general, the pace at which it takes place I've decided. In VR experience, you don't have that anymore. I don't decide whether you stay in one place for the five minutes or you whiz around. That's up to everyone differently. What I do, however, control is the possibility of where you go. So where a film controls your gaze, virtual reality experiences control the possibility of your gaze. So that means you're always having to think options. You're having to think... In Klimt's Magic Garden, I thought a lot about, I want some experiences for people who want to throw themselves off cliffs, who want that kind of kick, you know? So my brother's an architect. He's got a very fine sense of 3D. He wants those sorts of things, and the flowers and the stuff he finds, it's interesting, but it's not exciting like jumping and being then in the water, like you where you thought... Actors, for example, love throwing themselves on the floor being, all of that sort of stuff, because they're used to being two things at once in their role and their body. Their body's used to it. Their brain's used to it. Other people aren't. It's healthy. 'I'm on the edge of a cliff. It looks like a cliff. It feels like a cliff. So you may have told me my feet are on the ground and they're in Brussels in an art gallery, but hey.' I mean, the overriding part of your brain goes, 'There's a cliff, and you're going to kill me if I jump off here.' So your fear kicks in, you know, the basic form of your brain. So I think, you know, those are really important things to understand, and it's about responsibility. That's why I do slow VR, because I don't want the responsibility of triggering something in someone that might be... And people talking [drugs 00:40:02] about bad trips, a bad journey, through virtual reality, and we have to know, when we do VR, because it's immersive we know through the brain tests that it is extremely real, and the brain takes it as being extremely real to a form that we didn't have with video and that we didn't have with those other things because with video, your peripheral vision, the vision at the side, is always saying, 'I'm in a room,' no matter whether I'm watching Star Trek or football in the middle. But in VR, no. No peripheral vision. You're 100% in the event, and so that's something you've got to be responsible about. And the thing is, because it's gone so fast and I know this because some of my friends are the psychologists who do this testing with cognitive psychology they know very well where the gaze works with films, and they have the technology to do it. They don't at the moment have the technology to measure gaze within [VAR 00:41:00] very accurately. That means the technology is well ahead of the other technologies, which are kind of like the safety mechanisms. And that's why I personally, for my sense, would prefer to go a bit slower rather than doing other things. But I know other people do VR horror. I've been in a VR cinema where I was watching, I don't know, some cultural VR programs and then other people started screaming, and we all looked round and ripped our headsets off, and they were all watching VR horror, you know. I thought, 'This is a bit stupid. Don't mix horror with other things.' But, you know, those are the kind of... Some people like those things and some people don't, and it's down to personality.[Federica]: An installation like this is still a lot of work to put together. You have mentioned some of the people that work with you to make this happen. I think that you mentioned also that it took about a year to make it happen. Could you summarize a bit the team, the people who worked on this to give us an idea of how many people are required, what type of skills are required, to make something like this happen?
[Frederick]: Well, Klimt's Magic Garden was created over about a nine-month period. That was between the first gaining of the huge file that we'd got from MAK of the work and loading them into the computer through the thinking time what to do with it. And we went through about five or six iterations, and mainly it was myself and working with Markus Cermak, who's a 3D graphic artist, who is a wiz on... I think we did it on an Unreal Engine, that's right. It was an Unreal Engine. And he controls, he can do everything with that. And that's also a problem, being able to do everything because it's like, 'Well, we could do everything, so what do we do? Because we don't have the time in the render capacity to do everything and try it out.' So in a sense, that's where my role came in as the director. He was almost like the editor in the film, the film editor and the camera person, and I was the director and the author and the screenplay writer of the film. And so I said, 'Well, Markus, let's do this. Let's try that. Let's try that.' So the first level was actually cutting things out, and Markus and an assistant worked on that, and that's actually the most laborious part. It's the boring part. Once we actually had them, individual parts of the painting, as artifacts, then we could start working with the colour, distance, pace, size, all of those sorts of things.[Federica]: Sorry, everything we see in the garden actually comes from Klimt's painting, or something was also synthesized in the style of?
[Frederick]: Everything, and I set myself that as an artistic... You know, art often comes from setting your constraints, you know. That's quite important, often, in the artistic process to say, 'You know, I will try and do this, but I will limit this,' and the limitation that I set was, everything that we show was in the original painting. So that's why one of the early stages was really looking at everything that was there, every detail, every point, to see, 'Well, what could we turn that into? What is it? What is it in itself, and what's the quality of it?' And so we then kind of had to cut that up and look at that, and then my job was to find the meaning of it and to see how it would go. And Markus was brilliant at showing possibilities, and obviously he then had things he knew about the Unreal Engine where he said, 'Well, we could do this and this,' and he showed me and I said, 'Yes, that's exactly right. That's amazing,' or, 'That's not right.' And once we'd kind of got the technology, then we started showing people like my wife and my daughter and other people, and seeing their kind of reactions and what worked, what they didn't like. And then it came to a structure where people were kind of wandering around this garden aimlessly and after the initial kind of 'wow' effect, it was kind of like, well, people got bored. It's kind of like, 'Where should I go? What should I do?' And that's where we came, I then started saying, 'Okay, we need a screenplay,' and my wife does screenplays and she then looked at that and we broke it down into the story of expectation and fulfillment, and we started giving the people characters and purposes. And that's when we then started saying, 'Okay, but the locations have to kind of mean something. It's not just about being pretty. They have to have something to do with Klimt, but they also have to, something to do with the experience you're going to have.' So this is where we then looked at the landscape that inspired Klimt, which is the Attersee in the Austrian lake district, and so we said, 'Well, let's go for a location like that.' Then Markus found one of the patterns and managed to turn that into something that really looked like water and waves, and that was an amazing breakthrough. And so we could have lakes, and we could have water. I'd said I'd kind of wanted water, and then we kind of went looking. So it's, you know, it's kind of... It went backwards and forward in that sort of sense, and then it was kind of like, 'So what experiences can we have that will not frighten people, but will still amaze people, but that won't blow the computer?' Because the computing power in that space to always be calculating where are you, and particularly since I insisted on having real weather and real light with real believable shadows, that's a huge amount of computing power to be able to sort of have that for each journey, and that took up a lot of that capacity. And so we went through about seven iterations. We had, you know, progressives and conservatives within the museum who needed to be convinced, but in the end, that came through, and then the great job came through of Markus to work with the museum to actually set it up in the museum and actually to get it working and to get the booth next to each other working so they don't interfere with each other because that was the problem that we found that the signals from the Kinects could, if you didn't put a high enough wall between them, they would pollute each other's signals, and so it wouldn't work. And it could have been a very interesting artistic path to go with that, but we didn't want to go this glitch path maybe another time but we wanted to keep those as clean experiences, and so that was a great job that Markus also did in terms of the installation and getting it reliable. One of the areas we worked a lot on with the iterations is, what do people see right at the beginning. And we started with kind of written boards and instructions and all that, and we threw all of that away because when you're in there, people are so sort of out of it, people don't have time to read. It's got to be intuitive that things just start happening, and, you know, you've got to design it in that sort of intuitive hands-off way, because reading and all of that... And that, for me, again, was a point that I really didn't want any signage, any reading. I wanted it to be nudges, but not signage, and I didn't want people to feel they need to push any buttons or anything. You know, I don't like bureaucracy, and I didn't want a bureaucratic artwork. And that's the same way we didn't although there is a structure which is similar to games with a series of levels, we have them as islands. I didn't want any points or any this or anything that looked like a game because it just wouldn't have been right for what one's doing and to be honest, it would have been disrespectful in a garden and in relation to Klimt's work because that would be far too narrowing down the experience of like collecting, collecting, and competition, and that's the last thing you want with, I wanted with this work that there would be any form of competition, because it's just about people going in and having an experience in their own form on their own terms.[Federica]: You've certainly seen this work in different stages of advancement and development. Have you ever entered your own creation and put the headset on and experienced a sense of wonder?
[Frederick]: Yeah, I have it... I have kind of 'wow' moments again and again. I deliberately haven't been in all corners because I kind of want to save it up, and, you know, it's a bit like when you get your birthday cake and then you want to kind of save it for the next days and the next days, you know, because you don't want your birthday to be over, and so that's kind of what I feel a little bit about it. And the great thing is, because we have real light within the five minutes we have a day, a sunrise, a sunset sequence and a nighttime sequence with a moon, a full moon it means that we get all of those different lights, and depending on where you are, the shadows are different, the light is different, and so the number of variabilities between that cycle and that location you're in, it will always be different. And so I can remember some amazing sunsets and some amazing sunrises, and the moon is wonderful as well as its own kind of atmosphere. So yeah, it's something that I feel I always kind of want to dip into.[Federica]: What about sound and music?
[Frederick]: In the production process, we got to about two months, three months before, and I realized one thing we had forgotten and hadn't thought about was sound, and the soundtrack is really important. And I rang out my friend George Taylor, who also calls himself 'no one' but that's his artistic name and he's done all the music for my films for a long time, and he did the music for Pitoti Prometheus, and I said, 'I've got a job for you, but I need a film soundtrack, but it isn't a film, and you started off in your career doing game sounds. Well, it's kind of like a game, but it isn't a game.' And he said, 'Well, how long is it?' And I said, 'Well, it could be five minutes or it could be several hours because people can spend as long as they want.' 'Oh. So what is it, then?' 'Well,' I said, 'It's a VR experience, and I need a soundtrack. Are you up for it now?' You know, he was up for it, and so he started composing in a different way for this experience. We looked at the work and we took cues from the little bells on the expectation's headdress and some of the birds that are in the film, but not too many. And we have gamelan, and we have all sorts of things. And he created a soundtrack that in the end is a kind of a homage to Brian Eno's music for airports, and it's interesting because that's ambient and this really works well in this sort of space. And a lot of people come out and they say they like the music, and the music is kind of important, but each... It's not a soundtrack because it's sound zones, and depending on where people go, they will trigger different, not only different effects, things, small things getting big or things falling from the sky, but they will also trigger sounds. And so actually, when you look at it in the computer, you actually have a series of boxes with invisible lines around, and as people move through them, they trigger sounds. And so that becomes a very interesting way of looking at music and sound as well. George Taylor was... Well, when I was talking to George and answering his questions, I realized that we have to think about sound in virtual reality, with virtual experiences, in a completely different way, and it's about creating an ambience so that there is sound there, but also integrating [thing 00:53:05] sounds that will be triggered at specific points and only triggered if people come to various places and will stop again, but the whole thing needs to feel harmonic, so it becomes in a sense more orchestral, but you can't control when people will see what. So it became actually very important, and this is why it really needs a composer, not someone who's just put in a load of sound effects, because the Magic Garden works because George Taylor really got the pitch right and made sure that no sound, no matter what it was whether it was the birds or gamelan or the background ambience would destroy the sound of the other and it would feel harmonic but still have its own individuality and that you wouldn't go, it wouldn't get totally boring if people were doing nothing. So it really became a very interesting case where we were thinking about interaction, space, and time, which you usually think about in terms of music, wasn't so important anymore. And so we're getting very close to ambience and ambient music, which is, of course, interestingly enough, the first ever piece of ambient music was composed by Gustav Mahler on the Attersee, his Symphony No. 2 and Symphony No. 3, where he tried to compose with an orchestra the sound of summer on the lake in the Alps as a space, as a sound, as an ambience. And so here with Klimt, we're coming back to the Attersee with a similar form, but Mahler wouldn't really work there because Mahler's piece is time-based, whereas George had to have time but with, also within space, and so it was a really fascinating thing.[Federica]: Is it a concern of yours the fact that due to the technology it required and mostly due to the fact that you need to experience this firsthand, so you need to go to the museum, etc., and there's only two headsets available at a time, this is basically a work that is much harder to circulate than maybe a film which you can just even put online and then millions of people can see it, and they can actually see the real thing. They don't see... Like there is actually online a short video of something you could see inside of the Magic Garden. I know that some composers, for example, of electronic music where sound was moved in space and several channels, well, some of these composers refused to have a reduction on a stereo track just for the sake of releasing a commercial edition on CD, for example, so they will say, 'No, either you experience the real thing, or I'd rather have you experience nothing, because otherwise it's a different thing.' How do you approach this? What's your thought about setting up a virtual reality installation but then at the same time having a short video, a clip online, clearly 2D, that just shows, well, a reduced version of what you could experience yourself?
[Frederick]: Well, I'm not a puritan, you know. My taste in that sense is more catholic, more broad. I think it's perfectly okay to have a 2D video clip. There's also a 14-minute version which I call a director's cut, which is a recording of one of my journeys through the Magic Garden. Those are artworks in their own form, as video forms. They're done with a virtual camera, their own form of animation. They are not the full VR experience. They are a film of it. But then a photograph of the Taj Mahal is not the Taj Mahal. You have to go to India to see that and experience it, but photos are still quite, you know, have their own appeal as photographs, so I don't... I'm not puritan about that to say my work only needs to be there. I actually think it's good because it's a taster for that other sort of experience. The thing about technology at the moment is because some of the issue of the cost and also some of the maintenance issues with the public putting a headset on and off and those sorts of things, it is something at the moment that's restricted to museums. It could also be in VR arcades or those sorts of things, dedicated spaces. At the moment, I don't think that that's that bad, to be honest, because I think I'd rather have people's focused attention and wanting to go to a place where they say, 'I am going to do this to have this experience and give it my full attention. I'm not going to have my mobile on. I'm not going to have the cooking on. I'm not going to be listening to the radio. I'm going to go to a museum or I'm going to go to a place and I'm going to have this experience.' I think that that gives you a kind of a focus, and I think for art that that's a good thing, just like going to the cinema is a better way of seeing it on your laptop or on your mobile if you're going to watch a film. Yeah, you can watch a film, but do you understand it? Do you really feel it? And particularly with something that's so immersive. I think it's actually probably also a safety issue that when you're under the headset, you really are there. You're not anything... And to have someone else watching you is that... If you have those VR goggles and you're in the home where you're doing thing you definitely shouldn't be driving you can't. So I think that that's a positive side for the moment. In the long term, people will have VR headsets at home, and I would want that as well, and people have said, 'I wish I could take this home. I wish I could experience this at home,' and I'd like to make that available and I'm working on that as well. But at the moment for launching it and at this phase where, you know, we are five minutes... You know, film was created by the Lumière brothers, and it took about 30 years for cinema, like with Battleship Potemkin, as a film to be discovered. We are probably about like where film was five years after the Lumière brothers with VR, so that means that a huge number of the audience is, like most, about 70%, of the journalists say and these were cultural journalists I showed it to this morning had not had a VR experience before, and they were not games players either, so this is new. It's baby steps. So I'd rather people go to a dedicated place to have their first steps with VR in a safe and in a good environment rather than it... And bad VR works have given the genre a bad reputation among some people to say they have bad VR and say, 'I don't want anything to do with it.' Because that's out there as well, and it, like with stereoscopic 3D film, there was too much done bad stereoscopy, and it gave the medium a bit of a bad reputation, and that's what I'm definitely trying to say is, 'Trust me,' and then they'll trust the medium a little bit more. And to go to a museum or to a place where you get a dedicated experience, I think, is the best way to start. In 5 or 10 years' time, now, of course, people will be used to it.[Federica]: Yes. I said earlier that I think we will get more and more of virtual reality. It's going to become more common. And you confirm, you just said you think people will normally have this type of technology at home in a few years. So from still images to moving images intended as film, to then the film also moving around you, to virtual reality, there seems to be a path forward with a richer experience, and it seems like going back from virtual reality to the flat screen is going backwards, is, it gives you less. And you would say nobody wants less. As a filmmaker, what do you think about this fact that it seems like every other technology seems to kill the previous one or does it? Or it is just different types of tools and therefore VR will not kill cinema?
[Frederick]: Well, I also make cinema films. I've made cinema films, and I want to continue, and the point is, is that some people don't want to learn a technology. They just want to sit down, have an experience, get up and go. They also don't want to make any decisions. One of the points about having a film and watching television is to, as we say in London, veg out, is to just switch off and to, in a life where you're constantly during the day having to make decisions big, important decisions, cross the road, this, that, and that suddenly to sit down and just go, 'Okay, director. Just give me an experience, and you make the decisions. Take me on this ride. I want to be in... I don't want to be in the driver's seat. I want to be in the passenger's seat.' We psychologically have a need for that. We have a need for that to unwind, and we will continue to have that, and I want to provide that as well. The other thing is that, like with the Magic Garden, of course it would be amazing to say out of this, where could I put the greatest hits, the greatest moments of the whole garden and have the perfect journey through the garden? Yes, I could do that. It would be another kind of thing, but actually for this, I don't think it would be an interesting, as interesting as letting everybody have their own experience. But cinema won't die because people are lazy and people want to switch off, but people are also sometimes not lazy and want to be challenged, want to do something, and want to have, be in the driver's seat sometimes of an artwork, and that's when they'll go for a VR experience and be able to control it themselves. And VR offers the intensity, the visual pleasure, and the power of cinema, but allows you to be almost like the director on stage to say, 'Cut,' or, 'Slow down,' or, 'I want to go left or right,' and all of those sorts of things, so it's giving you that kind of power which I think people always want. The other reason that cinema will never die is, we've had over a hundred years of masterpieces that can only really be shown in cinema, and you can only experience that, and so you need to go there to experience that, and they are still amazingly powerful and important.[Federica]: Klimt's Magic Garden has been so successful that, for example, the exhibition in Vienna was extended until October 2018. We are now in Brussels. Can you summarize a little bit where this installation has been and where it will go, just in case somebody listens to this on time and can still actually go and experience this?
[Frederick]: So at the moment, as we speak, the 20th of September, 2018, Klimt's Magic Garden is showing here in Brussels and in the Ars Electronica Center in Linz in Austria and in the Museum of Applied Arts, at the MAK, in Vienna. In Vienna, it will shut in two weeks, the 8th of, I think the 7th of October is the end of that. In Linz in Austria, it will go until Christmas, and here will go into January. At the moment we're in discussions with the Royal Academy in London, with the West Court Gallery in Cambridge, and those are the two concrete places at the moment we're thinking of. As I say, the MAK, it's going to be, it's now part of the collection, and so there is talk of it having a long-term kind of permanent home in the MAK in probably in the Design Lab or whatever. But, as I say, it's flexible, and it, brought to various places, but it's, you know, it's... Because it's digital, it's transferable. Whoever has, you know, the setup, the HTC setup, can show it, and I hope it will, you know, go around the world to lots of different people. VR skeptics often say, 'Well, you actually, discrediting real forms of art. It's so ephemeral and if you are actually... You shouldn't mix the two. You shouldn't have a VR work about something and the original there as well because it devalues the original.' And so I know museums that say, 'We want nothing digital. You must just have this experience of the painting and the silence and that's it.' Okay. I can respect that. But what we found with Klimt's Magic Garden is that and this is my philosophy with all forms is that it's about backwards and forwards. The VR experience can show you things about Klimt's painting or any object that you don't see in the original and you don't experience in the same way because we make it bigger, because we make it move, because we make the light move across it, because we transform it. So the actual art experience of visiting a museum comes... I go into the VR experience, then I come back and see the original. Then I go back into the VR experience. Then I come back into original you're constantly, that backwards and forwards, again, kind of 'dialectic' is a word that's used for it, that's allowing you to understand both better, and that's kind of something that, in the terms of VR world, is already very big. That's my niche and one of my niches that I'm kind of looking at. And that's why the VR work has to be respectful to the original, and they work well together, but they need to be intelligent as well. It mustn't be a kitsch version, or it mustn't be this sort of stuff, and it mustn't demean the original. It has to be respectful. When I did my work about the rock carvings from the Valcamonica, you know, these are very thin, and they're 3D scans, but they're extremely thin because they've just been hacked into the rock. And there was talk of, 'Well, make them fatter, make them bigger, so that people can see them better.' And I said, 'No. Hands off. It has to be as pristine as that. That's the original, and we have to respect that.' And actually, what we did to make it work was that with artificial lighting and with low lighting, just in real life, if the light's right suddenly these figures come out of you and get bigger. It's the natural way that the prehistoric people wanted it to work. It's the natural way it works in that valley every morning and every evening when you get magic hour, and we respect that and put that into the digital experience as well. So they look weird, they look funny, and that's right because they are weird and they are funny because they're 4,000 years old and they were done at that time, and they are what they are. So the VR experience is also there telling you more about the nature of these analog originals or vice versa. And we had that with Klimt as well. So one of the things that's good fun and is a game for children as well is: See the VR work; now, go back to the original and find where we found those patterns. Can you see the flower that you stood underneath? Can you see the pattern where we took the waterfall? Can you see the figure that you crept inside? And there you go, 'Aha, that's the transformation. That's what digital can do.' This is also part of the magic, but also you realize, 'Wow, that really is the gold that Klimt used. That's how he painted it. That's how exact it is.' And that, for me, is a really, really important part of this work and something I've really learned from this work.[Federica]: Well, I would like to thank you very much for the time you spent with me. It's been a great pleasure meeting you. Thank you for being on Technoculture.
[Frederick]: Good. Well, thank you for your questions, and it's been great fun and yeah, I hope, you know, this podcast does some good and, as I say, if anyone's got any interesting questions, then contact me, email@example.com.[Federica]: We will certainly link this address in the description of this episode as well as your book, The Art of Projectionism, with the original story about ambient cinema. Thank you again, and now let's actually go to the vernissage of the Klimt exhibition right here at the Bazaar in Brussels.
[Frederick]: Okay. Cheers.[Federica]: Thank you for listening to Technoculture. Check out more episodes at technoculture-podcast.com, or visit our Facebook page @technoculturepodcast and our Twitter account, hashtag Technoculturepodcast.
Page created: December 2018
Update: February 2021
Last update: July 2021