For the past few years, I’ve been exploring using AR in my art studio practice to help me create, organize and share my artwork. When I was part of eleVR, I did two experiments: in early 2017 I used life-sized holograms to lay out and test large sculptures at my studio at the Banff Centre for Arts and Creativity [see blog post: Sculptural Experiments in AR], and in summer 2017, I made a series of small sculptures in my home studio by placing holograms on a table and “filling” them in with pieces of foam [see blog post: Augmenting an Art Studio Practice].
Since then, I have remodeled a 1,000 sq. ft. barn to use as my new art studio with the aim of getting back to some of these AR experiments in a larger, dedicated space. Two months ago, when the barn studio was finally ready for move-in, the first thing I did was set up different areas in the studio for making my work: a desk, a mold-making/casting table, a photo studio, an inspiration wall and free floor space. Once these “stations” were set up, I immediately started brainstorming ways to use AR to support my work at each of them in various ways. In this blog post, I will share some initial sketches of what these different ways of using AR at the studio stations might look like. My hope is that as I settle into the studio and develop these workflows, new needs and uses for AR will emerge that will support our “spatial thinking” research more generally.
Desk Station
As I continue to fill out the studio with supplies and materials, one thing I’ve been doing a lot is writing down shopping lists whenever I think of something I need. I find myself writing the lists on scrap pieces of paper, and either misplacing them or bringing them over to my desk to try to keep track of them. My desk is where I do most of the administrative work that I need to do on a laptop such as research (web searches, reading online articles, books and social media), communication (email, online chatting, video chatting), scheduling (calendar, todo-lists, work planners), and budgeting (online shopping, spreadsheets, accounting, etc.).
It occurred to me that instead of painstakingly typing up the pile of paper shopping lists that I had building up, I could simply snap a photo of them and place them on the wall above my laptop in AR. Here is a picture of one wishlist of items that I wrote down while I was working on a plaster mold:
I took a picture of the list, and then I put the list into my HoloLens, and placed it directly on the wall above my laptop, where I would be sure to see it:
This way, every time I returned to sit at my laptop (and as long as I had my AR headset on), I would be reminded of my shopping list and could more easily remember to order the things I needed online. It was surprising to me how something seemingly so simple could have a practical benefit right away.
This floating, digital shopping list then elicited ideas about a live, dynamic shopping list that I could edit remotely from anywhere in the studio, without having to stop to write things down. I started having ideas about all kinds of lists, notes, sketches and messages that I could send over to my desk while working away elsewhere in the studio. It would be interesting to prototype something like an “Augmented Reality Google Keep“, that you can use even while your hands are pre-occupied, dirty or you simply don’t have time to get out your smartphone.
The idea of connecting my “desk work” to the more “just in time” thinking while I’m busy in my studio is exactly the kind of productivity boost that I could really see AR providing for me. I’m going to keep paying attention as I work throughout the studio, and the next time I think of some administrative task that I’ll need to do at my desk later, I’ll make a note to see if there isn’t an AR tool that I could prototype to fill that productivity gap.
Mold-Making and Casting Station
One thing I’m working on in studio these days is making silicone molds of various objects, and then making plaster casts from the molds. The process is chaotic and messy, as it involves mixing potentially toxic powders and liquids into viscous, goopy materials. Keeping track of all of different ratios of materials needed to mix is sometimes overwhelming, so it’s another instance where I end up with hard-to-track piles of paper with hand-written notes on them. Just like at the desk, I thought to take pictures of all the notes and put the pictures in the HoloLens above the mold-making table:
I found it nice to be able to have these “floating” recipes above my work area, that I could keep looking at whenever I needed them. While it’s not much different than having a pinboard, I began to wonder if these digital lists couldn’t in the future be more context-sensitive, and highlight/grow/come into focus as I needed specific ones.
In addition to recipes for mixing, I’m also constantly thinking of new materials I might need. This was the table I was sitting at where I wrote up the shopping list I mentioned earlier. I realized it’d be useful to have a copy of the list that was above my desk also here at the mold-making table, so that I could edit it in realtime (perhaps via voice-command), and have the two lists always sync’ed:
On the actual mold-making table surface, I’m currently trying to make a new mold, starting with foam to define the shape I want. The shape I am after is something like a birthday cake that’s either been dropped or eaten into, and while I could sculpt a shape like that from memory, I thought I would recall the process I used back in 2017 of using a hologram reference to help me. I 3D-scanned a cheesecake a few months ago, and brought the 3D model into the HoloLens and placed it at the scale I wanted on the table. Then, while wearing the HoloLens, I cut pieces of foam and started gluing them together inside the shape of the hologram:
For me, there is something about having a three-dimensional reference, instead of needing to recall it in memory, that lets my hands take over and just “fill in” the reference space with materials. I don’t have to imagine the form, and instead I can spend more time responding to what’s in front of me. It’s liberating, and I find that I feel more connected to the object. This isn’t a new insight for me, but I realized that I continue to value this addition to my art-making process. I know that I make 2D paintings best when I have a live reference in front of me because I’m able to let my eyes and hands do most of the work, without needing to create intermediate mental imagery. Similarly in 3D, this process of having a multi-dimensional, “holographic still life” to sculpt from is invaluable, especially since it can be resized, rotated and positioned freely. It can also be endlessly replaced, in realtime, with other 3D models of objects I have scanned, or 3D models made by others that I have downloaded from the cloud, or … sometime in the near future, it can possibly be replaced by a 3D model generated by an AI, expanding my list of creative, visual possibilities.
Photo Booth Station
Whenever I want to document an artwork, I like to take clean pictures of it against a neutral background for easier sharing on the web. I have a permanent setup in the barn studio for that, and realized this could be yet another place where I could use AR to help me be more productive and add inspiration to my process. Sometimes, the studio booth is a place where I test out different orientations, groupings and arrangements of my artworks, and I make creative decisions intuitively, right on the tabletop, while snapping pictures periodically. I was curious how this process would feel with a hologram sitting there next to the objects being photographed, serving as structural inspiration.
In the picture below, the object on the left (a partial architectural structure) is actually a hologram from the standard set of holograms available in the HoloLens, and on the right is a stack of objects I made and collected in my studio. By having the architectural inspiration there on the table, I can again use it as a direct reference to help me think about volume, structural integrity, weight and balance while snapping pictures in between arrangements. Because the hologram only shows up in the HoloLens, it doesn’t snow up in the final documentation photo of the object on the right. Using holograms of previous work of mine, or of inspiration work, I can see what angles work, and maybe make spontaneous arrangements and new work in the photo shooting process.
To add even further inspiration, I also tested placing AR photographs of professional artworks that inspire me on the wall above the photo booth. In the picture below, there are two AR photographs of installed works by Doris Salcedo “hanging” above the photo booth. They serve as examples of photographic lighting and angles that I admire and want to try to imitate. Having them there to look at while I’m at the photo booth is useful for the obvious reasons; but again, since they are live, digital images, I can imaging them occasionally rotating through other artwork photographs that I admire. This way, instead of being printed images on paper that eventually get dusty, faded and wrinkled, these inspiration images can always feel fresh, clean and in the final medium of ‘light’ or ‘screen’ that I intend for own my images as well, since my photos are rarely meant for printing on paper anyway.
Inspiration Wall Station
The kinds of inspiration images I imagine using in AR at the photo booth are also useful to me at an even larger scale. In the barn studio, I have a dedicated wall just for inspiration images of artworks by other artists that I like to curate and keep around for a while as I work out new sculptural ideas. Another sculptor I’m thinking about in studio these days is Rachel Whiteread, who makes large-scale casts of domestic architectural interiors and structural elements. I have some casts of a door fragment that I would like to turn into an artwork eventually, so I found it helpful to put up some photos of Whiteread’s pieces on a wall to glance at from time to time. Just like in the photo booth, I could see these digital inspirations rotating through a list of other images I chose on the fly, or images being recommended to me by an AI which is researching images “like” the ones I enjoy by Whiteread. If I’m feeling extra productive, I could even interact with the suggested images by telling the AI which images I do or don’t respond to, by curating something like an “inspiration playlist” for myself.
While I had never really considered using AI in my studio practice, this “inspiration playlist wall” feels like a perfect use case for me. Curating and looking for images usually takes tedious time and work at my desk, while I usually prefer to be busy with my actual art-making. I’d love to protoype a simple AR app that might suggest images to me, both at the inspiration wall and in the photo booth wall, perhaps using Lobe or a similar app that lets you control your own deep-learning models.
Large Scale Sculpting Station
One of the areas I’m most excited about in my studio is the open-floor space in the middle where I hope to create large-scale sculptures. I will probably work on installations that combine found materials, found objects, and structures that I create myself out of raw elements such as foam, plaster and wood. I don’t have formal training in sculpture, so I will need the extra visual guidance for solving problems such as structural integrity (will these objects stand on their own?), conceptual clarity (what shapes am I trying to evoke, and why?) and relatability (do these shapes make sense to me? or to other people?). I can see AR helping me with all of these considerations in different ways, and have started using the HoloLens to place simple holograms in the space to see how they feel and interact with materials I am starting to collect. In this picture below, I have placed two holographic white plinths in the space: one with a holographic Drumstick resting on top, and another with a piece of recycled foam standing behind it.
Already, I am imagining AR apps that simulate physics and help me lay out structures to test their structural limitations and learn about how certain materials, shapes and arrangements do or don’t work well together. There may already be existing apps for this kind of layout planning for architecture and design using the HoloLens such as Dynamic 365 Layout, and I would be excited to test them here. And if those don’t yet simulate simple physics, I could possibly prototype a small app in WebVR and A-Frame, using something like the aframe-physics-system.
In addition to using AR to guide me in thinking through mechanical issues, I’m also imagining using it to help me brainstorm shapes, objects and spatial concepts to use in my work. Again, as with the photo booth and inspiration wall stations, I could see myself using AI to help me find 3D models with shapes similar to ones I am interested in (birthday cakes and Drumstick ice creams), or even using AI to generate brand new 3D models with forms similar to ones I’m using. While I don’t have much experience making sculptures yet, I do find it useful in making 2D drawings and paintings in sometimes “letting go” and seeing shapes, lines and compositions from a new perspective. I am guessing that a similar out-of-the-box nudge would be helpful at times when thinking about sculptural works. While making small and foam paper prototypes on a table is one way to think through formal ideas, I’m curious how working with faster, dynamic media in the form of holograms would feel in the frenzy of making, taking apart and re-imaging shapes right there on the floor.
When I’m done making work and feel ready to ‘step’ back and look at it as a whole, I’m often wondering not just how the piece reads to me, but now it might read to other people. I’d love to use the HoloLens in these moments to capture the work I’ve made as 3D scans to easily send them to others. If there were a way to do that directly from the headset, then I could imagine an app that would let me send something like ‘3D text message’ to colleagues for feedback. They could either receive the message on their phone to quickly view on a screen and pan using hand gestures, or if they also had a HoloLens headset, they could see the hologram at real scale themselves. I can already envision making a prototype for this kind of interaction using Microsoft Altspace‘s new MRE SDK Framework, which I will write more about in a future blog post.
These ideas of using AR in the floor space in my studio to both help me ideate and share sculptures are very exciting to me. I’m especially thinking of the possibilities for embodied remote collaboration. Once I am accustomed to working on my own sculptures in this way, what would it be like to collaborate with a remote colleague in realtime? It’s already commonplace to collaborate remotely on things like online documents and spreadsheets, and I’m looking forward to expanding media collaboration to include more embodied ways of sensing.
Studio Sharing Station
I’m always thinking about how to share my artworks with others, especially other artists, and fantasize about being able to easily send and receive messy studio ideas directly back and forth with them. M and I have been talking about this idea for a while, and a few years ago we prototyped a VR art studio called StackSpace as a testbed. Now with the barn studio, I wanted to take the idea into AR as well, to help think about how that kind of studio cross-collaboration can work not only in immersive media but also in mixed media.
For now, I have started to set up a “sharing station” where M and I can share visual ideas back and forth. The first thing I have put at this station is a 3D model of M’s home and studio that they created in Anyland. With this AR model sitting on a table, I immediately thought of a portal that could serve as a connection between our two spaces. While M is working in their studio, perhaps in their VR headset, they could occasionally send me messages to share a picture or 3D model of something they have made. The studio model could emit a glowing light to signal to me that a new message is coming, and I could come over whenever I’m feeling the need for extra inspiration.
Taking the idea one step further, and taking inspiration from Microsoft’s holoportation demo from a few years ago, M and I have been imagining what it would be like to actually visit each other’s studios in AR. While both wearing HoloLens headsets, we could turn ourselves into avatars that could visit each other’s spaces in realtime. I have some idea of what that might feel like in VR, having created an art studio for myself in Anyland a few years ago and inviting M, Vi and others to visit it. I remember the experience being not unlike being in a world-building game and inviting others to come see your corner of the virtual world. In AR, however, there is an extra leap to invite someone to an actual workspace you inhabit. The idea of studio visits is ingrained in the art-making process, and is a valuable way to invite in new ideas and exchange into your creative space. However, the obvious limitations of travel have made studio visits impractical for some, especially those living in more remote parts of the world. I’d love to experiment with M in using holoportation for “AR studio visits”, to see if this accessibility gap can be addressed at least in some small way.
Putting all the above ideas together, I could also imagine a small representation of not only M’s studio but also M and their artworks as well, all existing in realtime in my studio. Not necessarily permanent, this could be a mode of the “studio sharing station” that we turn on whenever we want to more closely collaborate, chat in realtime, or simple co-exist as we both work in our respective studios. Studio time can sometimes be isolating, and as I plan to spend more and more time in the barn, I like the idea of opening up the space using AR to invite others in to not only create and chat with me, but simply exist alongside and share in some creative time together.
Prototypes for the Stations
Now that I’m gearing up to start making art, I’m excited to use these different work stations as testbeds for AR productivity prototype apps. I hope that testing the apps in a real studio environment will help generate use cases we haven’t yet thought of, but that will serve to genuinely push my art practice and ideas forward. I look forward to seeing what comes next; stay tuned for updates on how that goes!