Presented at the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
Misremembering and mistranslating: the associations of GANs in an art context
1 Introduction
Research has looked at whether artificial intelligence, and more particularly machine learning, can create art [4, 2, 6]. However, the focus of this work has been to consider and judge the result as “art" through the impact of visual parameters on a viewer (i.e. “does this look like art?"). This ignores a vital consideration of an artist when producing a piece, that of the impact of the materials used have. Producing an image using a GAN versus any other way gives the viewer a different experience, expectation, history, traces and contexts to consider. What are these associations and how might they be used in a piece of work? Materiality is “one of the most contested concepts in contemporary art and is often side-lined in critical academic writing"[5]. It has only recently begun to be explored digital art, and has not to my knowledge been applied to work produced by neural nets. This is an important gap: images produced by GANs are becoming more and more prevalent in the international fine art scene (e.g. Ars Electronica 2017, Serpentine Gallery Miracle Marathon 2017) yet there is little language to talk about them in an art context beyond the scientific. Art should be able to comment on contemporary scientific theories through use in a creative practise whilst also “ensuring a critical distance that prevents the elevation of these fields to the status of unquestioned authority" [5]. Can a GAN or indeed training set become a “wilful actor and agent within artistic processes" [5] as other materials are? I will start to explore some of these questions by looking at the potential associations of training sets and GANs, in particular pix2pix, and how I have attempted to embed these associations in my own work.
2 Associations of Training Sets
Training sets are rarely discussed, yet are central to the eventual image output. Most of the training sets that are readily available are compiled by researchers according to a variety of methodologies, but because human subjectivity is always involved at some point in either the source content or in the process, they inevitably come to enshrine certain cultural or social attitudes. These datasets are often extremely large which means that artists cannot control though selection which biases and prejudices are being replicated and repeated. Furthermore, the invisible labour and associated power relations of those used to create the sets, often mechanical turks, and what this might mean is rarely acknowledged in discussion of GAN generated imagery as art. Small batch machine learning programmes (e.g. pix2pix) require a much smaller training sets which are possible to put together by hand. Artists can start to use these create smaller handpicked training sets deliberately to push back against and explore some of these issues.
3 Associations of GAN generated images
Whilst there is a certain amount of control afforded to the artist through the creation of a training set (images, labels), a GAN allows a quality where the “where materials develop uncanny lives of their own and display their power to metamorphose"[5]. This power to metamorphose without real control is something that within an art context is more closely associated with work that deals with biology or nature, than the digital. The style of images generated by GANs are distinctive (particularly in “poorly-trained" models) and echoes perhaps Georges Bataille’s concept of bas materialite - ruined, decaying and decomposed - something antithetical to the smooth surfaces of capitalist consumer goods [7] and, I would argue, a large proportion of the aesthetics of contemporary digital art. These associations and their meanings are yet to be fully unpicked by art historians and cultural theorists but are starting to be used by artists if Petra Lange-Berndt’s definition of materials in “electronic technologies" (i.e. the digital) is used. Here instead of imitation the result is “as hybrid, as mutant material, where a new substance appears that also critically comments on the initial components that started the experiment"[5]. GAN generated imagery can critically bring out and start explore some of the assumptions of creativity, authorship and control if the context of materiality is considered as evidenced by recent works by AI artists (e.g. Memo Akten, Mario Klingemann).
4 Fall of the House of Usher
I have tried to take both of the associations of training sets and GANs into consideration when making my own work, Fall of the House of Usher, a twelve minute animation made using pix2pix[8]. It is a piece that could have been hand animated but by choosing machine learning I was able to heighten and increase themes around the role of the creator, the reciprocity between art and technology, and aspects of memory in a way that would not be available to me otherwise. For example, by choosing to create my own training set by hand drawing in ink stills from the 1929 version of the film (fig 1, fig. 2), I was able to emphasis the history of drawing in interplay between the digital and the physical, question the role of the human and artificial intelligence, and start to think through questions around labour. By restricting the training set to the first four minutes of the film, I was able to control to a certain extent the levels of “correctness": as the animation progresses, it has less and less of a frame of reference to draw on, leading to uncanny moments that I cannot predict where the information starts to break down, particularly at the end of the piece. I deliberately take the “decay" offered by making an image in this way and turn it into a central part of the piece, echoing the destruction that happens in the narrative and the “copying" that occurs (the piece is a copy of a copy (film) of the original (book)) (fig. 3).
GAN images have allowed me to discover new possibilities[3] that I am now exploring in the next iteration of the piece. It has held up a mirror to my own process and allowed me to see elements that I was unaware of (how I draw eyes, what I choose to emphasise). I am now creating a new training set where I have drawn the GAN generated images (see fig. 4), copying the artefacts and mistakes (some of which I have made in the original training set, such as a chair that appears and disappears according to whether I remembered to draw it or not). Copies of copies of copies, decaying and decomposing, becoming more mutable and less controllable. The results of this model will be merged and absorbed into the animation to create a piece that is a series of duplications which in the words of Borges is “no less real, but closer to expectations" where over time and cycles comes “a purity of line not found in the original"[1].
fig 1: Hand drawn elements for training set for Fall of House of Usher.
fig 2: Training set for Fall of House of Usher.
Fig 3: GAN generated stills from the film
Fig 4: samples of redrawn GAN imagery in ink from Fall of House of Usher.
References:
[1] Jorge Luis Borges, Andrew Hurley, and Andrew Hurley. Collected fictions . Viking New York,1998.
[2] Simon Colton. The painting fool: Stories from building an automated painter. In Computers and creativity , pages 3–38. Springer, 2012.
[3] Mark d’Inverno, Jon McCormack, et al. Heroic versus collaborative AI for the arts. 2015.
[4] Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. CAN: Creative Adversarial Networks, Generating “Art" by Learning About Styles and Deviating from Style Norms. arXiv preprint arXiv:1706.07068 , 2017.
[5] Petra Lange-Berndt. How to be complicit with Materials . 2015.
[6] Alexander Mordvintsev, Christopher Olah, and Mike Tyka. Inceptionism: Going deeper into neural networks. Google Research Blog. Retrieved June , 20:14, 2015.
[7] Michael Richardson. Georges Bataille: essential writings . Sage, 1998.
[8] Anna Ridler. Fall of the House of Usher.