How Computers Imagine Humans?

João Martinho Moura (2017)

‘How Computers Imagine Humans?’
João Martinho Moura, 2017-2018.
Custom software;
two computers with built-in cameras;
sound speakers.
Dimensions variable.

 

Presented in China (2017) as research, Germany (2018) and Republic of Korea (2019).

June 22 – July 28, 2019
How Computers Imagine Humans?
Special Exhibition at ISEA 2019, invited work
25th International Symposium on Electronic Art

Asia Culture Center
Gwangju, Republic of Korea
[pictures]

Pictures (top) and video (bottom):
Exhibition in Duisburg, Germany, December 2018

Statement:

Artificial intelligence (AI) can bring unprecedented benefits to society, but also can generate risks on its misuse. Don’t forget that AI is us, made by us and it’s being fed and improved by us every second. 

In this media artwork, AI is used against AI to discover How Computers Imagine Humans, using a selected computer visual noise (one computer) and an AI face detector system (another computer). Both systems running in real-time against each other, using just built-in cameras to communicate. In current times where GANs (generative adversarial networks) are propagating and also saturating our screens, the subject of art made with AI it’s gaining attention and provoking reflections on how will we deal with this kind of medium in the future. So, inspired by the foundations of these GANs, I’m presenting a straightforward experience, using a very well known algorithm used for face detection (a classical one) and unusual use of its technique, intended to do the opposite of what it is supposed to achieve. Instead of trying to locate and capture faces, I generate facial images ‘imagined’ by a computer through the exploration of hypothetical possibilities, starting just from visual randomness (as generating noise signal it’s a basic procedure on computers).

More than what if offers in terms of visualization of what is behind algorithms, this work, as it is presented, with 2 machines interacting with each other without a wired or wireless connection, demonstrates the ‘knowledge’ we, humans, try to implement into machines to detect ourselves — awareness about these technologies and their effects (negative or positive) on our society. The result is a ghost-human face, made by mathematics and probabilities, appearing very slowly as the algorithms work over time. It is precisely the problem of false content generation by AI, be it an image, video, audio or text, and the risks of AI misuse, which this work addresses.

Video (1st exhibition, December 2018, Germany):

Picture: a mathematical result from the last exhibition, in Germany, 2018 (generated in a long take, +- 5 minutes of direct exposure to custom generated noise, developed by the author). Sample generated portrait, subject demonstrates expression, no gender is assumed by the author. Mathematically calculated from noise.

In recent years, face detection technologies have been widely used by artists to create digital art. Face detection provides new forms of interaction and allows digital artifacts to detect the presence of human beings, through video capture and facial detection, in real-time. In this work, I explore the algorithm proposed by Paul Viola and Michael Jones, presented in 2001, sixteen years ago, in order to generate imagined faces from visual randomness.

The first step of any face processing system is detecting the locations in images where faces are present (Yang, Kriegman, & Ahuja, 2002). In 2001, Paul Viola and Michael Jones introduced a machine learning approach for visual object detection which is capable of processing images extremely fast and achieving high detection rates (Viola & Jones, 2001). The Viola-Jones face detection algorithm is the first ever real-time face detection system in computer vision (Wang, 2014). OpenCV, a widely used free open-source library that enables computers to see (Bradski & Kaehler, 2008), implements a version of the Viola-Jones technique, which was extended by Rainer Lienhart and Jochen Maydt (Lienhart & Maydt, 2002). It is then used to detect objects in other images, through machine learning training processes. These objects can be faces or others, like pedestrians or eyes.

Having studied the Viola-Jones face detection algorithm, I got a very simple question: can a computer system imagine and draw a human face, having noise as a starting point?
So, I decided to use 2 computers, and their cameras. The 2 computers are looking at each other, communicating through visual signals, nothing more than that. One computer gives a signal to the other, requesting to produce noise. A sequence of aleatory and uncontrolled noise variants are presented, and false positive faces (simple isolated shapes) are found and analyzed, passing through a series of mathematical tests, and collected.

Although the title is subjectively questionable, it focuses on a very specific point: we humans have created methods and instructions so that computers can easily detect ourselves, and, in this work, this knowledge is used to generate abstract pictorial face results. I present an unusual use of the Viola-Jones technique, intended to do the opposite of what it is supposed to achieve: instead of trying to locate and capture faces, I generate facial images ‘imagined’ by a computer through the exploration of hypothetical possibilities. Ruth M. J. Byrne (2007) argues that imaginative thought is more rational than scientists have imagined and that counterfactual thoughts rely on thinking about possibilities, just as rational thoughts do. Byrne highlights that people can readily make some inferences based on the possibilities they have kept in their mind. In this work, inference based on possibilities is, in fact, happening.

We, as human beings, are able to train computers so that they can detect our faces with high precision. We may say that we are educating systems (providing them base knowledge) to recognize us through mathematical techniques and training models or supported learning algorithms. Here, noise (specific and custom visual randomized instructions developed by the author) serve as a counterfactual motif, to extract inferences based on the knowledge we store in computer systems, via analogical communication between two basic dumb machines.

This work represents reflections and questions in AI, exploiting trained learned models, unmasking data, algorithms and also knowledge. Inevitably, a new era of generated whathever multi-type forms is and will emmerge, and we, humans, need or must adapt to live with it.

Pictures of the exhibition at ISEA 2019. Korea:

Video concept (time was accelerated for this demonstration): 

Publication:
João Martinho Moura, Paulo Ferreira-Lopes (2017), Generative Face from Random Data, on “How Computers Imagine Humans”, ARTECH 2017 – 8th International Conference on Digital Arts, p85-91. ISBN: 978-1-4503-5273-4
Macau, China. ACM New York, NY, USA
under the supervision of Prof. Paulo Ferreira-Lopes
Link (ACM)
Link (SCOPUS)

Main references:

Yang, M. H., Kriegman, D. J., & Ahuja, N. (2002). Detecting faces in images: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(1), 34–58.

Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 (Vol. 1, p. I-511-I-518). IEEE Comput. Soc. 

R. M. J. Byrne, Dec. 2007. Précis of The rational imagination: how people create alternatives to reality. The Behavioral and brain sciences, vol. 30, no. 5–6, pp. 439–476. DOI: 10.1017/S0140525X07002579. ISBN: 0262025841.

Wang, Y.-Q. (2014). An Analysis of the Viola-Jones Face Detection Algorithm. Image Processing On Line, 4, 128–148. 

Bradski, G., & Kaehler, A. (2008). Learning OpenCV: Computer Vision with the OpenCV Library. OReilly Media Inc (Vol. 1). O’Reilly. 

Lienhart, R., & Maydt, J. (2002). An extended set of Haar-like features for rapid object detection. Proceedings. International Conference on Image Processing, 1, 900–903. 

more references in the paper

 

Related and previous work:
Generative Face Algorithms, João Martinho Moura, 2014
Carpax e o Homem Surdo, João Martinho Moura, 2008