An immersive project to experience a high-dimensional field of vision

Postcatálogo data

Idea, concept and coordination

Nuria Rodríguez Ortega.

VR development

Opossum Studios and Leticia Crespillo Marí

Computational image analysis

Javier de la Rosa

Editing and documentation

María Ortiz Tello. Vicerrectorado de Cultura (UMA) e iArtHIs_Lab, 2020

Production

Universidad de Málaga and IArtHis_Lab / 2020

Related projects

- Barr X Inception CNN
- ArtCatalog

Description

Image analysis using CNN Inception-UMAP-HBSCAN, virtual reality and immersive installation.

Presentation place

Exhibition hall of the Vice-Rector's Office for Culture of the University of Malaga.

Exhibitions

Catálogos desencadenados (17-12-2020, 29-01-2021). Exhibition hall of the Vice-Rectorate for Culture of the University of Malaga.

Presentations

- Congreso CIMED. I Congreso Internacional de Museos y Estrategias Digitales. UPV, March 25 and 26, 2021.
- Foro Cities & Museums. Palacio de Ferias y Congresos de Málaga, June 21-22, 2021.

Awards

CIMED 2021 Award for the best communication presented at the congress.

PUBLICATION

Poscatálogo, or how and why to physically experience a high-dimensional vector space

Here you can access the publication derived from the congress

See in DOI
Download the PDF file

Barr X Inception CNN

Visit the project website
Visit the project

ArtCatalog

Visit artcatalog website
Visit the website

Opossum Studios

Visit Opossum Studios website
Go to Opossum

Work description

The immersive installation Postcatálogo, made ad hoc for the exhibition Catálogos desencadenados organized by the Vice-Rectorate for Culture of the University of Malaga (12-17-2020, 01-29-2021 ), addresses three objectives.

1. First, it tries to answer the question of how to make the high-dimensional vector spaces associated with computational image processing intelligible for human cognition beyond the usual application of dimensionality reduction algorithms.

2. Secondly, it asks how the constitutive ordering function of catalogs will take shape in a medial ecology where the centrality that until recently occupied the book artifact is shifting towards new contexts configured by mixed realities (digital and non-digital ) that include the subject as an active entity in its ambivalent cognitive-emotional-bodily dimension.

3. Thirdly, and insofar as it allows us to pose complex intellectual problems, this project seeks to show through a concrete example the potential discursive dimension of virtual and immersive reality beyond its use as recreation, reconstruction or simulation of reality.

The Postcatalogue project is an evolution of the previous project Barr X Inception CNN, experimental project developed with the postgraduate students of the master's degree in Social Developments of Artistic Culture at the University of Malaga during the spring of 2020. Barr X Inception CNN investigates the new alterity that confronts the human being of the 21st century with a non-human «other» –the machinic other-, trying to establish possible spaces for negotiation between the human cognition and artificial cognition within the framework of what Katherine Hayles (2017) has called «cognitive assemblages».

To materialize this inquiry, we take as a starting point the well-known diagram that Alfred H. Barr (first director of the MOMA in New York) made in 1936 for the exhibition Cubism and Abstract Art. In this diagram, Alfred H. Barr instituted one of the most influential theoretical paradigms of the discourses of modernity, by establishing a genealogical narrative based on formal influences to explain the evolution of the art field during the first decades of the 20th century in the framework of his teleological conception of art.

Based on this diagram, the question we asked ourselves was the following: what differences could exist between the relationships established by Alfred H. Barr in his diagram, where various isms, poetics and authors are included, and those relationships that a AI device that operates strictly through the mathematical computation of visual data translated into numbers?

To carry out this project, we transformed this diagram into a high-dimensional space in which the works corresponding to the isms and poetics present in the Barr diagram are distributed in clusters according to the visual similarities detected by a convolutional neural network Inception. In this way, the distance between the images, transformed into numerical vectors, is significant of their greater or lesser degree of visual similarity, mathematically computed. The result of this first experiment has allowed us to reflect on the similarities and differences between human vision - with its layers of cultural, disciplinary, epistemological and subjective modeling - and artificial vision, which operates through mathematical models and probabilistic calculation.

In a second phase of this investigation, we set out to reformulate the visual space generated by CCN Inception and the UMAP distribution algorithm in an immersive habitable installation. The immersive installation makes it possible for us to wander through the visual field and physically experience the degrees of visual similarity between the images by traveling the (mathematical) distance established between them.

In relation to its starting objectives, Poscatálogo raises three possible (non-linear) narratives, which can be activated (or not) when the subject enters the installation.

In the first place, as a reformulation and new interpretation of the Barr diagram, the participating subject can select some of the isms and poetics present in the diagrammatic representation and check how their works are distributed in vector space. In this way, the differences between the genealogical narrative proposed by Barr and the functioning of the neural network Inception become clear, which, by emphasizing the processes of proximity and visual contiguity (beyond classificatory schemes), contributes to make the configuration of the artistic iconosphere more complex by inscribing the images in an ecosystem of multiple and multidirectional relationships.

Second, the participant can select one of the algorithmically generated clusters and visit it. Given that the vector space has been transformed into a habitable, passable, physically and physically experiential environment, where mathematical distances are translated into physical distances, the participating subject can become aware of the gradative-visual relationship between the images of the artistic iconosphere through as it travels through space. In this way, the subject's own walking becomes a semantic exercise that produces meaning in the process of being carried out; that is, a performative semantics, which is produced in real time, dynamically and subjectively.

Third, in a kind of suspension of the logic of thought and updating of the free play of Kant's faculties, the participating subject can recreate himself in the experience of inhabiting a field of images that unfold around him. as you walk, with no other intellectual concern beyond the enjoyment of the experience.

Finally, it cannot be ignored that Poscatálogo is, in essence also, a curatorial exercise of selection, arrangement and exhibition of artistic works in the form of digital representations carried out in co-creation with an intelligence device artificial. Consequently, Poscatálogo also raises a reflection on the new modalities of generative curatorships, and on the processes of co-creation and distributed authorship typical of contemporary creative practices based on the use of algorithms and intelligence devices. Artificial.

Ultimately, immersive experiences represent an opportunity to make complex arguments that pose intellectual challenges to human cognition. Beyond formulas based on didactic (or non-didactic) entertainment of the public, or processes of reconstruction, recreation, simulation of reality…, immersive installations can be treated as discursive territories and spaces for critical discussion in which the participating subject holds a central position.

Poscatálogo is a production of iArtHis_Lab in collaboration with Opossum Studios for the Vice-Rector's Office for Culture of the University of Malaga.

Nuria Rodríguez Ortega

DOWNLOAD

the installation and discover it

Here you can access Barr X Inception CNN, the on-site installation, so you can see what it looks like. You must download the file (453 MB), run it and enjoy it with your Virtual Reality glasses.

Download the file

Project team

Nuria Rodríguez

Idea, concept and coordination

José Gabriel Ruiz

Opossum Studios

Javier de la Rosa

Computational analysis

Leticia Crespillo

VR development

María Ortiz

Editing and documentation

Contact us

Do you want to tell us or ask us something?

Fill in the form using this link