Cinergie – Il cinema e le altre arti. N.19 (2021), 21–32
ISSN 2280-9481

Between Persuasion and Dissuasion: Narratological Meta-operativity in Augmented Experience Design

Federico BiggioUniversity of Torino (Italy) - Université Paris VIII Vincennes-Saint Denis (France)

Federico Biggio is a Ph.D. candidate in Semiotics and Media at the Department of Excellence of Philosophy and Educational Sciences of the University of Turin and at the Department of Hypermedia of the University of Paris 8 Vincennes-Saint Denis. His research areas concern the cultures of digital media, in particular those related to the such called “immersive media”. Moreover he is interested in the digital humanities and in the analysis of computational techniques in cultural and academic field. After film studies, in 2016 he graduated in Communication and Media Culture with a thesis on augmented reality and wearable technologies and he obtained an annual research grant at the Interdepartmental Department of Territory Sciences at the Polytechnic of Turin. Among his publications, Augmented consciousness: Artificial gazes fifty years after Gene Youngblood’s Expanded Cinema (2020), Toward a semiotics of augmented reality (2020), Semiotics of distances in virtual and augmented environments (2021).

Submitted: 2021-01-31 – Revised version: 2021-07-05 – Accepted: 2021-07-07 – Published: 2021-08-04

Abstract

The idea I will try to argue in this article is that a supposed “embodied media phobia” can more rightly be conceived of as an ideological catalyst of dysphoric beliefs about contemporary computational media rather than as entailing a factual risk for the future to be brought about by virtual and augmented reality. In this regard, the proliferation of sci-fi and of dystopian narratives within the contemporary film scene – prototypically represented by the TV series Black Mirror – conditions and even promotes the development of self-reflective thinking regarding the “dystopia in our daily lives” (Attimonelli and Susca 2020). Such ideology will be demystified in a further way too, by proposing a reflection around the concept of meta-operativity: according to the aesthetic theory advanced by Emilio Garroni (1977) and Pietro Montani (2014, 2018), if the interaction with embodied media can be conceived of as a process able to drive the development of meta-operative competence, then the symbolic value of dystopian stories can be understood as a strategy to foster meta-textual reading and a self-reflective interpretation of one’s own experience.

Keywords: Meta-operativity; Virtual Reality; Interactive storytelling; Empowerment; Immersiveness.

1 The theory of meta-operativity against “embodied media phobia”

In the contemporary imaginary, immersive media such as virtual, augmented and mixed reality represent the new frontier of cinematic viewing experience. However, the technologies related to such emerging media require the hybridization between users’ bodies and digital artifacts – a requirement which is often negatively regarded. In the course of the article, I will refer mainly to sci-fi dystopic narrations such as the Netflix series Black Mirror (Brooker 2011-2019) and Kiss Me First (Manson-Smith, 2018) or to experimental short films by Keiichi Matsuda (2016, 2019); regarding non-narrative texts, I will refer mainly to the philosophical field (Berardi 2014; Montani 2014; Žižek 2017) and to the fields of media studies and of media semiotics (Manovich 2006; Borelli 2020; Damone and Scelzi 2018).

All of these discourses appear as they fostered the spreading of a certain “embodied media interaction phobia.” By “embodied media interaction,” we mean the adoption of head-mounted displays such as headsets and see-through goggles, not only for audiovisual enjoyment, but for entertainment activities in general. According to Paul Dourish (2001), “embodied interaction is not a technology or a set of rules. It is a perspective on the relationship between people and systems. The questions of how it should be developed, explored and instantiated remain open research problems” (192). In particular, such questions mainly concern privacy issues and the idea that embodied technologies participate in the world they represent, sometimes by altering it. Besides, by “embodied media interaction phobia” we mean the attribution of disvalues to the practices of human-computer interaction with embodied media. For this reason, we will understand such discourses as representative of an ideological standpoint of “apocalyptic” type. The term refers to the dualism advanced by Umberto Eco (1964) between who adheres to and adopts “mass” cultural products within his/her own existence (“integrated”) and who express a critical and demystifying attitude toward those same products (“apocalyptic”).

In light of these considerations, the texts I will consider appear to put the focus not so much on privacy, but rather on the “immersiveness” of the experience and on its capacity to deceive the user’s perception. In this respect, we are moving away from a narrative logic in the strict sense, according to which immersion has to be conceived of as an intensive and oriented movement by which to enter the “story,” and are approaching a more general logic of the organization of subjective entertainment, both playful and serious.

In order to reason in an apodictic way and to understand the reasons behind such a conception, in the first section of the article, I will deal with “apocalyptic” discourses about interaction with immersive media.

Mainly, apocalyptic discourses promote a negative conception of the immersive experience by fostering the idea that the main danger for users is the “anaesthetization of sensibility.” The concept has been advanced by Pietro Montani (2014) who, by reformulating Dewey (1934), defined it as the negative effect of technical artifacts that human beings use – always and naturally – to perceive and understand the phenomenal world (i.e., the “real reality”, cf. Montani 2018). In particular, the negative effect concerns “the overall orientation of the technical design in the direction of a contraction and a channeling of feelings and, ultimately, the”drainage" of the emotional and cognitive processes that differentiate perception from sensation" (Montani 2014: 48, my translation). This line of thought has become dominant also by virtue of the widespread idea that “immersion is mentally absorbing and is a process, a change, a transition from one mental state to another [...] characterized by the decrease of critical distance from what is shown and by the increasing emotional involvement in what is happening” (Grau 2003: 13). Let’s look at a specific case. Augmented reality applications such as Ikea Place or Metro provide for the delegation of perception elaboration processes to computer vision software which, operating computationally, measure real spaces so as to return an output to the users. In the meantime, users are free to enjoy the medial experience and the sublime mathematical accuracy of the calculus. However, by delegating perceptual and operational processes to a computational entity, users can allow it to anticipate – for example through algorithmic strategies of recommendation – the contents to be experienced, ending with the production of an image that is distanced from a heterogeneous and idiosyncratic representation of the world:

as Augmented Reality will become ubiquitous, it will likely take over most aspects of our daily interactions with surrounding objects and human beings, making it practically impossible to distance ourselves from this added dimension of future society, much in the same way that most people can no more leave their house without making sure they have their mobile phones on them (Palermos, 2017: 134).

Although these ideas are at stake because of their relevance, dysphoric judgments of “apocalyptic” discourses seem to exaggerate the idea that embodied media cause the anesthetization of the user’s sensibility, especially when identifying the immersiveness of the experience to be the cause of these negative consequences.

Firstly, we will try to deepen the issue of immersiveness. In fact, while there is no doubt that embodied media interaction constitutes an advancement in the human-computer interaction paradigm, the idea that ever more sophisticated and automated devices could cause the ‘drainage’ of human sensibility and intellect seems to be related more to a certain fear of the cyborg, which may be more aptly identified as a timeless topic of the sci-fi genre than as a factually-based concern. In this respect, we will discuss the concept of presence, by understanding it as “the human reaction to immersion” (Slater 2003): it is precisely the immersiveness facet of experience that can be deemed to cause the anaesthetization of the sensibility of users.

Secondly, in order to restitute the ability of users to not be overwhelmed by technological progress, I will deal with the concept of meta-operativity. From a theoretical standpoint, Montani’s aesthetic theory stems from the one proposed in the sixties by Emilio Garroni, which focused on the concept of meta-operativity. According to Garroni (1977, 2005), meta-operativity is a human capacity and, precisely, the speakers’ linguistic competence, to think in advance and in a strategical way about an operation, before executing it. In particular, the meta-operative dimension would concern the human capacity to produce an artifact of which the end does not lie within itself, but in its capacity to create another, further artifact. The most often proposed example is that of a primate sculpting a silex in order to obtain some splinters useful for carrying out some kind of operation. Similarly, Montani proposed the idea of “interactive imagination” (Montani 2014: 75). By emphasizing its interactive character, imagination can be described as a faculty which is naturally designed for externalization in a technique. According to Montani, action in the real world by means of technological prostheses foster the imagination’s ability to develop a creative attitude (Montani 2014: 12, my translation).

With these thoughts in mind, we will try, on the one hand, to understand to what extent the “embodied media interaction phobia” of “apocalyptic” discourses can be viewed in a metaphorical way – if they fail to accurately depict the contemporary factuality of the interaction of users with embodied media, at least they can be useful to describe the users’ dealings with computational media metaphorically.

On the other hand, we will try to understand to what extent it is possible to speak of meta-operativity not only to define the technical empowerment of users by the use of embodied media, but also to define a broader self-reflective interpretation of the experience of fruition to account for cases in which it ultimately aims to foster user cognition towards choosing to act to support ethical and humanitarian causes.

2 Apology of immersiveness

Nowadays, emerging virtual as well as augmented reality media have been referred to as “immersive” media and, in some cases, as “pervasive” ones, implying a dysphoric connotation.

A serial product such as Kiss Me First (Manson-Smith, 2018), loosely inspired by the Blue Whale news phenomenon, tells the story of young men who get lost within a computer-generated world with tragic consequences. Blue Whale is remembered as a social network challenge which, since 2016, is claimed to have brought teenagers to self-harm and suicide. On the other hand, the Kiss Me First series was an adaption from a novel dealing with the cult of suicide and with online identity theft: in the course of the narrative, these two crucial themes are developed so as to stage the possible negative consequences of interaction with the embodied media. Among these consequences, there is a clear portrayal of the inability to distinguish between the real and virtual worlds, as well as between acting in the former or in the latter.

Not only has it been immersive experiences in virtual reality which have been stigmatized for “isolating” the individual in an artificial, programmed and automated environment: when Pokémon Go players from all over the world began to invade the streets in 2016 in search of fantastic creatures scattered throughout the territory, both newspapers and scientific communities expressed great concern about the decline in users’ attention to “real” reality. An article by CNN reported the episode of a man who accidentally fell off a cliff while using this application,1 and a group of doctors at the University of Padua even spoke of a form of “blindness” characterizing Pokémon Go players in public contexts where, in particular, there are other individuals not participating in the game (Barbieri et al. 2017).

In the same vein, an audiovisual product such as Hyper-Reality (Matsuda, 2016) depicts users as immersed in a pervasive augmented reality from which it is no longer possible to catch a slight of the real world. In this short film, the user is depicted as alienated and flooded by pop-up windows and interactive elements displayed in the field of view. The visual design of this short film is far from representing a transparent experience of the augmented world: the unknown protagonist doesn’t ‘see through’ the interface, she sees only the augmented world. Matsuda’s central critique again concerns the inability to distinguish between the real world and augmented reality.

Matsuda’s short film can be divided into three parts: the first shows an everyday situation of commuting by public transportation where AR technology shapes every minute and every centimetre; in the second part the protagonist experiences an attempt to hack her digital identity; and in the third part she ends up selecting a new identity. The clip occurs in three different locations: a bus, a supermarket and a street. Each place exemplifies the visual transformations AR brings, sometimes in a sharp contrast to a grey and dull reality. The price for this augmentation is an inability to look, to gaze, to decide. It is evident from the first minutes, but becomes acute towards the end of the clip (Wellner 2020).

Fig. 1 – Shots from Hyper-Reality (Matsuda 2016)

Moreover, in the second part of the short film, a blackout occurs so that every device – including that of the protagonist – turns off, leaving one with the view of a reality of gray tones in which each individual is isolated within himself/herself.

Another movie to address the topic of augmented reality with a dysphoric connotation was Auggie (Kane 2019).

Fig. 2 – Shots from Auggie (Kane 2019)

Just as in Her (Jonze, 2013), in this movie also the protagonist becomes in possession of a device – a pair of glasses – that allows the visualization of the hologram of a sensual woman programmed to respond to all the wishes of its owner. Beyond the issues of gender, on which we do not dwell, it is interesting to note that, even in this film, after the initial euphoria, the protagonist becomes prey to self-denigration. Initially, the man is excited by this presence, to the point of rediscovering the pleasure of falling in love. However, the director’s gaze is critical: it does not fail to stage the absurdity of such interaction, one leading an individual to speak towards an empty armchair or to try to touch a hologram with one’s real hand (Fig. 2).

Neither in the scientific field has the implementation of digital prostheses into the human body and the use of wearable technologies been interpreted as a technical evolution of devices aimed at optimizing the phenomenal experience, as Steve Mann (1998) argued, but rather as a technical involution.

In the not so distant future, users wearing Google­ Glass will receive information about their objects of vision directly on the screen before their eyes, in the space between themselves and those objects. In other words, Google Glass will be a wearable computer with an optical interface displaying information about objects in the user's field of vision. […] Little by little the entire world – already entirely mapped by Google maps – will be re-coded by Google Glass, so you can access those previously undergone experiences that Google Glass makes available for you. This implies that you will no longer experience the world, but, rather, that you will simply use or receive, or access) previously-experienced data about an object that is no longer the object of your own experience but purely a reference to a pre-packaged world (Berardi 2013, 20-22).

Similarly, in a very provocative article, Slavoj Žižek argued that “ideology is the original Augmented Reality” (Žižek 2017). By describing the SixthSense prototype by Pranav Mistry2, the philosopher discussed the double idea that, on the one hand, AR is not really a real cutting-edge innovation because, on the other one, it denotes a process which features the mode of working of ideology. In Žižek’s view, the object of the discourse is the ideology: it is the actual “augmented reality”.

[…] this magic effect of SixthSense does not simply represent a radical break with our everyday experience; rather, it openly stages what was always the case. That is to say: In our everyday experience of reality, the “big Other” – the dense symbolic texture of knowledge, expectations, prejudices, and so on – continuously fills in the gaps in our perception. For example, when a Western racist stumble upon a poor Arab on the street, does he not “project” a complex of such prejudices and expectations onto the Arab, and thus “perceive” him in a certain way? This is why SixthSense presents us with another case of ideology at work in technology: The device imitates and materializes the ideological mechanism of (mis)recognition which overdetermines our everyday perceptions and interactions (Žižek 2017, my italic).

Although the Žižek’s article is not a very discussion about the actual mode of working of AR, it is interesting the way in which he deals with the belief that the AR was related to a subjective distortion of the reality. AR is conceived as a medium able to alter users’ understanding of the real world, to fill in the gaps of users’ perceptions, as well as ideology does. In fact, according to Žižek, what people “(re)construct” as their own experience is already supported by judgmental decisions.

According to such ideologies, technical devices would lead to the “alienation” of the individual, in a perspective not dissimilar to that proposed by the Frankfurt School in the critique of the cultural industry (Adorno & Horkheimer 2002, Marcuse 1964). Embodied media technologies have been intended as instruments of “delegation and exemption” (Montani 2018), which receive from the human body the task of carrying out functions that users are not equipped to ensure by themselves, or “functions that the human body knows how to perform but which are considered alienable because the technical devices demonstrate that they can perform better than us, thus freeing us from different types of tasks”3 (Montani 2018: 2, my translation).

For example, as Montani argued in Tecnologie della sensibilità, an embodied medium such as Google Glass would risk describing reality just in one facet, by depriving the user of the unpredictability of the experience (Montani 2014: 90). However, as I have still argued, it is hard to conceive of embodied media as deactivators of the naturalness of human experience by sustaining the idea that the immersiveness of the experience would entail the “anesthetization of users’ sensibility.”

The first reason for this difficulty concerns the concept of presence. Several researchers have highlighted the conceptual confusion surrounding the notion of presence as well as that immersion (not only in virtual environments), although it has not long been defined in an empirical way. In fact, the notion of immersion has often been confused with that of presence. The contribution of Slater is pivotal for understanding the difference between the two notions.

Let’s reserve the term “immersion” to stand simply for what the technology delivers from an objective point of view. The more that a system delivers displays (in all sensory modalities) and tracking that preserves fidelity in relation to their equivalent real-world sensory modalities, the more that it is “immersive.” This is something that can be objectively assessed, and relates to different issues than how it is perceived by humans (Slater 2003).

According to Slater, if immersiveness is a feature of visual texts, presence is a consequence of the perception of such feature by users: “presence is a human reaction to immersion” (Slater 2003).

By referring to “apocalyptic” narratives, the first idea to foster the constitution of the “embodied media phobia” seems to be precisely the degree of immersiveness of images produced with augmented and virtual reality. When images produce a sense of immersiveness (in a virtual world, in an alternative reality), they are conceived to be alienating. However, if immersiveness is an objective quality of the texts and the concept of presence refers to an effect of meaning which users may feel while they enjoy an immersive experience, the capacity of texts to alienate users depends on the latter’s behavior in relation to such texts (and not on a feature of the text: arguing that an immersive text automatically produces a sense of presence means conceiving of human-computer interaction in a deterministic way).

To this effect, Calleja (2011) denounced readings of the term “immersion” through two incompatible concepts, in particular immersion as “absorption” and immersion as “transport.”

In this perspective, we have to deal rather with the cognitive attention of the subject as well as with the degree of the user’s sense of presence which depends on a subjective reaction to the immersive text. Bernard Guelton (2014) also warned against the risk of conceiving of immersiveness as related to technological devices, by proposing a critique of the “monolithic approach” to immersion. He argued that we have to consider “immersion” besides what he called “conscience ordinaire” (ordinary consciousness). In a situation of “immersion,” we deal with the intensity of the subject’s attention to the objects of the experience, determined by a fusion of sensory information.

En situation d’immersion réelle, l’activation de l’attention du sujet est produite directement par son environnement. Perception et action y sont étroitement corrélées. Être impliqué fortement dans une émotion, une situation urbaine ou un paysage par exemple relève d’autant d’immersions en situation réelle qui se distinguent sans trop de difficultés d’immersions fictionnelles ou virtuelles. Ici, point de représentation intentionnellement médiatisé par des tiers, de leurres ou d’artefacts. Mais l’immersion en situation réelle peut aussi être comprise en opposition à une conscience ordinaire dans laquelle l’attention du sujet est diffuse, fragmentée ou attentive à plusieurs objets à la fois (Guelton 2014: 11).

In this regard, he proposed to think of a fading attention (which however is not a prerogative of interaction with technological devices):

Prenons l’exemple banal où nous avons à réaliser plusieurs tâches simultanément: conduire une automobile, entretenir une conversation avec un passager, programmer les actions à réaliser dans les heures ou la journée qui va suivre. Il se peut également que deux tâches qui demandent le même niveau d’attention entrent en conflit et contredisent d’une autre manière toute possibilité d’une conscience immersive (Guelton 2014: 13).

Hence, he proposed to think of immersion as a “tension” characterized by the splitting and fragmentation of the subject’s attention, by a certain degree of intensity and by the neutralization of irrelevant stimuli.

The second reason for this difficulty precisely concerns the previously mentioned assumptions of aesthetic theory, according to which technical empowerment would always occur during interactions with embodied media. During the experience of virtual reality, the device’s motion-tracking sensors are able to intercept the movement of the viewer’s head and to provoke a perspective shift within the media environment represented in the text: this process is always perceived and understood by users who develop aesthetic meta-operativity. At this point, it is possible to repropose the paradigm of technical empowerment, advanced by techno-aesthetic theory (Montani 2014, 2018), which affirms the validity of the opportunities for evolutionary development offered by immersive media, on the basis of the idea that the living being is always, by its nature, technically “increased.” According to a such perspective, technical artifacts such as headsets for virtual reality, when integrated with the body as an “extension,” realize a “syncretic unity” (Montani 2018: 2, my translation), composed, in fact, of a technological prosthesis and of a human body, which ultimately becomes able to discover its own self and its own potentialities through the use of such technology.

More radically, in augmented reality experiences, users themselves have to choose, through the use of the device, the type of content to be displayed on their devices’ screens. In this view, by interfacing with a technical artifact, users become able to “discover themselves and their potentials” (Montani 2018: 2, my translation), that is, to watch themselves acting in the virtual world): this can happen “only in the course of an actual activity” (2, my translation), that is, an act of fruition. It is at this level that “aesthetic meta-operativity” occurs. Users become able to reshape the world-environment (or at least that which they may have mediately perceived) with a more or less freedom (in the manner provided by the text) and to interact semiotically with objects within it (according to the codes of meaning established by the text). In comparison to before, users enjoy of an enriched world which the technological prothesis has provided to reveal.

3 The symbolic forms of embodied media interaction

Moreover, a condition where embodied media would work as deactivators of the naturalness of human experience does not describe our current time. The immersive media with which we are dealing with in the contemporary media landscape are still “emerging media.” This means that they are not “habitual” or of daily use.

In our view, the purpose of dissuasive discourses about virtual and augmented reality can be found in their capacity to represent, in a symbolic and metaphorical way, the common dysphoric beliefs about the condition of contemporary users interacting with computational media. In fact, dissuasive discourses lead to conceive of immersive fruition as an “isolation” in an essentially subjective (user-oriented) and self-referential environment. If we look at the conceptual image to which such discourses refer, it seems they pounce upon the tropes of the “filter bubble” (Parisier 2011). In a such view, virtual and augmented reality depictions in “apocalyptic” discourses correspond to meaningful forms so as to convene a more consolidated and objective counter-ideology towards the role of the computational architectures of social media platforms in user interactions with digital media. Prevailing over the optimization of individual experience involves the alienation of ideological self-referentiality, this being allowed by the algorithmic logic that presides over the organization of the so-called echo-chambers (Persily et al. 2020), and it may be described as the progressive removal or rejection of the real dimension of existence, as represented for instance by the phenomenon of the “hikikomori” (which designates a social withdrawal leading people to social isolation and self-confinement; often such praxis is related to an excessive use of digital devices and social media).

The same can be argued about critical discourses against contemporary media experiences afforded by interactive storytelling products that promise to ensure a participatory and “active” role for the viewer. The fruition model established by these textual forms is purely participatory and cooperative (Eco 1979): the reader becomes co-author of the text and the navigation of the “possible world” is structured in a hypertextual way and is closely linked to the pleasure of control (Grodal 2000) that the user experiences during the fruition.

However, although its mode of fruition has constituted a novelty in comparison to other non-interactive products, an interactive storytelling product such as Bandersnatch (Slade, 2018) – the interactive storytelling episode of Black Mirror – has been equally considered in a negative way: in particular, it was argued that the ability to determine the flux of the narration is merely an illusion (Elnahla 2018; Montani 2019; D’Aloia 2020).

However, in accordance with the ideas exposed earlier, the same illusion of being able to direct the story in an ever-original way could be interpreted as a rhetorical strategy of the text. In this view, aesthetics meta-operativity works to “reveal” the control and recommendation mechanisms that characterize the internal organization of the web-based media environments in which the user of contemporary digital media navigates on a daily basis. This meta-experiential function, which stems from a meta-textual interpretation of the interactive movie, is still comparable to the dissuasive rhetoric previously considered. This form of implicit critique could be interpreted as a way to stimulate users to develop a narratological meta-operativity4: the text provides for its own meta-textual reading, by inviting users to develop a self-reflective interpretation of the experience of fruition (the one depicted in the story as well as the one enjoyed by the users themselves). In this view, the text would represent a sort of “antidote” which can be useful during interactions with more consolidated computational media. That is, it helps users develop a meta-experiential competence that endows them with the capability, which is an evolutionarily advantageous one, to critically reflect on their own experience of web-surfing.

4 Narratological meta-operativity in the immersive documentary

Finally, it is hard to conceive of embodied media as deactivators of the naturalness of human experience because, in some cases, the interaction with such media has the objective of fostering empathy in viewers, who are understood by the text as being not mere receivers of a cultural product, but as social and active subjects in the real world.

In narrative texts related to the documentary genre, the authorial intention to elicit a meta-experiential competence in the viewers that drive them to reflect not only on the type of technical experience in which they are involved (aesthetical meta-operativity) but, more generally, on the cognitive and evolutionary possibilities offered by the technically increased experience (narratological meta-operativity) is more evident.

The documentary genre among immersive media is well represented by the virtual and augmented reality cinema of Gabo Arora and Chris Milk, with works such as Clouds Over Sidra (2015), or by the interactive storytelling forms in augmented reality such as Outthink Hidden (New York Times, 2017).

Clouds Over Sidra, a short film commissioned by the United Nations, tells the story of a girl who lives in a refugee camp in Jordan. The primary purpose of the audiovisual product was not so much to “tell a story” in order to entertain a general public as it was to rather document the precarious reality of the places depicted through the representation of a day in the life of an ordinary person. As the filmmakers have argued, virtual reality has been interpreted in terms of being a “machine of empathy” (Milk 2015) which will raise public awareness and give the viewer a close view upon phenomena that are both current, distant and difficult to understand for the Western world.

On the other hand, Outthink Hidden is an augmented reality application developed by the New York Times Company’s T Brand Studio released at the launch of Hidden Figures (Melfi 2017). It offered location-based stories of remarkable but relatively unknown scientists: users could place virtual statues in ten cities around the US. Here, again, what is at stake is the willingness to restore visibility to previously discriminated figures.

As in Clouds Over Sidra, or even in Outthink Hidden, the crucial point of the spectatorial experience lies in the opportunity to put “real reality” on stage, enriched by the immersiveness of the image and the sense of presence that flows from it. It is not the very possible or alternative worlds depicted by “apocalyptic” narrations, but an actual augmented world in comparison to the world experienced prior to the viewing process, one believed to represent an improvement over the world as seen prior to the viewing experience, because at least users know something more about the world they inhabit. In this sense, it is precisely immersiveness that leads to empathy: these textual forms are able to “create an empathy functional to the fruition of ethical themes, mixing narration and attraction together” (Dalpozzo 2018: 103).

Obviously, the documentary genre is a borderline case and the “illusion of non-mediation” (Lombard & Ditton 1997), which features the immersiveness of the text, denotes a strategy of enunciation which is anchored to the narrative dimension set by the authors. In this view, the meta-operational competence that users can develop has to do with their own ability to use the text in the way foreseen by its technicalities (aesthetical meta-operativity).

Nonetheless, on the other hand, these examples are similar to dissuasive journalistic narrations and interactive storytelling texts insofar as they provide for their own meta-textual reading, by inviting the user to develop a self-reflective interpretation of the same real world inhabited by the viewers (narratological meta-operativity).

Hence, as the dissuasive journalistic narrations warn (and in a certain sense denounce) the “anesthetized” condition of the contemporary digital user, so the documentary texts of this type, by leveraging the expressive possibilities offered by immersive media, challenge the viewers by trying to persuade them to act ethically. The meta-operational value of this discursive operation lies precisely in the ability to reflect on one’s own phenomenal experience in relation to the represented objects and to reorganize this relationship on the basis of the new knowledge obtained during fruition.

5 Conclusions

In the course of this article, I have tried to demystify some dysphoric beliefs which feature the cultural imagery of embodied media interaction. Apocalyptic narrations and journalistic sensationalist rhetoric fail to accurately describe the factual reality of human-computer interaction taking place during the experience of embodied media.

To this effect, I reproposed the aesthetic theory on meta-operativity and technical empowerment. In its most successful applications, augmented reality has proven to be a medium capable of optimizing the phenomenal experience of the world, without anesthetizing the emotional involvement of the user. One of the meta-operational advantages that augmented reality experiences bring is, for example, the possibility of predicting a future situation before it is actually realized (this is the case with marketing applications). This concerns the implementation of immersive tools in everyday life and, consequently, the creativity of the uses that can be made of these tools.

Furthermore, I have argued that such texts can also be studied in order to determine their symbolical and metaphorical value, as they can be understood as depictions of a set of fears about more consolidated computational media.

Therefore, whether the object of fruition is a “real” and material object, one that is physically present, that the dysphoric narration allows to approach in a critical way, or whether it is a distant and exotic place, which the artifact allows to bring hic et nunc and to be enjoyed illusorily and immediately, it seems fair to say that the hypothesis of a technical and aesthetical empowerment can be valid for assessing the realization of a syncretic unit that produces the development of a meta-experiential competence in the individual.

In this perspective, immersive media cease to be conceived of merely as devices of fruition, but rather present themselves as creative and functional devices able to increase human imagination and to bring an experiential gain to the phenomenal experience in which the very protagonists of a potential narrative are the users themselves. The texts call the viewers to reflect about the ethics of the production of digital systems as well as to express their individual creativity by using the technical artifacts.

To conclude, it is not only science fiction and journalistic narrations that promote the development of a meta-experiential competence oriented towards the correct interpretation of a phenomenon of digital culture. It is instead also the ability of technical devices and of interactive textual forms, in their generality, to promote the development of a narratological meta-competence – which is evolutionarily advantageous – in the context of the phenomenal experience that can first be imagined narratively and, subsequently, intentionally used for creative purposes by users. This ultimately fuels the redesign and evolution of the devices involved and of the semiotic systems of interactive textual forms.

References

Attimonelli, Claudia and Vincenzo Susca (2020). Un oscuro riflettere: Black Mirror e l’aurora digitale. Milan: Mimesis.

Barbieri, Stefania and Vincenzo Pierantonio (2017). “Pedestrian Inattention Blindness While Playing Pokémon Go as an Emerging Health-Risk Behavior: A Case Report.” JMed Internet Res 19(4): 331-339. https://doi.org/10.2196/jmir.6596.

Berardi, Franco “Bifo” (2013). Neuro-Totalitarianism in Techno-maya Goog-colonization of Experience and Neuro-plastic Alternative. Los Angeles: Semiotext(e).

Borelli, Giorgio (2018). “Realtà aumentata e general intellect. Ipotesi per una semiotica materialistica degli smartmedia.” E|C XXII(23): 82-39. https://mimesisjournals.com/ojs/index.php/ec/article/view/522 (last accessed 6-07-2021).

Calleja, Gordon (2011). In–Game. From Immersion to Incorporation. Cambridge, MA: The MIT Press.

D’Aloia, Adriano (2020). “Against interactivity. Phenomenological notes on Black Mirror: Bandersnatch.” Series. International Journal of TV Serial Narratives VI(2): 21-32. https://doi.org/10.6092/issn.2421-454X/11410.

Dalpozzo, Cristiano (2018). “Cinema e realtà virtuale, ovvero ‘The early virtual (post)cinema of attractions’.La realtà virtuale. Dispositivi, estetiche, immagini, edited by Cristiano Dalpozzo, Federica Negri and Arianna Novaga, 87-105. Milan: Mimesis.

Damone, Giuseppe and Raffaella Scelzi (2018). “The hidden media in augmented reality.” E|C XII(23), 75-81. https://mimesisjournals.com/ojs/index.php/ec/article/view/521. (last accessed 6-07-2021).

Dewey, John (1934). Art as Experience. New York: Balch & Company.

Dourish, Paul (2001). Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA: The MIT Press.

Eco, Umberto (1964). Apocalittici e integrati. Milan: Bompiani.

Eco, Umberto (1979). Lector in fabula. Milan: Bompiani.

Elnahla, Nada (2019). “Black Mirror: Bandersnatch and How Netflix Manipulates Us, the New Gods.” Consumption Markets & Culture 23(5): 506-511. https://doi.org/10.1080/10253866.2019.1653288.

Garroni, Emilio (1977). Ricognizione della semiotica. Rome: Officina edizioni.

Garroni, Emilio (2005). Immagine, linguaggio, figura. Rome-Bari: Laterza.

Grau, Oliver (2003). Virtual Art. From illusion to immersion. Cambridge, MA: The MIT Press.

Guelton, Berndard (2014). Les figures de l’immersion. Rennes: Presses Universitaires de Rennes.

Horkheimer, Max and Adorno Theodor W. (2002). Dialectic of Enlightenment: Philosophical Fragments. Edited by Gunzelin Schmid Noerr. Translated by Edmund Jephcott. Stanford: Stanford University Press.

Lombard, Matthew and Theresa Ditton (1997). “At the Heart of It All: The Concept of Presence.” Journal of Computer-Mediated Communication 3(2). http://jcmc.indiana .edu/vol3/issue2/lombard.html (last accessed 6-07-2021).

Mann, Steve (1998). “Humanistic computing: ‘WearComp’ as a new framework and application for intelligent signal processing.” Proceedings of the IEEE 86(11): 2123-2151.

Manovich, Lev (2006). “The Poetics of Augmented Space.” Visual Communication 5(2): 219-240.

Marcuse, Herbert (1964). One-Dimensional Man: Studies in the Ideology of Advanced Industrial Society. Boston: Beacon Press.

Milk, Chris (2015). “How virtual reality can create the ultimate empathy machine.” TED Talk: https://www.ted.com/talks/chris_milk_how_virtual_reality_can_create_the_ultimate_empathy_machine (last accessed 6-07-2021).

Montani, Pietro (2014). Tecnologie della sensibilità. Estetica e immaginazione interattiva. Milan: Raffaello Cortina.

Montani, Pietro (2018). “Empowerment tecnico e assunzione di responsabilità. Verso un’etica del digitale.” Lessico di Etica Pubblica IX(1): 1-10.

Montani, Pietro (2019). “Elogio in 10 punti di Bandersnatch.” Fata Morgana Web, January 14, https://www.fatamorganaweb.it/bandersnatch-black-mirror/ (last accessed 6-07-2021).

Palermos, Spyridon Orestis (2017). “Augmented Skepticism: The Epistemological Design of Augmented Reality.” In Augmented reality. Reflections on its Contribution to Knowledge Formation, edited by José María Ariso, 133-150. Berlin: De Gruyter.

Pariser, Eli (2011). The Filter Bubble. What the Internet is Hiding from You. London: Penguin Press.

Persily, Nathaniel, Tucker Joshua and Barberà Pablo (2020). “Social media, echo chambers, and political polarization.” In Social Media and Democracy: The State of the Field, Prospects for Reform, edited by Persily Nathaniel and Tucker Joshua, 34-55. Cambridge: Cambridge University Press.

Slater, Mel (2003). “A Note on Presence Terminology.” http://www0.cs.ucl.ac.uk/research/vr/Projects/Presencia/ConsortiumPublications/ucl_cs_papers/presence-terminology.htm (last accessed 6-07-2021).

Wellner, Galit (2018). “Posthuman Imagination: From Modernity to Augmented Reality”. Journal of Posthuman Studies 2(1): 45-66.

Žižek, Slavoj (2017). “Ideology Is the Original Augmented Reality.” https://nautil.us/issue/54/the-unspoken/ideology-is-the-original-augmented-reality (last accessed 6-07-2021).


  1. https://edition.cnn.com/2016/07/15/health/pokemon-go-players-fall-down-cliff/index.html (last accessed 6-07-2021).↩︎

  2. SixthSense is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. Retrieved from: https://www.pranavmistry.com/archived/projects/sixthsense/ (last accessed 6-07-2021).↩︎

  3. The functions that the human being can perform and that have been delegated to technical devices are those of imagination. Montani (2014) spoke of interactive imagination.↩︎

  4. We would like to make the concept of “meta-experiential competence” coincide with the result of what, for semiotic aesthetics, is the meta-operational performance. Meta-operativity, which is exclusively the prerogative of human beings, is an operation, an activity that produces an instrument that has no end in the instrument itself, but in creating a further instrument through the first. The technological and techno-textual systems that we will analyze in this contribution are technical artifacts produced within the contemporary media environment with the aim of bringing the human being to evolve (cognitively and intellectually).↩︎