Tuning one and other

ABSTRACT

Being and bearing witness is foundational to social structures. Different kinds of mediated presence, which have been introduced by information and communication technology in the twentieth century, affect how people are witness to each other. To better understand trade-offs for trust, the so-called YUTPA framework is developed, which allows for identifying different presence configurations. As a techno biography, the memory of four live events is re-visited to explore the force of mediating presence for being and bearing witness in the new presence configurations that have emerged in the use of information and communication technology. These live events happened in different decades with different levels of technology and all represent a threshold in the developing media schemata in the Amsterdam digital cultural scene since the 1980s. This article argues that for being and bearing witness in mediated presence, psychological capacities for processes of imagination, attribution and immediate adaptation are pivotal. Witnessing the self appears to be a key capacity in these processes. While steering towards well-being and survival, ethical positions that reflect trade-offs for trust and truth, are chosen. It is the physical presence of the witness as the beholder of sensations, emotions and feelings, that steers towards well-being and survival.

KEYWORDS:

• Trust
• presence
• body
• performance
• networked events

1. Introduction: a technobiography of breaking media schemata

In the last three decades information and communication structures between people, organizations, businesses and utilities, have deeply changed because of information and communication technology. The everyday relationship of millions of people with technology has been deeply affected by the speed and scale of these new technologies. Especially since the COVID-19 pandemic when so many societies went into lockdown, millions of people acquired experience of online presence together. Communicating in a mediated presence has become a mundane everyday experience around the world. Early artistic experiments with these technologies carried a beauty and depth that are not visible anymore in the many mundane interfaces on mobile phones and computer screens which offer communication with other places in other time zones to millions of human beings today. In these early experiments, however, fundamental questions about the potential use of these technologies were raised which are relevant today.

People have been telling each other stories about the new experiences these technologies offered them. From these stories, techno-biographies can be written that shed light on seemingly minuscule changes that indicate fundamental changes in society.¹ This contribution can be read as a specific techno-biography that focuses on the building of trust in live online networked events that changed the media schemata of those involved in the small digital cultural elite of Amsterdam since the 1980s. Participation in these live online networked events triggered many stories about what happened, how people felt, and about which new possibilities could be imagined because of what just happened. These stories constitute techno-biographies of the emergent network society at the time. These stories also changed the media schemata which define what is acceptable to larger audiences as part of our reality and in what way.² The way youngsters build a social reality with a mobile phone for example differs from how the elderly do. Media schemata are not only socially, economically and culturally but also generation-dependent.³

The live online networked events that are discussed below, all challenged the media schemata of this avant-garde art scene in Amsterdam at the time. In the technological avant-garde arts scenes, often designers were present who worked with smaller tech companies that worked with larger media & tech companies and so the ideas and experiments of artists also challenged in the end the large global technology developers.⁴ The author of this contribution was part of these developments. Sometimes she initiated, orchestrated and produced such events. In other moments she was just a visitor. As a result of her involvement in this avant-garde art scene in Amsterdam, she was witness to specific key socio/technical inflection points in which new imagination and new potential became visible to become ubiquitous only a decade later.⁵

Artists study the way experiences can be created whether in paint, stone or ICT. They are experts in creating experiences for others. In the processes of artistic research, three qualities are offered: radical realism, non-conceptualism and contingency. One can argue that through the ‘supply chain’ of new ideas, artistic research has had a significant impact on the development of information and communication technology.⁶ Only those who were present though can testify about how live online networked events challenged the status quo of media schemata at the time. As such this contribution can also be read as an ‘act of parrhesia’ because the author of this contribution was part of these developments as a visitor, an orchestrator, a producer, and a cultural entrepreneur. The author has the subject position to testify about what happened.⁷

In this contribution, the focus of the techno-biography is on how trust is built in the new media schemata that emerge in mediated presence. To this end, the YUTPA framework is introduced. Different online networked events are discussed which each caused a break in current media schemata to introduce a new potential presence configuration. They illustrate how mediated presence caused great joy by the sheer possibility of connection. After a while, it also became clear how mediated presence can cause real stress and hurt. New possibilities of acting in mediated presence while remaining to be anonymous, appeared to be very detrimental. Fundamental to being and bearing witness in mediated presence in benevolent ways is to develop the capacity to engage with the flow that technology offers to be able to listen and tune to others and experience togetherness as a result.

This article argues that psychological capacities of imagination, attribution and immediate adaptation are core to being and bearing witness in mediated presence. Witnessing the self appears to be a key capacity in processes of attribution and adaptation while steering towards well-being and survival. Being witness, bearing witness and self-witnessing are the grounds upon which ethical positions, that reflect trade-offs for trust and truth, are chosen.

2. YUTPA: to be with you in unity of time, place and action

Mediating presence has had many spiritual forms in a great variety of human cultures over centuries. Technology has facilitated in the last hundred years a more mundane and very literal mediating of presence by transmitting sound, movement and light in endless variations. Digital technology has added to this mediation an immediacy in scale and speed that was never possible before. However, even though technology has deeply affected us, human beings remain the same. Our physical body is born, resides in places and after a shorter or longer time, it will die. With our mind, we can go anywhere, but our body is part of the physical world and ultimately defines our capacity to engage in any other world.

Presence can be defined as the capacity to steer towards well-being and survival and we do this based on sensations, emotions and more complex feelings.⁸ When the heater is too hot, we pull away our hands. When the alley does not smell good, we take another route. When one has a really good time in a park, one chooses to go again to that specific park location. Even complex feelings like solidarity, compassion or risk assessments influence how we steer and perform presence. In the act of steering towards well-being and survival, we make trade-offs between what we know and do not know, between what we like and do not like, between our perceived vulnerability and our assessment of the trustworthiness of the environment we find ourselves in.⁹

In the dissertation Presence and the Design of Trust (Nevejan 2007) a granular analysis of two networked events, which happened in 1989 and 1990 in Paradiso Amsterdam, led to the conclusion that three kinds of presence define trade-offs for trust and truth:

• natural presence, defined by our physical being

• mediated presence, defined by mediation through technology

• witnessed presence, defined by how we are witnesses to one another in both natural and mediated presence

Any presence configuration is the result of the interaction between these three kinds of presence. Seemingly effortlessly millions of people navigate the increasingly complex presence configurations that have become part of professional and personal environments. To understand the trade-offs that happen in the different presence configurations, the dissertation introduces the YUTPA framework.¹⁰

The original way of being together with other human beings is when our bodies are physically in the same space, we share time, are in a relationship with each other and act together. This is how human societies have structured social relations, transactions and judicial frameworks for centuries. To be with You in Unity of Time, Place and Action (an acronym for YUTPA) is the core configuration of human togetherness.¹¹ Here we love and laugh, we are born and die, here we negotiate our presence, our trust and our truths. We make trade-offs in trust and we do this in the four dimensions of YUTPA. Sometimes this takes a split second, sometimes it is the result of longer consideration. Trust deeply affects how we perform our presence.

Digital technology offers many new configurations in which the four dimensions of Time, Place, Action and Relation create different Unities in which we can be present by using the human capacity to imagine, attribute and adapt.¹² In different YUTPA configurations trade-offs of trust are made in different ways and as humans over the years, we learn how to make different trade-offs of trust in these configurations. Digital technology offers a new level of agency to many people in many formats (contexts). When analysing what happens through the lens of YUTPA, one analyses what happens in each of the dimensions of Time, Place, Relation and Action in relation to one another.

Following Damasio, one can argue that the acceptance of YUTPA trade-offs for trust happens on three levels of consciousness.¹³ Proto consciousness informs us about the state of our own body, core consciousness informs us about the environment in which we find ourselves, and extended consciousness includes all that we know and experienced before. In each layer of consciousness the four dimensions of the YUTPA framework manifest and affect trade-offs for trust and truth.

In the last decades, several artistic experiments in live online networked events deliberately challenged the interaction between these different levels of consciousness in the exploration of new YUTPA configurations. These experiments were pivotal for the understanding of what was happening because of the ubiquitous introduction of information and communication technologies. Several of such significant moments in the technobiography of the author of this contribution, who was part of a technological avant-garde in Europe at the time, are described below. In an act of parrhesia, she bears witness to what happened in the dramatic moments when new YUTPA configurations emerged.

3. Bridging trust: making a connection beyond what is deemed possible

In the first decades of these new network technologies, both artistic research and presence research were deeply concerned with establishing the ‘realness’ of the new YUTPA configurations. Exploring human agency in these new mediated environments leads to a variety of social presence or co-presence experiences.¹⁴ Even before digital technology became ubiquitous, an old-fashioned phone line offered an impressive moment of the realness of togetherness in mediated interaction.¹⁵

1989: Czech Mediated Clapping Jam

It was a rainy cold night on 6 February in Paradiso in 1989, a 19th-century church building, which was now a music hall of international renown. About 350 people gathered to take part in an evening on Czech Underground in collaboration with Charta’77 and AIDA both support organizations for artists facing political prosecution. The Berlin Wall had not come down yet and the repression was intense in those last months before all these regimes would crumble down.

Charta ‘77 was an organisation by and for political refugees from Czechoslovakia consisting mostly of artists and intellectuals. The MC of the evening was the chair of the Dutch Union of Artists, Hans Boswinkel, announcing speakers and musicians who were all showing their solidarity by performing for free. At one moment it was planned to phone Prague where about 20 dissidents had gathered in a living room for the occasion.

It was the days before the Internet was widely known, there were no mobile phones and there was no Skype, Teams or Zoom. However, through a special switchboard, it was possible to establish a connection with a normal telephone, which was connected to what was called ‘the fork’ and transmitted the phone sound via the PA system into the hall.¹⁶

The Czech refugees in the hall phoned their friends in Prague and the phone rang over the PA system loud and clear in the grand hall of Paradiso. Then it was picked up and the audience heard the voice of a person in Prague. The MC described that over 300 people were gathered in Paradiso who all stood in solidarity with them. In response, the people gathered in the Prague living room applauded. This was heard over the PA system in Paradiso. In response the Amsterdam crowd applauded after which the Prague living room made a rhythm in clapping and Amsterdam answered copying the rhythm in Prague, adding a few claps extra, which was then copied by Prague and vice versa again and again and again … For the next 10 minutes or so, it was a radiant interaction of clapping between the living room in Prague and the audience in Paradiso. All the words that could not be spoken, all the fear, anger, courage and sadness that comes with oppression came together in this meaningful connection over the old telephone line offering a lavish expression of the joy of being in touch and affirming our collective humanity, be it via wires, through the iron curtain anyhow, by a reciprocal clapping of hands.

The simple phone line connection made it possible to witness each other in a limited audio space only for 10 min, yet the effect of being witness to each other through the iron curtain was huge. The phone line permitted both audiences to synchronously share time (Now), even though they did not share a place that evening (Not here). Some people in both audiences were friends, they had shared place and time and relations and activities before (You). When the audience in Paradiso saw the joy on the faces here when they heard the voices of their friends there, the bridge of trust was built. The friends in Paradiso became what was then called the ‘social interface’ for the larger audience in Paradiso.¹⁷ Through the joy on the faces here, the whole audience became witnesses to others far away in the living room in Prague.¹⁸ The relationship became even more tangible when both audiences engaged in a shared action adapting immediately to the emergent improvisation (Do). The reciprocal clapping between the two distant spaces in one synchronous audio space created the experience of being together in a shared sphere of real interaction.¹⁹ Because of the relationship between the Prague friends, all the others in the two distant spaces could join in the joy of the connection. Despite the low bandwidth and the latency, the YUTPA configuration of Not-Here/Now/Act/ You mediated the trusted relationship. Because the audience experienced the connection between the friends (You), trust spread as it were to all present in the space with either one of them. Having a social interface, showing a relation for bridging the trusted relation, is pivotal for connecting audiences in real live events. Human imagination underlies this capacity to attribute and adapt to a new configuration of interaction that emerges as a result of technology.²⁰ In the 1989 Mediated Clapping Jam the audience participated in solidarity with those who were suffering from fear and oppression and with the help of the phone lines this generated the joy of connection.

4. Mediated presence is real

The 1995 Ping Body performance of Stelarc showed a very different effect of being a witness in a live online networked event. In this online performance, the presenter of the conference could act and influence what happened to a human body elsewhere in a very direct manner that was facilitated by technology.²¹

Ping Body, Telepolis Luxembourg 1995

Stelarc, an Australian artist who focuses on the posthuman, offered a shocking experience of mediated presence experience when he was performing during the Telepolis festival in Luxembourg while being wired up to three other podia in Paris, Helsinki and Amsterdam.²² In Amsterdam during the Doors of Perception conference, the audience saw Stelarc’s body in Luxembourg on a huge screen. His body was covered with electrical wires and through the computer the remote locations in Paris, Helsinki and Amsterdam could activate these electrical points and, without Stelarc’s willing interference, could make his arms and his legs move. When someone tapped on the button of his arm on the screen in Paradiso Amsterdam, Stelarc’s arm would rise on the stage at Telepolis in Luxembourg. When someone tapped the image of his leg in Helsinki, his legs would start to lift. In the first minutes of the performance, the audience enjoyed seeing this new possibility of connectivity. Then the remote tapping became more intense, quicker and quicker. The remote audiences could hear the breath of Stelarc going faster and faster. Even though he did not trigger the movement, it was exhausting him just as much. Stelarc challenged the remote locations to tap quicker and quicker. More and more people in the audience became uncomfortable. There was an agreement with Stelarc that he would signal when the movement would be too intense, but he did not and the remote tapping kept on going. It was also unclear which location was responsible for which specific movement by now. The whole interaction looked more and more like public torture and more and more people in the audience started to leave. In the end, the remote locations could not handle it anymore, stopped tapping and the performance was over.

Stelarc offered a distinct YUTPA configuration. Based on synchronous communication, the performance and the audience in the 4 locations shared the Now, while at the same time creating semi-anonymity between the different halls (Not-You). Stelarc built his relationship with the audiences in minutes using the presenters as his ‘social interface’ with the people at Telepolis in Luxemburg, Paradiso in Amsterdam, the Pompidou Centre in Paris and the Media lab in Helsinki (You). While not being in the same place physically, the synchronous audio (Now) and synchronicity of action (Do) that was witnessed by four audiences, made a very strong mediated presence. Stelarc offered his own body to this YUTPA configuration giving up his possibility to act and offering this to the 3 other audiences for which he made the act of witnessing and the act of triggering automatic physical movement elsewhere the core of the experience. (Now, Act, Not-You, Not-Here). For the audiences, Stelarc’s Ping Body was a highly confrontational experience because it showed the potential of detrimental agency in mediated presence. While only the presenters were communicating with Stelarc, it showed the peer pressure of an audience that obeys the rules of the performance to be quiet and wait, while the presenters kept on tapping while the screen and audio clearly showed Stelarc’s distress. It also became evident how hard it is to endure to see the pain of others in real-time in a shared audio and visually mediated space when one does not know each other.²³ Being witness to pain and exhaustion while being part of the action, even as an audience, is very hard to endure. When being witness to pain, even in a performance context and with consent, this triggers our steering towards well-being and survival and people want to get away from such pain.

Finding that human agency can be as detrimental through mediated presence as in real presence, has been a huge incentive for military developments in the last decades. As of 2001 according to the New York Times, drone pilots who were located in the United States, killed hundreds of people while sitting comfortably in their home town without facing any risk.²⁴ Using the drone to observe their victims for weeks sometimes, the pilots started to feel more and more of a relationship with the victims so the eventual killing of them induced great remorse. Distance (Not-Here) and anonymity (Not-You) create a moral void in which anything can happen even when real people are visibly real-time involved (Now). Where YUTPA trade-offs for trust generate potentially supportive environments, once the balance goes to no trust, anything is possible. When unity is broken, when being and bearing witness is anonymous, our immediate ethical sense disappears and the bestiality of humankind can prevail.²⁵ On a longer period, however, not only the most direct victims of technological warfare faced terrible consequences. Many drone pilots have had serious post-traumatic stress symptoms that affected their personal lives in very detrimental ways as a result of this unrecognized effect of mediated presence. Seeing pain in others is hard to endure, even in a mediated presence. Acting in a mediated presence appears to be very real, it can even generate traumatic experiences.

5. Detrimental trust: anonymous-mediated presence shatters accountability

Challenging new media schemata from an artistic perspective Martin Butler²⁶ created the Girlfriend Experiment in 2007 in Mediamatic Amsterdam.²⁷ From 2005 the online landscape had changed significantly with the ubiquitous spread of social networks, including the large multiplayer online environment, like for example Second Life. Here people interacted with avatars with each other.²⁸

2005 The Girlfriend Experience – Martin Butler

The Girlfriend Experiment is an Online multi-player game with real-life avatars of flesh and blood. The audience sees some rooms behind a wall of glass in which a group of actors/friends are gathered. They all wear white clothes and little ear microphones. They are filmed and simultaneously broadcast live on the Internet. The audience can hear what they say.

People on the Internet can adopt one of the actors/friends in the house as an avatar and tell him or her what to do via the little earphones. The avatar-owner, the actor and the audience all hear the live audio in the house (Now). Actors do not hear each other’s instructions through the little earphones, nor does the audience hear what the ‘avatar-owner’ demands from the actor he or she is instructing. So there is a void in communication in which certain cracks start to unfold and actors start to do unexpected things that cannot be anticipated. Only the avatar-owner can act, according to the rules (Do), while the actors and the audience cannot act (Not-Do). Only the Internet avatar owner is elsewhere (Not-Here) while the audience and the actors are in one place (Here). There was also an anonymous Internet audience (Not-You) that could hear and see (Now) but was elsewhere (Not-Here) and could not do anything (Not-Do).

Being in the audience of the Girlfriend experience was a special experience because the movement was sometimes very slow and information very limited. The avatar-owners were an anonymous driving force and sometimes one could see the actors being confused. The avatar owners started to ask the actors/friends to execute sexual acts. What seemed experimental, interesting and edgy, soon slipped into violent requests and the actors refused to play. Trying to keep up with the one-to-one relationship with the avatar-owner, the avatar-actors tried to talk back and refused to do what was asked while keeping up the performance. Finally, they stepped out of their role and the gameplay was over.

On reaching the threshold of what the actors were willing to accept, it was clear that the rules had to be changed. The anonymous internet avatar owners did not share a place (Not-Here) and were not in a relationship with the actors (Not-You). Also, the audience did not know who the internet avatar owners were. The anonymity offered a disguise for acting out without taking responsibility for their behaviour. Without having a relationship with the actors, without having to face physically what happened to the actors, without having a space with the audience and without peer pressure to obey and behave, the anonymous internet avatar owners easily slipped into abuse of power. Like other social structures, mediated communication needs to be subject to rules, responsibility and accountability.

The Girlfriend Experience showed how anonymity in online-mediated communication environments easily shatters accountability and responsibility for one’s actions. After 2005, when social networks became a ubiquitous feature in social life, their risks became overall apparent. Bullying and disinformation have become a paramount feature of social networks next to the great pleasure of being connected with colleagues, friends and family. While the concept of creating architectures of trust in social networks requires a personal choice to befriend every person who follows another person, the harvesting of data of millions of people by a single company creates new opportunities for influencing people’s voting behaviour for example, as Cambridge Analytica scandal around the 2016 USA elections showed.²⁹ Manipulating advertizements, software bots and fake accounts as well as online group pressures result in detrimental behaviour to specific individuals and disinformation to many. Because this artistic research experiment used people of flesh and blood to be avatars, the underlying dynamic that is caused by anonymity surfaced. In anonymity easily others can be dehumanized.

6. Trusting the flow: tuning to the rhythm of other

Once architectures of trust are built in which the joy of connection is coupled with structures for responsibility and accountability, mediated presence can have a significant impact in many local contexts as the ubiquitous spread of the Internet has shown. During the COVID-19 pandemic, the Venice Exploratorium took place as part of a research program that was called ‘Values for Survival’ to address the need for new values to survive climate change as humankind and which took place in the context of the complementary research program for the Dutch contribution to the 17th Architecture Exhibition of La Biennale di Venezia.³⁰ In the Venice Exploratorium over 50 artists and scientists collaborated in elevent teams In these well-documented collaborations. It appeared that media schemata and YUTPA configurations are today to be negotiated when one embarks on a collaboration. The positioning of specific roles for documentation, orchestration and reflection is in the context of mediated presence, fundamental for successful online networked events that take place in the context of diverse collaborations. It also appeared that each of the teams had developed a specific rhythm in creative exchanges which participants indicated as a source for growing trust between the members of the teams. Rhythm deeply affects trade-offs for trust and can result in a strong sense of togetherness in online communication.

As an ultimate experiment to understand the emergence of trust through rhythm, an online concert between three musicians from different rhythm traditions was carried out for the launch of the Venice Exploratorium.³¹ The musicians did not know each other and could not meet and had to deal with a significant latency because their locations were far apart. When rhythm is foundational to trust, can musicians play together online when they do not share a rhythm tradition, do not know each other and have to overcome a serious latency in every improvisation? ³²

2021 Tuning to the rhythm of others – Mistah Issac, Sirishkumar Manji, Reinier van Houdt

Mistah Issac (guitar and vocals) coming from Angola and living in Lisbon, Sirishkumar Manji (table and vocal) coming from India and living in London and Reinier van Houdt (modern piano) coming from the Netherlands living in Rotterdam, agreed to engage in four rehearsal sessions and one performance in an online networked event. The sessions were facilitated by the remote audio company Source-Elements. Each musician represented a different rhythmic tradition and this experiment was trying to find out how much diversity we can handle when we are online and latency challenges playing together.

During the first session different riffs, ragas and phrases were tried. The latency was serious. One could not hear one’s sound with the sound of others. In responding to others one’s sound arrived too late to be experienced as a response. The sound arrived too late all the time. As a result, the musicians did not resonate with each other at all. Leadership was questioned and who was going to decide what to do. Improvisations stopped even before they began. After the first session, all three musicians were upset expressing distrust to the others, being frustrated to the bone. In the second session, a similar pattern was happening up to the point that they wanted to stop the experiment even though they needed the money they would earn. Then Rebekah Wilson, composer and musician herself and CEO of Source Elements and following the experiment in the background, took over the screen and said ‘I like to tell you something before you stop. Do realize that latency is just a psychological effect and that your rhythms can tune to each other, I am sure. It is just a psychological effect. Please, go with the flow, give it another try.³³ A significant silence followed in which each musician tried to understand her words. And then they started to play again, together this time trusting the flow and including the latency as one of the musical elements they were improvising with. An extra space had opened up, new a new possibility had presented itself.³⁴

Tuning to diverse rhythms appears to be also a psychological space, a willful decision to open up to a dynamic one cannot control. Tuning to rhythm then requires the capacity to improvise and surrender to a quality and result that is beyond one’s doing yet a direct result of one’s actions anyway. Latency is, as it were, the sensorial representation of a complexity we are part of yet do not understand the causality from. One can also argue that latency prohibits being witness to each other because it hinders being witness to the self. The causality of the relation with one’s sound is broken.

In the larger artistic research project ‘Witnessing You’ artists answered the question ‘What happens when one witnesses another? In this study, it appears that being witness to the self is foundational to being able to be a witness or bear witness. In being-witness-to-self ethical decisions for performing one’s presence are taken. For the musicians to lose the causal relation with their sound, not being able to be witness to themself as a musician, damaged the capacity to listen to the others as well and destroyed the collaboration.

The ‘psychological effect’ that Wilson pointed to, refers to the human capacity to imagine, attribute and adapt which defines human relations in all diversity and to human-machine interaction as well. Wilson’s psychological effect points to a decision to trust the flow of interaction even though causality as one knows it in the physical world is dysfunctional. When the musicians decided to trust the flow, not as a surrendering, but as a new context they could play with, then they could play. By using their imagination in the processes of attribution and immediate adaptation, a new self-witnessing emerged that is based on this new improvised reality caused by latency.

Being in different locations (Not-Here), sharing time but having to deal with latency (Now/Not-Now) and sharing action even though it was unpleasant (Do), but not knowing each other in the beginning(Not-You) and coming from different rhythm traditions (Not-You), resulted in a serious distrust in each other. When they learned to play together in real-time with the latency(Now) and overcame the unpleasant experience of not being able to synchronize (from Not-You to You), the trade-off for trust changed. They entered into a collaboration and many enjoyed the concert they gave in the end.

Next to documentation, orchestration and reflection in given media settings, the sensorial physical experience of technology cannot be underestimated in its effect on our imagination and capacity to attribute and adapt. One can also see in this experiment that the developing media schemata may be painful in the beginning, yet once they are overcome, they set a new standard for interaction and the trade-offs for trust. A new capacity develops and new media schemata emerge.

7. Conclusion: tuning to one and other

As a practitioner, the author has been part of the emerging digital culture since the 1980s in Amsterdam. She visited and created performative contexts in which different ideas, networks and practices could come together in cultural and political spheres in different cultural venues in Amsterdam. In these ‘shows’ the potential good and the potential detrimental of the new digital technologies were presented and discussed. Often artists were the first to explore the experiential potential of these new presence configurations that were made possible by ICT. In doing so, they were breaking the current media schemata and opening up the imagination on the impact of such new interactions. The four artworks that are discussed in this article, were breaking the current media schemata at the time and had an impact on the collective experience and imagination of the audiences who were present in those places at the time. Such moments, in which new possibilities are explored and reflected upon, affect the development of technology itself and change the way society accepts and trusts different mediated presences.

To understand how changing trade-offs for trust affected the development of different media schemata, four artworks have been analysed with the YUTPA framework. It is the body that beholds the sensations, emotions and complex feelings like solidarity and compassion which drive the steering towards well-being and survival that results in the performance of presence. This steering happens in four dimensions Time, Place, Action and Relation. To be with You in Unity of Time, Place and Action (YUTPA) is the basic configuration of being together in physical presence as human beings who are related and can act together. Because of information and communication technology development, a variety of new YUTPA configurations for being together as human beings has emerged and in the last three decades millions of people have developed new capacities for being able to participate in these new presence configurations. As a result, new media schemata emerge and keep emerging.

The four artworks that are analysed with the YUTPA framework in this paper, offered specific new presence configurations that broke the established media schemata at the time. The 1989: Czech Mediated Clapping Jam showed that bridging trust is possible by making a connection through the ‘iron curtain’ which was beyond what was deemed possible at the time. The 1995: Ping Body, Telepolis Luxembourg by Stelarc, showed how it is possible to hurt another person who is mediated on a screen and how this affects the mediated witnesses as well. The 2005 The Girlfriend Experience by Martin Butler showed how anonymity in mediated presence shatters accountability and detrimental trust defines what happens next. The 2021 Tuning to the rhythm of others – Mistah Issac, Sirishkumar Manji, Reinier van Houdt showed that latency requires a new capacity for improvisation.

A first conclusion of the YUTPA analyses of the four artworks in different decades is that in moments of a break in media schemata, awareness of the physicality of human presence appears to be crucial in how we are witness to one another. To be a witness and bear witness is foundational to social structures because that is how the trade-offs for trust and truth are defined. The new YUTPA configurations affect witnessing. It is the body that steers towards well-being. It is the body that is witness to self and others and in which ethical positions are taken. When being a witness and bearing witness is not possible because a specific YUTPA configuration allows for anonymous participation, people easily ‘dehumanize’ each other and become numb to the pain of others. New YUTPA configurations have to allow for taking responsibility and being held accountable in both on- and offline-related contexts.

A second conclusion is that ‘ Tuning to one another’ in online presence configurations, is both a sensorial physical act and an imaginary act of attribution and adaptation at the same time. It is a sensorial physical act because the body tunes easily to the rhythm of other realities. It is the tuning of rhythms that ultimately allows for togetherness in both on- and offline environments. However, in an online musical setting, latency is a major hindrance to playing together. The rhythm experiment that is discussed in this article shows, however, that it is possible to overcome such a hindrance. Once people allow for a mediated witnessing of the self and decide to trust the partial perception of the mediated presence of others while engaging in an improvisational shared fragmented flow and allowing imagination to take over, playing together becomes possible. When being and bearing witness to one another with latency as a significant factor in online synchronous communication, this requires a capacity for improvisation that brings processes of attribution, synchronization and adaptation to a new level of experience. This is only possible when responsibility and accountability in being and bearing witness to self and each other, are clearly defined and a trusted context is created.

8. Epilogue into the future

In the coming era, when technologies for Artificial Intelligence and virtual and uugmented reality are expected to merge more and more, accelerating interactive collective imagination and ubiquitous disinformation will become part of the daily experience of millions of people. These new practices that resonate with what was once formulated as propaganda, seriously undermine resilience and architectures of trust as they were known before. Human beings are very vulnerable to such detrimental synchronization, imagination and adaptation which confuse the strive for well-being and survival.

2023 Mirror V1 -DV1, or Minoreia canceled³⁵

In 2023 in Upstream Gallery, Amsterdam, digital artist Auriea Harvey had a solo exhibition in which she showed the artwork Minoreia Canceled, which showed a new change in the possibility of being witness to self and each other online.³⁶ Interacting with old societies, whilst at the same time being among the most advanced artistic practitioners of Virtual Reality over three decades, she combines the old craft disciplines of sculpture in bronze with new craft skills of configuring Virtual Reality environments. The Minoriea is a 35-centimetre bronze statue depicting an elaborate mythical figure that is referencing not specific old forgotten cultures. Above her head is a halo or monstrance seeming to demand respect of a spiritual dimension of her being. Harvey mentions her in terms of being her avatar in the real world. Close to the little statue is a grand mirror, 1 metre in diameter. Here we see the face of Minoriea, mirroring herself and mirroring visitors that are close by. By moving in front of the mirror, Minoriea is in some movements visible as one or two visitors are. The images merge, move apart and merge again, never offering a ‘complete view’. The Minoriea is also captive of the mirror not being able to leave its frontier. Seeing a tiny light face in the rusted copper mirror surrounded by Minoriea’s dark-skinned face is very moving. It allows for the deep connection of seeing one in another and others in one. Looking back to the little statue of the Minoriea after playing with the mirror movements, the statue has acquired impressive power. Imagination and capacity to attribute and adapt, have taken over the production of meaning. The Minoriea shines radiantly in the room and all feel part of her.

Harvey made the bronze statue in the real world her avatar while seeking connection with others in the digital reflection of this avatar via the mirror in which visitors could only be physically present in the real world. It feels like an inside-out YUTPA configuration in which the symbiosis between one and other, between the self and the world, is complete. The slightest change in posture or attention though ruined the reflection and the symbiosis with it. It was a dynamic symbiosis between the I and the Self and the image of the other, between the I and the Self and the world. The relation between the ‘I’ and ‘self’ is immediate. It seems that in this immediacy reflections emerge on the three other YUTPA dimensions in which memory and experience on Time, Place and Action affect the process of synchronization, imagination and adaptation deeply.

The process of self-witnessing has a long tradition in spiritual and religious traditions. It is fundamental to praying and meditation in which on a deep psychological level trade-offs for trust and truth are sought for. In ‘Mirror V1-DV1, or Minoreia cancelled’, the reflection of the self, is seen in the reflection of the other. Me and the Minoreia are one. It resonates with the experience of the divine.

Being witness to ‘I’ and self may appear to be a significant entrance into the new kind of trade-offs for trust and truth, that humankind will need once AI is deeply integrated in our daily world. In the long duration of being witness to the physical, emotional, and spiritual ‘I’ and self, the body constantly helps to deconstruct. Switching between real and virtual worlds and merging online and offline experiences into one, the witnessing of the self happens all the time through all trajectories through all configurations we go through. In moments when being and bearing to self and being and bearing witness to others becomes one, in such moments suddenly trust and truth can shine. Be it for a split second, a veil was lifted. The memory and experience of such a moment will stay and help to deconstruct the next moments of confusion. Learning to self-witness while tuning to others in the many emerging YUTPA configurations may appear to be pivotal for the collective well-being and survival of humankind in the era to come.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Caroline Nevejan

Prof. dr. Caroline Nevejan is a researcher and designer who has been involved with the emerging network society and digital culture since the 1980's. Nevejan is a regular presenter at national and international forums. She is an advisor to local, national and European policymakers. For more information: www.nevejan.org.

Notes

1 Kennedy (2003).

2 IJsselsteijn (2004).

3 Volkmer (2006).

4 Dodgson, Gann, and Salter (2005).

5 The notion of ‘key socio/technical inflection points’ is formulated in a personal email from editor David Garcia, referring to moments of transition of media schemata, as is formulated in this text.

6 Borgdorff (2012).

7 Oliver (2001).

8 Damasio (2003).

9 Nevejan (2023).

10 The four YUTPA dimensions of Time, Place, Action and Relation are introduced in the dissertation ‘Presence and the Design of Trust’. In later research at Delft University of Technology, distinct factors that define trade-offs in each of the YUTPA dimensions are identified. See footnote 11 and footnote 12.

11 Nevejan and Brazier (2015).

12 Nevejan and Brazier (2012).

13 Damasio (1999).

14 Riva et al. (2003).

15 This moment happened in a public event that was initiated, orchestrated and produced by the author of this contribution, in collaboration with the political organizations Charta ’77 and AIDA.

16 A normal telephone at the time was connected via a physical copper wire to cables in the ground in the street which were all leading to switchboards where different physical lines came together. Because of the Cold War between the Unites States of America and the Soviet-Union communication between East- and Western-Europe was hardly possible before the Berlin Wall came down on the 9th of November 1989.

17 A few years later Performing Arts Labs in London invited over a dozen ‘mediated show makers’ for a media lab that was called Live Arts Online. We were all involved in connecting different spaces with live audiences to each other and had many ‘war stories’ to share. In such events Mr. Murphy, who guaranteed that anything that can go wrong will go wrong, was a regular visitor. Technology was still very raw, phonelines had to be adapted, video projectors had to be calibrated, and audio lines had to deal with severe latency. In the sharing of the many stories, we agreed that introducing the ‘social interface’ was crucial for success. Performing Arts Labs was a London based art foundation, directed by Susan Benn, which offered selected successful performing artists and opportunity to explore new work in a 10 day residential lab at Bore Place, close to Sevenoaks.

18 Nevejan (2007).

19 Sloterdijk (2011).

20 Tokoro and Steels (2003).

21 Stelarc’s performance happened in the context of the Doors of Perception 3 conference. The author was co-curator and producer of this event.

22 Stelarc’ Ping Body was performed at the November 1995 Telepolis ‘Fractal Flesh’ event, Paris (the Pompidou Centre), Helsinki (The Media Lab) and Amsterdam (for the Doors of Perception Conference) were electronically linked through a performance website allowing the audience to remotely access, view and actuate Stelarc’s body via a computer-interfaced muscle-stimulation system based at the main performance site in Luxembourg. http://www.medienkunstnetz.de/works/ping-body/, accessed 10/10/2023.

23 The Ping Body experience in Paradiso resonates with the classical Milgram experiment in which persons were seduced to give electrical shocks to a person in another room they could see, yet this other person could not see them: Milgram (1963).

24 https://www.nytimes.com/2022/04/15/us/drones-airstrikes-ptsd.html (accessed 10/10/2023)

25 McLeod (2023), Miller (2016).

26 Martin Butler is known for his collaborative approach to theatre-making, bringing together artists from different disciplines to create works that are both visually stunning and intellectually engaging. Butler’s direction is characterised by its attention to detail, strong visual sense, and ability to create immersive, multi-sensory experiences for audiences. Butler has worked on a range of productions, from large-scale spectacles to intimate, experimental works (https://martinbutlers.com, accessed 19th of September 2023).

27 Mediamatic is an art centre dedicated to new developments in the arts since 1983. We organize lectures, workshops and art projects, focusing on nature, biotechnology and art+science in a strong international network (https://www.mediamatic.net/en/page/10341/about-mediamatic, accessed 19th of September 2023).

28 The author was a visitor to his event.

29 Wylie (2019).

30 The Venice Exploatroium was curated by prof. dr. Caroline Nevejan with graphic scholar and designer Huda AbiFarès and Jane da Mosto, environmental scientist and leading ‘We Are Here Venice. When the COVID-19 pandemic exploded many countries went into lockdown and the Venice architecture exhibition was postponed for a year, a new question emerged. Would it be possible to have a local impact in Venice even though one could not go to Venice? Can such interventions be beneficial to Venice even though artists and scientists are not there to feel in which direction survival and well-being are to be found? All teams documented their trajectories here: https://openresearch.amsterdam/en/page/52688/values-for-survival-2020-2021 (accessed 17th of september 2023).

31 Interviews with the musicians and the recording of the experimental sessions can be found here: https://openresearch.amsterdam/en/page/70235/opening-concert---tuning-to-rhythm (Accessed 19th of September 2023).

32 With Rebekah Wilson, the author was co-curator and co-producer of this special experiment.

33 Source Elements is founded by musicians, composers and sound engineers and engagses regularly in experiments next to being a successful company for remote audio: https://www.source-elements.com (accessed 17th of September 2023).

34 To better understand this ‘psychological effect’ a series of experiments is being conducted by source-elements in the following years of which results will become available in 2024.

35 The author was visitor to this exhibition at the Upstream Galery in Amsterdam.

36 Auriea Harvey is a digital artist and sculptor living and working in Rome. Her practice encompasses virtual and tangible sculptures, drawings and simulations that blend digital and handmade production including 3D printing, augmented and virtual reality (https://www.upstreamgallery.nl/artists/87/auriea-harvey, accessed 19th September 2023).