Computational Narrative Phantasmal Media & Virtual Identity
“My career is a quest to use the computer as a medium to spur the imagination and galvanize people to critically reflect upon society.”
Prof. D. Fox Harrell, MIT CSAIL
0. Computational Narrative
Professor Fox Harrell on computational narratives: “Computational narratives take many forms including virtual and augmented reality systems, immersive experiences, videogames, electronic literature, and new forms unanticipated by any of those. One approach has been to develop systems in which users participate as main characters in very traditional stories (like the holodeck in Star Trek). I have sought, instead, to generate new forms of interactive and immersive narrative experiences using AI to dynamically change aspects of stories such as metaphors used, cultural perspective conveyed, emotional tone evoked, and more. This has required not only developing new computational approaches, but also developing a novel computational model of what cognitive scientists call “conceptual blending” – the cognitive processes of creatively combining and elaborating new ideas. It also required drawing upon more contemporary work in the arts and literature than is usually cited in AI research.”
- Phantasmal Media: An Approach to Imagination, Computation, and Expression – the book published by MIT Press
In Phantasmal Media the expressive power of computational media is considered and presented. As the recommendation of the book emphasizes, Prof. Harrell “argues, forcefully and persuasively, that the great expressive potential of computational media comes from the ability to construct and reveal phantasms—blends of cultural ideas and sensory imagination. These ubiquitous and often-unseen phantasms—cognitive phenomena that include sense of self, metaphors, social categories, narrative, and poetic thinking—influence almost all our everyday experiences. Harrell offers an approach for understanding and designing computational systems that have the power to evoke these phantasms, paying special attention to the exposure of oppressive phantasms and the creation of empowering ones. He argues for the importance of cultural content, diverse worldviews, and social values in computing. The expressive power of phantasms is not purely aesthetic, he contends; phantasmal media can express and construct the types of meaning central to the human condition.”
“My cognitive science term [phantasm] does not refer to ghosts, poltergeists, or specters. Phantasms are a combination of imagery (mental or sensory) and ideas ranging from our senses of self to social ills and other everyday experiences.”
The key examples are available as follows:
- “Someone looking at a large mansion with Greek columns in the front might not only admire the architecture, but also immediately understand that it connotes other ideas such as wealth, power, or privilege. The image of the house, influenced by social and cultural, is a phantasma combination of a mental image and ideas.
- Phantasms emphasize the role of worldview in the imagination.
- Phantasms emphasize semi-visible ideas in culture—within the worldview in which they are based, phantasms seem natural and uncontroversial. Phantasms are revealed only when compared to concepts and images grounded in multiple worldviews.
- Phantasms are defined in cognitive science terms, building on theories of conceptual metaphor (Lakoff and Johnson), conceptual blending (Fauconnier and Turner), material anchors (Hutchins), and elastic anchors (Chow and Harrell).
- Phantasms emphasize “distributed cognition,” in particular that we think using images out there in the world, not just in our heads —phantasms emphasize the subjective, mysterious, and invisibility natures of widespread cultural images.”
The key ideas of computational or cultural media and further information about the book are available on the page of MIT yPress.
2. Virtual Identities and Avatar Dream – a cultural and ethical approach
Professor Harrell coined the term Avatar Dream to describe a socially-widespread vision of a future in which people are capable of presenting themselves as whomever and whatever they want to be using computers—as can be seen in examples ranging from recent social VR systems back to avatars in what was termed “cyberspace” in old cyberpunk fiction. Far from arguing for blindly pursuing the Avatar Dream, however, Harrell states “The Avatar Dream poses major social and ethical challenges. I argue that we need to reimagine the Avatar Dream as one where the potential social and cultural impacts of virtual identities are considered intrinsic to the engineering practices of inventing them /…/ It is important that virtual environments — and the virtual identities within them — can convey or even augment how we express ourselves. Yet, some user groups are underrepresented or unfairly stigmatized by algorithmic bias and the actions of other users. Toward addressing these social and ethical challenges, I have pursued two strands of research. The first is computationally analyzing social phenomena in virtual identity systems. For example, we have employed machine learning (ML) approaches to study how people use social media differently in diverse cultures globally so that developers can best support how they negotiate their own cultural values.” Illustrative videos, such as Reimagining the Avatar Dream & Virtual Identities discuss these issues, and also the phenomena of computational narratives, video games, empowered users and algorithmic bias.
——
D. Fox Harrell, Ph.D., is Professor of Digital Media & Artificial Intelligence in the Comparative Media Studies Program and Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. In addition, he is the director of the MIT Center for Advanced Virtuality.
His research explores the relationship between imagination and computation, and it involves developing new forms of Virtual Reality, computational narrative, video gaming for social impact, and related digital media forms based in computer science, cognitive science, and digital media arts. In his work, he explores how computers can be used for creative expression and social empowerment. Read more about his research & creative work in narratives, social media, gatekeeping and the future of storytelling.
