Overview

“Inside the Mind” is a virtual reality experience that transfers the viewer into a mindscape. In the mindscape that is represented as a bedroom, the viewer can explore the elements that make up the mindscape to discover more details about who this mindscape belongs to. The viewer is not manipulating this mindscape but is merely acting as an observer. Utilizing the themes of uncanny valley and absurdity with audio and visual elements, the identity of this mindscape can be explored.
The narrative of this piece is about being randomly placed inside a random person’s head at a random point in time. There is no clear beginning or end, or any logical connections between any objects. This narrative plays off of the idea that AI is not 100% accurate in portraying. This can be seen within the narration and visuals within the experience.
 The intention of utilizing virtual reality as a medium is to demonstrate that it is not only for games that have player-driven plots but also for immersive art experiences. As technology becomes an integral tool in new media art, this experience is both an exercise and a demonstration of how interactive installations can utilize virtual reality as a tool for more sensory immersive qualities for storytelling.
The overall experience can last 8 minutes, but participants can choose when to stop the experience, as the beginning and end are not clear.

​Exhibited at:
2023 - Light Upon Arrival, Bastille Design Center, Paris, France.
2023- (how did we get here?), Paris College of Art Gallery, Paris, France.

Keywords: Virtual Reality, Nonlinear, Mind-Castle

Programs utilized: Autodesk Maya, Blender, Unity, Adobe Photoshop, Adobe Substance

References

Mike Kelly. Educational Complex. 1995. Sculpture. Foam , fiberglass, plywood, plexiglass, and mattress. 57 3/4 × 192 3/16 × 96 1/8in. (146.7 × 488.2 × 244.2 cm).

Educational Complex by Mike Kelly

This piece by Mike Kelly draws upon his memory of his childhood school. This is his attempt to recreate it from memory alone, with gaps within spaces that he cannot fully remember.

Gone Home by Fullbright

This game is an exploration game where the player comes home to find their family missing. They must explore the horse and interact with objects to gain clues as to what happened to their family. 

Fullbright. Gone Home. 2013. Video Game.

Fullbright. Tacoma. 2017. Video Game.

Tacoma by Fullbright

This game is an exploration game where the player can explore the past events of a ship crew to learn about their lives on the ship and what happened to them.

This is aUn Chien Andalou

This 1929 silent film by Luis Buñuel and Salvador Dalí. The narrative for this film makes no logical sense. This is incorporated into this project’s narrative.

Film Poster for Un Chien Andalou.

/SubSimuatlorGPT2. 2022. Screenshot.

r/youtheasshole

AITA (Am I the Asshole) is a popular subreddit used by people to describe scenarios to understand if they were in the right or wrong. Taken one step further, an AI has recreated this subreddit and all posts and responses are written completely by AI. Some seem plausible, some seem outrageous. I used this as inspiration towards the narrative, making absurd connections between objects and narration as if the mindscape belongs to AI.

Theoretical References

Mind Castle (Method of Loci) - Greek poet Simonides of Ceos in 477 BC
The Method of Loci is a method of memorizing a list of items by thinking of a familiar location (i.e., your home, a neighborhood block, etc.) and imagining specific items residing in specific locations. Whenever you need to recall the list, you imagine yourself walking through the location to each item.

Uncanny Valley by Masahiro Mori
The phenomenon in which a humanoid figure creates uneasiness towards a viewer upon seeing this figure or interacting with it. 

Process

Kirby Mealer. 3D Room progress. Maya. 2023.

Kirby Mealer. Unity Scene. Unity. 2023.

Kirby Mealer. Unity Scene. Unity. 2023.

3D Modeling
For all 3D objects within the scene aside from the playable character, they were created within Maya. These objects were arranged very similarly to how they would be in the final Unity scene to ensure that the scale in relation to each object was accurate. For the furniture, I referenced the Ikea catalogue to get orthographic views of furniture but purposefully omitted details to keep the objects vaguely similar to their real-world references. 
The purpose behind modeling everything from scratch was to ensure that I had complete control over how the room looked and could easily animate and texture the meshes as needed without needing to worry about unreliable topology. 
To put a piece of myself into this project, I utilized old 3D assets that I made during my undergraduate. These models include the dog, the clay hippo, the desk plant, the pens, the Solo cup, and the mug. This both saved time and allowed me to incorporate a more unique personality into the bedroom. In a way, this mindscape partially belongs to me too with the incorporation of these easter eggs.


The only asset not created by me was the VR playable character, which was from the VR IK Toolkit. It was modified and retextured for the purpose of this project.

• Story: Logical and Illogical Phases

For the story, I wanted to have two phases for each object: a logical and an illogical phase. This was both to ensure that every object had a clear distinction between the two phases and to keep it simple for time.
 For the logical phases, I utilized ChatGPT to create some basic narration about each object that is vague enough to be relatable but not very personality-defining. This is to allow the owner of this mind to be relatable but vague. Not one person could be assigned as the owner of this mindscape. In relation to this narration, the objects animate in a normal way, such as a dog wagging its tail after it is activated.
For the illogical phase, I referenced the film Un Chien Andalou, which had nightmare-like actions within the film, such as ants crawling out of a hole in a hand. I wanted to incorporate this randomness within my project and put these random elements within the illogical phases of the objects. The narration for this phase is written by me so that I can exaggerate the uncanniness.

• Story: Utilizing Chat GPT

To help write the story, I utilized ChatGPT. The reason behind using AI is because I wanted to incorporate the Uncanny Valley in non-visual ways, and having an uncanny narrative was one way to do just that. The concept behind the owner of the mindscape is a computer that wants to be human, so using ChatGPT was a logical choice in incorporating this concept. Because I wanted dialogue that was vague enough to allow anyone to relate to it, the vagueness of the responses given by ChatGPT3 gave me what I desired. All the dialogue within the logical phase of each object was written by ChatGPT, and the illogical phase was written by me to add a juxtaposition to the dialogues for each object/sequence.

• Key take-aways using Chat GPT

As I explored ChatGPT for narrative purposes, I came to the conclusion that it is not reliable as an author. I had to modify every phrase of narration it gave me. Because I was using ChatGPT to get an uncanny tone to the narration, I found that I was successful. However, for practical uses for storytelling, I find it to be more of a hassle to get what I wanted out of it than useful. 

Animation with Blend Shapes
For some objects, I needed a way to animate the mesh in Unity. In Blender, I created blend shapes that allow a mesh to change from one state to another. I can then key these blend shapes in Unity and the timing is according to the narration within the project.

​Kirby Mealer. Bed in Blender. Screen Capture. 2023.

Kirby Mealer. Unity Timeline. Screen Capture. 2023.

Unity Timeline
To use as minimal an amount of code as possible, I utilized Unity’s Timeline feature to create a sequence that each object will follow. These sequences are triggered when the participant clicks on the object and stop when it reaches a signal within the timeline. It then plays from where it stopped the next time it is clicked. When the timeline ends, it resets, and the participant can trigger the sequence again.

Using Unity Buttons
To activate each object, there are invisible buttons surrounding each object. As Unity has built-in buttons that I can use to play timelines, this allowed me to have an easy way to have each object activate without having to code.

Kirby Mealer. Unity Scene. Unity. 2023.

Kirby Mealer. Unity Scene. Unity. 2023.
​

Timelines Within Timelines
Some objects are contained within other timelines, meaning they can only be seen if the original timeline is activated or several timelines are activated in specific orders. For example, a book appears on the bed at the end of the bookshelf timeline. However, if one were to interact with the bed first and the bed is in its second phase where it is raised, you will not see the book if you interact with the bookshelf next. This adds an extra layer to the narrative, creating the possibility to miss some elements.

Particles
I utilize particle effects in Unity to signify to the participant that the objects can be clicked. While the timeline is triggered, the particles disappear and only reappear when the timeline can be triggered again.

Kirby Mealer. Unity Scene. Unity. 2023.

Final Renders

Exhibition