When extreme climate change forces humanity to flee Earth for the Moon, what choices get you there? An AI agent rewrites the story around every decision, while Visko renders each scene as live video inside a Meta Quest 3 experience.
Award & Team
System
The headset calls OpenAI directly. The AI acts as a story director, returning structured data every turn — the scene, two choices, their wellness costs, and video prompts. Those prompts go to Visko, and the generated video streams back into Unity. Carrying this loop through an entire playthrough was the hardest part of the build.
Built for a standalone headset. Everything runs on Meta Quest 3, with no PC or server kept running during play.
One story controller holds the inventory, choices, and wellness score, so the UI, the physical objects, and the video always agree on where the player is.
The player picks one of two options inside the VR panel.
OpenAI writes the next scene, two choices, and wellness costs from the full history.
Video prompts are turned into a live, generated scene.
Video streams onto a curved screen wrapped in a 360° skybox.
My Role
I started the project and led a team of four. I designed the game flow, the UI, and its components in Figma, and created the background and visual assets the team built from. Our developer translated those designs into readable VR panels in Unity. I also produced the showcase video, capturing in-game footage and editing it with VLLO and Figma.
Green and blue variants with default, hover, and selected states.
The +5 / −5 popup and the wellness bar's good, warning, and critical states.
Target, fuel gauge, and throws-left counter for the launch pad.
Design Approach
Starting point: this year's Immerse the Bay theme, "Take Me to the Moon." We imagined the journey beginning somewhere every player could picture — a Stanford dorm room — and focused on storytelling so anyone could pick it up and see themselves in it.
Turn a fixed destination into a question about character: not if you escape, but who you become on the way.
Lock six locations and leave the events at each one open, so the AI has guardrails without losing variety.
Let the AI write each scene in real time, remembering earlier choices so the journey feels connected.
Designing the AI's Rules
The AI writes every line the player reads, so the copy had to be designed as a system, not as individual screens. We gave the story director a set of rules that keep each scene short enough to read at a glance over moving video, and every choice hard enough to matter.
Every decision is a real dilemma between two grabbable objects, never something small like left or right.
Labels address the player as "you" and start with an action, so they read instantly in VR.
One sentence for the outcome, then two for the next scene: what's in front of you, then the problem.
Each scene has one thing happening and at most one new character, so the story never outruns the player.
Every decision has one good and one bad option, and the player isn't told which is which. The bad option always has to be tempting: safer, faster, or more loyal to someone the player cares about. Helping someone costs the player something, and helping yourself costs someone else.
The Journey
You get 15 seconds to grab three of five items from your room and place them in your inventory tray. Those items come back at the launch pad, where you throw them to fuel the rocket. The locations are fixed; what happens at each one is written live, and people you help or abandon can return later.
Wake your roommate, or slip out alone.
On a burning campus: steal a motorbike, run the Caltrain tracks, or flag down an ambulance.
Fuel the rocket by hand, throwing your items into the tank.
The rocket is too heavy. Cut loose the strangers on your ladder, or pull them in and risk everything.
Your roommate returns to the story — beside you, or calling from Earth.
Your choices are judged at the colony gate.
Design Challenges & Solutions
Challenge 1
The friction: this was my first VR hackathon, and I had never designed spatial UI. Flat-screen rules broke in the headset: text over constantly changing live video became hard to read.
Before the event, I studied Meta's Quest design guidelines and Apple's Vision Pro Human Interface Guidelines. During the build, I applied them to rethink panel size, viewing distance, tilt, and contrast for the headset.
I could make spatial layout decisions with confidence from day one, and every scene kept a consistent look, feel, and navigation, readable no matter what video played behind it.
Challenge 2
The friction: the storyline shifted constantly as the team iterated, and every change meant new components and assets, often within minutes, under tight time pressure.
I used AI throughout my workflow to generate and adapt components and assets quickly. When I found out late that the video storyboard had to be vertical, not the wide format I had started with, AI let me update every frame in a fraction of the time.
Prototyping time dropped sharply, and design never became the bottleneck as the story evolved.
Challenge 3
The friction: the AI writes every scene and option live, so we never knew how long any line of copy would be.
Designed flexible layouts that hold AI-generated text of any length without breaking, instead of designing around fixed copy.
The UI stayed intact across playthroughs where no two scenes were ever the same.
Challenge 4
The friction: we wanted players to do more than point and click, but there was no room for a tutorial in a fast-paced escape.
Built physical mini-games tied to the opening — grabbing items in 15 seconds, throwing them into the fuel tank — so actions explain themselves.
The three items grabbed at the start come back at the launch pad, linking interaction to story.
Challenge 5
The friction: an open-ended AI story can drift, and we had designed far more than one weekend could hold.
Mapped scenarios on a whiteboard before building, fixed six locations, and cut scope until one clear idea remained. A premise document set the world, tone, and rules for the AI, and the AI remembers earlier choices to keep the journey connected.
The constraint of six checkpoints sparked more diverse ideas within those limits, while keeping the generated story on track.
Challenge 6
The friction: a design that looks right in Figma can feel wrong once it's placed in a 3D scene, and a flat screen alone doesn't feel like a world.
I worked closely with the developers, communicating placement, scale, and positioning in detail as they built each scene in Unity. I also created 360° backgrounds that surround the player at every checkpoint.
The designs translated into VR as intended, and the judges said the 360° environments made the experience feel more vivid and immersive.
Challenge 7
The friction: generating, streaming, and decoding each scene takes time. A blank screen or a sudden cut would break immersion right after every choice.
The screen never goes empty: it holds the last frame until the next one is ready. After a choice, the current video morphs into the outcome while the next scene prepares in the background. Choices appear only after a few seconds of visible video, and a Continue button lets the player move on when they're ready.
Transitions feel like one continuous shot. Testing in the headset caught what looked fine on a monitor: UI blocking the video, unclear interactions, and transitions that ran too long.
Everyone reaches the Moon. Not everyone is welcome there. Every player starts at 100 wellness, and each choice moves it up or down by 5, judged by how you treat others. Score 85 or more to become a Moon Citizen; anything less and you're turned away at the colony gate.
Feedback
A +5 or −5 popup appears after each decision, so players feel the weight of a choice without being told the rules in advance.
A bar stays in view from the first decision onward and shifts from green to orange to red as the score drops.
The end screen shows the verdict, final wellness, successful throws, the full decision history, and a traveler type.
Verdict
Tuning the score took several rounds of live testing, so the verdict reflects what kind of person you are under pressure.
Never put yourself first. Everyone you met made it because of you.
Left your roommate in the first minute, then spent the journey making it right.
Would do anything for your own people. Strangers had to earn it.
Hit every throw at the launch pad to fuel the rocket.
Not perfect, and you knew it. You did enough good to be let in.
"You made it to the Moon. The people here know what you did to get a seat."
Learnings & What's Next