VR Reaction Game
XR/VR/AR
Role
Prototyper &
Designer
collaborators
Vanessa Neef
Madeleine Stålhammer
Our goal for the reaction game was to keep players in a “flow state”, not too easy, not too hard, so we created a machine learning model that adjusts difficulty based on performance. Using Unity ML-Agents, we trained a PPO Machine learning model with a simulated player, resulting in much more stable and balanced gameplay compared to a rule-based approach. Additionally, we log performance and difficulty adjustments to quantify how effectively the agent maintains players within the intended flow zone over time.
Prototype of the Reaction Game
We developed a VR reaction-time game in Unity where the user faces a floating grid of tiles. For each grid state, one tile becomes the target colour and the user must press it before the next state appears.
To keep the task engaging and personalised, we use Unity ML-Agents to adapt difficulty in real time. The agent observes recent player performance, along with the current grid update speed and the current number of distractor colours.
Based on these observations, the agent adjusts (1) the frequency of introducing additional distractor colours (up to five total colours) and (2) the speed at which the grid state changes.
Challenges we encountered
For the ML itself, a key challenge was deciding what information to give the agent. I tried to think about it like if I were the coach. If I were watching someone play and wanted to adjust difficulty, what would I observe? I settled on the last 10 reaction times, the hit ratio, and the current difficulty level. Enough to understand the player's performance but still simple for the time frame we had. The biggest lessons were that VR requires completely different thinking from screen-based design/development, and that machine learning design is as much about asking the right questions as it is about technical implementation.
Other projects in this course
Multiplayer Jenga game
Unity Prototype of multiplayers
Created a multiplayer VR experience with Unity’s Netcode for GameObjects (NGO). In pairs of two we built a real-time multiplayer Jenga game. The challenge here was to ensure players stay in sync, minimize latency, prevent cheating, and manage ownership and communication between client and server.
CAD & Technical details
Blender Sketch and AR on Iphone
For this task we modeled a few objects in Blender and used these to build a scene with duplication and modification. Then we built a mobile AR project in Unity and deployed it to your mobile phone.