NeuroScape
Neuroadaptive Interface · Spatial Audio · LLM · Mindfulness · EEG
Designing adaptive intelligence for responsive meditation environments.
Overview
Most adaptive interfaces require explicit interaction: a click, a command, or a conversation. NeuroScape explores a different interaction paradigm, one in which an environment quietly senses changes in the user and responds through the surrounding sensory experience. I designed and developed NeuroScape as a closed-loop neuroadaptive meditation environment, connecting participant-relative EEG interpretation, LLM-based scene reasoning, and spatial audio rendering. Rather than optimizing toward a single physiological state, the project investigates a broader question: How should an intelligent environment decide when—and how—to adapt to a person without taking over their experience?
01/Problem
Meditation is dynamic. Most meditation audio isn’t.
[Mindfulness meditation] involves an ongoing process of sustaining awareness, noticing when attention drifts, and gently reorienting toward present-moment experience. Yet most digital meditation experiences remain largely predetermined. Guided recordings, music, and ambient soundscapes may create supportive conditions, but they do not change as the practitioner’s internal state fluctuates.
[Physiological sensing] creates an intriguing possibility: could the environment itself become responsive without requiring the user to stop meditating and interact with an interface?
However, [responsiveness] introduces a deeper design problem. A system that reacts too often may itself become a distraction. A salient sound intended to redirect attention may instead capture it. An unexpected transition may disrupt the coherence of the environment.
What's more, consumer EEG cannot reliably tell us exactly what a person is thinking or whether they are “focused.” So the challenge becomes:
How can an adaptive environment respond to uncertain signals while preserving continuity, agency, and space for the user’s own attentional regulation?
02/Research
From Neurofeedback to Adaptive Environments
I investigated three intersecting areas of research: mindfulness and auditory support, EEG-based neuroadaptive interfaces, and AI-generated adaptive environments.
Existing meditation technologies show how sound can support attention without requiring visual interaction. Neuroadaptive systems demonstrate that physiological signals can provide implicit context about changing user states. Meanwhile, generative AI makes it increasingly possible to construct dynamic sensory experiences rather than selecting from fixed feedback mappings.
However, combining these capabilities creates a gap. Many neurofeedback systems establish relatively direct mappings:
[Signal → Interpretation → Feedback]
But an immersive environment is more complex. A change needs to make sense not only relative to the user, but also relative to what is already happening in the environment. A bird should not simply appear because an EEG metric crosses a threshold. The system needs to consider what sounds are already present, what has recently changed, where the participant is within the imagined environment, how salient another event would be, and whether changing anything at all is appropriate.
Design Opportunity
This led to a different model:
[Physiological Context → Environmental Context → AI Reasoning → Situated Adaptation]
Here, AI acts not as a direct classifier-to-feedback mapping, but as a semantic bridge between uncertain physiological evidence and a continuously evolving sensory environment.
03/Designing Adaptive Soundscape
Adaptation is a decision, not a reaction.
The core concept behind NeuroScape is that an adaptive system should not automatically react whenever its input changes. Instead, NeuroScape separates adaptation into two questions:
-
[Should the environment change?] The system considers participant-relative EEG trends, signal quality, recent adaptations, and the current scene before determining whether intervention is appropriate.
-
[If it should change, what change makes sense here?] The LLM scene planner reasons within a constrained action space to determine how the soundscape can evolve while remaining semantically and spatially coherent.
At periodic checkpoints, the system can therefore choose to maintain, modify, or introduce elements of the environment. This distinction became fundamental to the project: Sometimes the most appropriate adaptation is not to adapt at all. NeuroScape therefore treats intelligence less as constant responsiveness and more as situated judgment about when intervention is useful, what form it should take, and when the system should remain quiet.
Three-Layered Soundscape
To make environmental adaptation controllable and interpretable, I decomposed the soundscape into three functional layers.
-
[Ambient Layer] Continuous environmental textures such as wind, water, foliage, or distant natural sounds establish the overall atmosphere and spatial continuity.
-
[Event Layer] Discrete environmental events—such as a bird call, rustling leaves, or a nearby water sound—create moments of perceptual salience and can subtly reorganize attention.
-
[Action Layer] Body- and movement-related sounds, such as footsteps or nearby interaction cues, connect the participant to an imagined position and trajectory within the environment.
Together, these layers form a structured action space for adaptation. Rather than asking an LLM to freely generate an auditory experience, NeuroScape lets the planner manipulate bounded properties—such as sound selection, intensity, position, movement, timing, and transitions—while preserving the larger environmental scene. The three layers therefore are not fixed mappings to psychological states. They are a design grammar through which adaptive intelligence can modify an environment without reconstructing it from scratch.
04/System Design
Closing the Loop
NeuroScape operates as a continuous closed loop:
[Sense → Interpret → Contextualize → Reason → Adapt → Experience → Sense again]
01. Interpret participant-relative EEG
Participants first complete a calibration session that establishes their own physiological reference. During meditation, incoming EEG is interpreted relative to this baseline rather than treated as an absolute measure of cognitive state. The resulting features provide coarse contextual evidence about patterns such as attention, relaxation, and stability, together with signal quality and confidence. This distinction is important: NeuroScape does not claim to “read” whether someone is focused or distracted. EEG contributes one uncertain source of context to the system’s decisions.
02. Build environmental context
The system simultaneously maintains information about the current scene, recent history, and scene graph. The scene graph describes available sounds and their roles, intensity, spatial behavior, compatible contexts, and transition constraints. This gives the planner memory of the world it is modifying rather than treating every adaptation as an isolated generation.
03. Reason about adaptation
At periodic checkpoints, the LLM Scene Planner receives both participant-relative EEG context and environmental context. Rather than freely generating content, it reasons within explicit constraints:
[Maintain → Modify → Introduce]
If adaptation is warranted, the planner produces a bounded semantic-spatial scene patch describing what should change and why.
04. Validate before acting
LLM outputs are passed through deterministic validation before entering the runtime. This separates semantic reasoning from execution, preventing the model from directly controlling the auditory environment and ensuring that adaptations remain within predefined system constraints.
05. Render the spatial environment
Accepted changes are rendered through the spatial-audio runtime, where sound sources can appear at different positions, distances, intensities, and trajectories around the participant. The resulting auditory environment becomes the participant’s new sensory context—and their subsequent physiological activity feeds back into the next cycle. The result is not EEG controlling sound directly, but a mediated loop in which sensing informs reasoning and reasoning reshapes experience.
05/Prototype
Making an invisible adaptive process observable
Because participants experience NeuroScape with their eyes closed, the primary interface is the spatial soundscape itself. A researcher-facing runtime interface was designed to make the otherwise invisible adaptive process observable.
The dashboard exposes five views of the system simultaneously:
-
[Journey Plan] tracks the participant’s progression through the semantic environment—for example, moving from a forest clearing toward a stream bank.
-
[Active Soundscape] shows which ambient, event, and action sounds are currently instantiated.
-
[Runtime World] provides a lightweight 3D representation of the participant and the spatial positions or trajectories of active sounds.
-
[Participant-relative EEG Context] visualizes incoming physiological context relative to calibration.
-
[AI Adaptation] exposes the planner’s reasoning at each checkpoint, including why the system chose to maintain or modify the current scene.
This interface was designed primarily for observability and research, allowing us to inspect how sensing, reasoning, and environmental behavior unfolded together over time.
After-session Dashboard
06/User Study
How does an adaptive environment enter into the ongoing process of attention?
Rather than assuming that an adaptive environment would outperform a static one, I designed the study to examine how adaptation changes the process of meditation. We conducted a counterbalanced within-participant study with 23 participants, comparing:
-
[Adaptive] participant-relative EEG informed periodic decisions to maintain or modify the soundscape.
-
[Non-adaptive] participants experienced a stable forest soundscape while wearing the same EEG and audio setup.
Each study session lasted approximately 60–70 minutes. Participants first completed a five-minute EEG calibration, followed by two approximately ten-minute meditation sessions in counterbalanced order. EEG was recorded in both conditions so participants could not infer which session was adaptive. After each condition, participants reported their experiences of present-moment attention, mind wandering, attentional reorientation, relaxation, spatial presence, soundscape coherence, intrusiveness, and overall helpfulness. We complemented these measures with semi-structured interviews and time-aligned EEG and system traces.
Participants did not experience environmental change in one consistent way.
The same type of sound could function as an attentional cue, helping someone notice and reorient; as a scaffold, sustaining engagement with the environment; or as an interruption, becoming a new object competing for attention.
Post-session ratings for the Stable and Adaptive Soundscapes. Boxes show the interquartile range, horizontal lines indicate medians, points represent individual ratings, diamonds indicate means, and gray lines connect ratings from the same participant. The bracket marks the significant condition difference after Holm correction across the eight reported measures (∗∗𝑝Holm < .01).
System behavior across the 23 adaptive sessions. The left timelines show eligible decision checkpoints and the timing of applied adaptations during each 10-minute session, overlaid on the active soundscape scene. Marker color indicates planner intent, diamonds indicate scene transitions, dark outlines identify EEG-informed or mixed-basis decisions, hollow markers indicate applied plan changes without evidence of a newly initiated audio element, and crosses indicate proposals rejected during validation. The right-hand matrix summarizes, for each participant, the number of applied adaptations, applied EEG-targeted adaptations, exposures to ambient, event, and action sound layers, scene trajectory, valid EEG coverage, rejected proposals, mean checkpoint-to-audio-start latency, and logged errors.
07/Findings
Responsiveness ≠ Constant Change
A responsive system does not need to continuously demonstrate that it is responding. Sometimes maintaining a stable environment was more supportive than introducing another adaptation. Good adaptation includes knowing when not to intervene.
Continuity matters as much as responsiveness
Participants often experienced the soundscape as an imagined world rather than a collection of individual sounds. Changes that made semantic and spatial sense could deepen this world. Changes that felt abrupt or inconsistent could break it. The adaptive condition therefore introduced a fundamental trade-off: Responsiveness can create relevance, but excessive change can reduce coherence.
Attention is not a single state to optimize
Meditation involves multiple processes: sustaining awareness, drifting, noticing that drift, and returning. An environmental event might temporarily capture attention while simultaneously helping a participant become aware of where their attention had gone. This makes “more attention” an insufficient objective for adaptive meditation systems.
Different people want different relationships with adaptive intelligence
Participants differed in how much initiative they wanted the environment to take. Some valued subtle external cues. Others preferred a stable environment that left regulation primarily to themselves. Personalization therefore involves more than calibrating EEG thresholds—it may require adapting the system's level and style of agency.
08/Reflection
From adaptive interfaces to adaptive relationships
NeuroScape began with a relatively technical question: Can EEG be used to adapt an immersive meditation environment in real time? Building and studying the system led me toward a different question: What kind of relationship should an intelligent environment have with the person it is sensing?
Physiological sensing gives AI systems access to increasingly implicit signals about us. Generative models give those systems greater capacity to interpret context and intervene in our surroundings. But greater responsiveness does not automatically produce better interaction. NeuroScape suggests that the design challenge is instead to create systems that can interpret uncertain evidence, reason about context, preserve continuity, calibrate their own initiative, and sometimes choose not to act.
For me, this project reframed adaptive intelligence from a problem of optimizing human states into a problem of negotiating agency between people and intelligent environments.
As AI becomes capable of sensing our bodies and intervening in our experiences, how do we design that intelligence while preserving our own capacity to notice, choose, and regulate ourselves?
Institution
Harvard GSD
Timeline
2026 Spring, 3 months
Team
Jianing Nomy Yu
Jingyi Zoe Liu
Xinyan Teresa Li
*Katarina Richter-Lunn (*supervisor)