NOEMA
Wearable AI · Spatial Audio · Generative AI · Physical Computing
Exploring AI as a sensory extension that sees, interprets, and reimagines the world with you.
Overview
NOEMA is a wearable AI system that transforms visual surroundings into spatialized auditory experiences. Through environmental sensing, generative storytelling, and spatial audio, it explores how AI might become more than an interface—acting instead as a sensory extension that interprets and reimagines the world alongside the wearer.
01/Problem
What if AI became a layer of perception?
Most AI interfaces today are designed to answer questions, generate content, or improve efficiency. Wearable AI introduces a different possibility: when computation stays close to the body, senses, and surrounding environment, it can begin to shape not only what we know, but how we experience the world. In this project, we ask: How might wearable and generative AI reshape our perception of our surroundings through sound?
Why sound?
Sound allows information to remain present without occupying the visual field. More importantly, spatial audio can preserve qualities of physical space such as direction, distance, movement, and atmosphere, making it a compelling medium for exploring AI-mediated perception.
02/Inspiration
Looking back to imagine wearable AI differently.
NOEMA draws from a lineage of devices and artworks that have extended, redirected, or questioned human perception, from early acoustic listening apparatuses to experimental wearable interfaces that intervene in vision and hearing. Rather than treating wearables simply as smaller screens or more convenient assistants, these references inspired us to ask how a device worn on the body might actively reshape sensory experience.
03/Concept
From AI assistant to sensory extension
"Noema", meaning "mental object" in ancient Greek, is now used as a term in phenomenology referring to "the object or content of a thought, judgment, or perception." NOEMA extends this idea into a wearable Large Language Object (LLO): a system that transforms visual surroundings into spatialized auditory experiences.
"An AI that sees with you
but does not simply tell you what it sees."
Rather than aiming for a perfectly objective description of the environment, NOEMA allows interpretation, ambiguity, storytelling, and imagination to become part of the perceptual experience.
Embodied Interaction
Eye closure acts as the primary interaction trigger. When the wearer closes their eyes, NOEMA begins translating the surrounding environment into an auditory experience. Opening the eyes pauses the system and returns perception to the physical world.
04/System Design
NOEMA converts the surrounding environment into a spatial soundscape through a multimodal pipeline. Multiple camera views capture the wearer’s surroundings, while an AI model identifies relevant subjects, spatial relationships, and scene context. These elements are then transformed into generated audio and positioned back into space according to their relative direction and location.
05/Prototype
Software
NOEMA’s software prototype uses a sequence of prompts to progressively transform environmental perception into generative audio. We designed the pipeline around three stages:
-
See — ground visible subjects in physical direction and position.
-
Narrate — turn spatial relationships into a coherent story.
-
Generate — transform environmental elements into music, lyrics, and metaphor.
Hardware
The physical prototype integrates cameras, audio components, batteries, and custom-fabricated enclosures into a wearable form. Designing the device required balancing three competing priorities:
-
Perception — positioning cameras around the head to approximate a first-person spatial field.
-
Function — integrating sensing, audio, and electronics into the frame.
-
Wearability — iterating on scale, balance, comfort, and how the object sits on the body.
The final form emerged through repeated sketching, 3D modeling, fabrication, and fit testing.
06/Critical Reflection
Beyond AI as a Tool
NOEMA deepened my curiosity about AI-augmented reality, perception, and memory. When AI becomes more than an efficiency tool, but can be worn, embodied, and continuously present, how will the relationship between human and AI be shifted? Will human perception and machine interpretation continuously shape one another?
AI as Perceptual Extension: AI does not merely understand reality. It can participate in constructing how reality is perceived.
Institution
MIT Design Intelligence Lab
Timeline
2026 Spring, 6 weeks
Team
Jianing Nomy Yu
Qiyao Melo Chen
*Marcelo Coelho (*supervisor)