Our final project for 4.043 Interactive Intelligence Design Studio class. Josh, Krystal, and I made a fun thing. Here’s the full design crit deck here. Here’s our paper here
En Clair is a walkman interpreter for communication, because nothing is ever in plain language. It’s an investigation of a hardware AI object for subtext and context; you get to decide what filter each cartridge acts as. We were inspired by retro hardware designs, and Josh cooked up an awesome one. I worked on the embedded and hardware stuff to connect to the model as well as model prompt parameters, and Krystal set up the model connections and software iterations for it.
Abstract
Human communication is never lossless. What one person intends to say and what another person perceives are shaped by invisible filters — assumptions, emotional states, prior history, and cognitive biases — that neither party can directly observe. En Clair is a portable interactive device that makes this hidden filtering process audible and tangible. Used by a listener during a conversation, the device captures speech directed at them, processes it through a large language model conditioned by a user-defined perceptual filter, and plays back a transformed version of what was said through noise-cancelling headphones. The listener hears not what was said, but what their filter makes of it. Filters are encoded on physical NFC cartridges, each programmed by the user through a short spoken description; the same sentence may sound entirely different depending on which cartridge is loaded. Inspired by the Sony Walkman’s reframing of music listening as a personal and portable experience, En Clair turns miscommunication into a designed object — one that surfaces the subjective, constructed nature of what we hear. We describe the hardware architecture, software pipeline, and conceptual framing of the system, and discuss implications for the design of AI-mediated communication tools.
Introduction
Every act of listening involves interpretation. Long before a word is consciously processed, the brain applies filters — shaped by mood, relationship history, cultural context, cognitive load — that transform a speaker’s meaning into something that belongs, at least in part, to the listener. This filtering is normal, unavoidable, and almost entirely invisible. We rarely notice it happening, and we rarely have access to what it has done.
En Clair is a device that makes the invisible audible. The name En Clair, French for “in plain text,” is an irony: the device’s output is anything but plain. It is filtered, constructed, and subjective by design. It is carried by a listener during a conversation. The person speaking to them talks normally; the listener, wearing noise-cancelling headphones, receives a version of those words that has been processed through an LLM-based perceptual filter and played back through those same headphones via text-to-speech. The headphones do double duty: they block the actual voice of the speaker, and they deliver the filtered version in its place. The listener hears not what was said, but what their filter constructs from it. The filter is not fixed — it is defined by the listener themselves, spoken aloud and stored in reference to a small NFC cartridge that slots into the device. Each cartridge represents a different lens through which speech can be received: one might render everything through a lens of suspicion, another through excessive optimism, another through the interpretive habits of someone who always hears an implicit criticism. Swapping cartridges changes the filter; the same conversation produces entirely different audio.
The device is not a practical communication tool. It is an art installation and a designed object — an instrument for encountering, in a concrete and sometimes unsettling way, the degree to which what we hear is a construction. Its form deliberately echoes the Sony Walkman, a device that similarly privatized and transformed an experience (music listening) by inserting a technology between source and receiver.
This paper describes the design of En Clair: its conceptual basis, hardware and software architecture, the interaction model of cartridge-based filter programming, and the questions it raises about the role of AI in mediating human communication.
The paper is organized as follows. Section 2 situates En Clair within related work on mediated communication, data art, and physical AI. Section 3 describes the methodology. Sections 4–6 detail the software, electronics, and product design. Section 7 covers results and technical implementation. Sections 8–10 discuss implications, limitations, and future directions.
Methodology
En Clair was developed as a design-research project within the Interaction Intelligence studio (MIT 4.043), using iterative prototyping as the primary methodology. The project proceeded in parallel hardware and software tracks that were periodically integrated and tested together.
The conceptual framing preceded technical decisions. We began by defining the core proposition — that the device should externalize perceptual filtering rather than improve communication — and used this to evaluate all subsequent design choices. For example, the decision to deliver filtered speech as audio through headphones rather than as text on a display was driven by the desire for the experience to feel immersive and complete: the listener hears a filtered voice in the same sensory channel they would have heard the original. Similarly, the cartridge interaction was designed before the electronics that would implement it, because the act of physically inserting a filter object was considered essential to the meaning of the piece.
Software prototyping began with a browser-based prototype to validate the core pipeline: speech input → transcription → LLM filtering → audio output. This allowed rapid iteration on prompt architecture and filter behavior independent of hardware. The hardware track developed the ESP32-S3 firmware and device electronics in parallel, converging once both tracks had reached stable baselines.
User testing was conducted informally within the studio context, with fellow students and instructors experiencing the device during a conversation and providing feedback on the interaction model and the quality of the filtered output. This feedback shaped the final cartridge programming interaction and the device’s status display states.
Software Design
The software pipeline runs on a laptop acting as a server, connected to the ESP32-S3 device over USB serial. The pipeline has two distinct modes: cartridge programming and filtered listening. Cartridge programming. When a new, unseen NFC cartridge is inserted, the device prompts the user to describe the filter they want it to embody. The user holds a button and speaks a natural-language description — “someone who always assumes the worst,” “a person who only hears what confirms what they already believe,” “a very literal interpreter who misses all subtext.” This audio is captured by the INMP441 I2S microphone, streamed over serial to the laptop, transcribed via the OpenAI Whisper API, and then sent to Claude with a meta-prompt that converts the description into a system prompt suitable for use as a perceptual filter. The generated system prompt, along with a short name for the cartridge, is stored in a JSON database indexed by the NFC tag UID. On subsequent insertions of the same cartridge, this stored prompt is loaded immediately.
Filtered listening. During normal operation, pressing the listen button activates the microphone. Audio is streamed to the laptop, transcribed by Whisper, and then sent to Claude using the cartridge’s stored system prompt. The filtered text is returned, converted to speech via the ElevenLabs TTS API, and played back through the listener’s headphones. The conversation history is maintained across turns within a session to allow context-aware filtering — the filter can accumulate a model of the conversation, not just react to isolated utterances.
Prompt architecture. The system prompt architecture was designed to prevent a failure mode we call POV drift: as conversation history accumulates, the LLM output tends to shift from filtered second-person speech (“what the listener hears”) into interior monologue or third-person narration. We addressed this through two mechanisms. First, the system prompt explicitly labels perspective preservation as non-negotiable and enumerates all pronoun cases. Second, every user message sent to the API is wrapped with a per-turn anchor that restates the perspective constraint; this wrapper is also stored in the conversation history, so the constraint appears in every historical turn the model sees, not only in the system prompt. This substantially reduced drift in testing.
Electronics Design
The device is built around an ESP32-S3 DevKit as the central microcontroller, connected to a laptop over USB-C for power and serial communication. Audio input. An INMP441 MEMS microphone communicates with the ESP32-S3 over I2S (SD → GPIO 0, SCK → GPIO 45, WS → GPIO 48, L/R → GPIO 47). Audio is captured as 16-bit PCM at 16 kHz and streamed over serial to the laptop for processing.
Display. A Waveshare 2.13-inch e-paper display (V4) communicates over SPI (DIN → GPIO 10, CLK → GPIO 11, CS → GPIO 9, DC → GPIO 12, RST → GPIO 13, BUSY → GPIO 14). The e-ink display serves as the device’s status and interaction interface — showing cartridge state, filter names, and prompts during the cartridge programming flow — rather than as the output channel for filtered speech. Filtered speech is delivered as audio.
NFC. A PN532 NFC/RFID module (V3) communicates over I2C (SDA → GPIO 18, SCL → GPIO 46) and reads the NFC tags embedded in cartridges. Tag UIDs are sent to the laptop server for database lookup. Buttons. Four buttons handle the interaction model: a listen button (GPIO 2) activates the microphone for filtered listening; a cartridge listen button (GPIO 21) activates the microphone for cartridge programming; a confirm/select button (GPIO 20) confirms cartridge actions; and an eject button signals cartridge removal.
PCB. The device electronics are mounted on a custom double-sided FR1 PCB, milled on a Bantam desktop PCB mill. The ESP32-S3 devkit is mounted as an SMD component using its castellated edges, eliminating the need for plated through-holes.
Object Design
En clair can be handheld or worn at the user’s waist, drawing direct inspiration from the Sony Walkman form factor. Just as the Walkman transformed music listening into a personal, on-the-go experience, en clair seeks to carry that same personal device motif into a tool capable of adapting to a user’s individual needs across spontaneous interactions throughout their day. The device’s interaction is anchored by a modified scissor linkage mechanism, which enables it to remain hip-mounted in the Walkman tradition while simultaneously rotating the e-ink display upward to face the user whenever the device is opened. This pairing of screen rotation with the cartridge-swapping interaction was deliberate — on-screen prompting is only necessary during the filter selection process, so the display is made most accessible at that moment.
Cartridges serve as the primary input medium, each embedded with an NFC tag read by an onboard transceiver. This approach allowed for uniquely identifiable, swappable cartridges without requiring complex electrical solutions for unique cartridge signatures. Upon inserting a cartridge, the e-ink display presents the filter personality’s title and summary, prompting the user to either confirm the selection or record over it to create a new one. If the user opts to create a new filter, they can define a personality through natural language voice prompting. An e-ink screen was chosen deliberately over high-resolution or touch displays — its low power draw, legibility in varied lighting conditions, and visually quiet aesthetic reinforce the device’s ethos of calm, intentional interaction rather than constant digital engagement. Once satisfied with their filter, the user closes the device and presses the record button to begin transcribing an interaction, pressing it again when they wish to stop.
Results
Technical Implementation
The server runs as a Node.js/Express process (server.mjs) that manages serial communication with the ESP32-S3, NFC cartridge database I/O, OpenAI Whisper API calls for transcription, Claude API calls for filtering and cartridge prompt generation, and ElevenLabs TTS for converting filtered text to speech that is played back to the listener through their headphones. The NFC cartridge database is stored as a flat JSON file indexed by tag UID, with each entry containing the generated system prompt and a short human-readable filter name. New cartridges trigger a two-stage Claude call: first a meta-prompt generates a perceptual filter system prompt from the user’s spoken description, then that generated prompt is used for all subsequent filtering on that cartridge.
The ESP32-S3 firmware implements a state machine with four principal states: IDLE (waiting for cartridge), TAG_NEW (prompting user to define a new cartridge), TAG_KNOWN (confirming a known cartridge), and ACTIVE_FILTER (listening and displaying). Transitions between states are driven by NFC events, button presses, and messages from the laptop server. A waitingForServer flag gates NFC polling during server round-trips to prevent race conditions.
The e-ink display renders filtered text using a strip-based rendering approach in landscape orientation, with the Waveshare library modified to remove AVR-specific memory qualifiers (PROGMEM, avr/pgmspace.h) that are incompatible with the ESP32-S3’s Von Neumann architecture.
Filter behavior. In informal testing, the cartridge-defined filters produced outputs that were consistently recognizable as transformations of the input, while being meaningfully different from it. Filters defined around emotional biases (e.g., “someone who always feels criticized”) produced outputs that consistently reframed neutral speech as targeted or judgmental. Filters defined around interpretive habits (e.g., “a very literal reader”) stripped implied meaning and subtext. The two-stage meta-prompt approach — using Claude to generate the system prompt from a natural language description — proved more reliable than asking users to write system prompts directly.
A particularly striking result emerged when filters were defined using cultural or relational archetypes. A filter described as “an Asian mom” produced outputs that consistently reframed positive statements as implicit calls for improvement — for example, rendering “I’m proud of you” as “you can do better.” This was not a designed behavior; it emerged directly from the LLM’s encoding of cultural stereotypes in its training data. The filter did not invent this pattern; it surfaced one that already exists in the model. This raises questions the device was not originally intended to ask: when a filter reproduces a stereotype, is it externalizing a real perceptual pattern that some people experience, or is it fabricating and reifying a caricature? The device cannot answer this, but it makes the question concrete in a way that is difficult to dismiss.