- Type
- AI Tool
- Platforms
- iOS
- Status
- Live
- Role
- Design + engineering
The story
Why it exists
Apple Watch cannot reliably hold the outbound WebSocket needed for a realtime OpenAI voice session.
My role
Independent product designer and engineer. I designed the watch interaction, the phone bridge, and the realtime assistant pipeline.
Product move
Let the watch handle voice and interface while a paired iPhone owns the OpenAI connection and relays audio through WatchConnectivity.
What shipped
A working two-mode voice assistant with streaming speech, deeper reasoned answers, web search, watch states, phone history, and live diagnostics.
See the product
Product judgment
Key decisions
- 01
Bridge through the phone
The iPhone owns networking and credentials. The watch stays focused on capture, playback, and state.
- 02
Make system state visible
Listening, thinking, replying, interruption, and reachability are clear instead of hidden behind one ambiguous spinner.
- 03
Design for wrist-drop reality
Always-on-display behavior and a process-lifetime workaround keep stale state from misleading the user.
Build notes
Fast Mode uses OpenAI realtime speech-to-speech. Think Mode uses a turn-based reasoning pipeline with server-side web search.
A private HKWorkoutSession keeps the watch process alive without reading or writing health data. The OpenAI key never reaches the watch.
Built withSwift, SwiftUI, watchOS, WatchConnectivity, OpenAI Realtime API, OpenAI Codex
