The Challenge
When I started this project in Q1 2025, IBM faced a critical moment. We had $26M in revenue from 500+ watsonx Assistant voice customers requesting new voice agent features in order to migration to watsonx Orchestrate. Major clients, including UPS, CVS, and USAA, were requesting this capability.
The problem was clear: builders had to manually call in to test voice flows, resulting in robotic voices and inconsistent feedback. No controls existed for adjusting verbosity or speed. Without web access or a keypad interface, the entire process prevented rapid iteration and deployment.
My Role
As Sr. Design Manager and Lead, I led a team of seven designers while driving strategy across Product, Engineering, and AI Research. I personally led customer engagements with UPS, CVS, and USAA to validate design direction. Beyond team leadership, I stayed hands-on to solve complex problems: balancing design vision with platform constraints to avoid latency
The Solution
I scoped a plan to provide an MVP that incorporated the new orb component with four states: Idle (gentle pulsing, ready to listen), Listening (active feedback), Processing (thinking/formulating), and Speaking (dynamic movement synced to speech). The design integrated ElevenLabs Conversational AI to provide unified text-to-speech and speech-to-text, enabling browser-based testing with human-quality voice.
Impact
Builder confidence increased 29 percentage points from 57% to 86% across 50+ participants. We secured 8 enterprise pilots launching in Q1 2026 (Banorte, Riyadh Airlines), representing $10M in potential revenue. Response latency consistently hit 150-375ms, well below the 400ms threshold for natural conversation. The work earned a Red Dot Design Award.
Key Lessons
Balance matters: Negotiating between design vision and technical constraints pushed both sides to find better solutions. Phasing works: Our three-year roadmap secured buy-in and increased velocity. Validate early: Workshops and customer interviews prevented misalignment. Tools unlock possibilities: Finding Three.js transformed what we could achieve.
