[Remote] CX Automation Performance Analyst
Note: The job is a remote job and is open to candidates in USA. Betterment is a leading, technology-driven financial services company that offers investing, savings, and retirement solutions. The CX Automation Performance Analyst is responsible for managing the post-launch performance of AI programs, ensuring they function effectively and continuously improve through ongoing optimization and documentation.
Responsibilities
- Build and maintain dashboards tracking performance across all live AI programs — automation rate, containment, AHT, CSAT, and model quality metrics — using tools like Zendesk Explore, Looker, or similar
- Identify performance degradation, intent gaps, hallucination, model drift, and guardrail failures across Zendesk AI, chatbots, ticketbots, and automation workflows
- Own the incident response process for AI quality issues — when performance degrades or a guardrail fails, you triage, escalate, and drive resolution with a clear post-mortem
- Serve as the primary admin for net-new AI tooling post-launch — including VoiceAI platforms, Zendesk AI agents, and any other tools handed off from the implementation phase
- Maintain active optimization backlogs for launched programs, ensuring improvements don't stall after initial implementation
- Drive post-launch iteration in partnership with the Chatbot Strategy Manager and tooling teams, tracking outcomes rigorously
- Own the AI program documentation library: runbooks, flow documentation, QA procedures, training guides, and internal knowledge base articles
- Maintain accurate records of AI program behavior, guardrails, and change history to support compliance and audit readiness
- Partner with the CX AI & Automation Manager during launch phases to establish performance baselines, define QA criteria, and ensure clean handoff documentation
- Build feedback loops from AI performance data to inform product, engineering, and CX leadership decisions — your insights should shape what gets built next, not just what gets fixed
Skills
- Hands-on experience analyzing performance data from AI or automation tools in a support environment
- Understanding of common AI failure modes — hallucination, model drift, guardrail gaps, and intent misclassification — and the ability to design QA processes that catch them before they reach customers
- Proficiency with data visualization and reporting tools (e.g., Zendesk Explore, Looker, Tableau, or similar) and comfortable writing queries to validate AI performance data
- Strong documentation skills — you write clearly, organize well, and build resources people actually use
- Analytical and detail-oriented: comfortable building dashboards, running QA analysis, and spotting patterns in messy data
- Experience with Zendesk (or comparable CX platforms) and AI tools such as Zendesk AI, Decagon, or similar
- Experience in fintech, financial services, or another regulated industry where AI outputs are subject to compliance review and audit
- A continuous improvement mindset and systems thinker — you see how knowledge structure, prompt design, monitoring, and feedback loops connect, and you're never satisfied with 'it launched'
Benefits
- Medical, dental, and vision coverage
- Life and AD&D insurance
- Short- and long-term disability
- Infertility support and WPATH-aligned transgender health benefits
- An Employee Assistance Program (EAP)
- Transit benefits and FSA and HSA options
- Equity for all employees, including new hire and refresher grants.
- Flexible paid time off, paid parental leave, and a fully paid four-week sabbatical in your sixth year.
- Company-paid professional coaching for all employees.
- Day-one 401(k) match plus matching on qualified student loan payments.
Company Overview
Company H1B Sponsorship