Let the AI answer customers?
Work · Board of 3We built an AI assistant for support. In testing it resolves 62% of tickets correctly, and 4% of its answers are confidently wrong. Do we let it reply to customers directly, or only draft for agents?
Chief of staff
Accuracy, trust and support load — worth a board. I’d seat The Data Scientist, The Customer and The Ethicist.
The Data ScientistAI/ML Specialist
The CustomerUser Voice
The EthicistValues Guardian
The board’s call
Drafts only for now; let it answer directly on the three ticket types where it’s right 95% of the time.
Four customers in a hundred getting a confident wrong answer is a trust problem. On password resets, invoices and shipping status, the error rate is a fraction of that.

The Data Scientist disagreed
“Your test set is 400 tickets. That 95% could move a lot on real traffic — measure for two weeks before you switch.”
Example · Board of 3 · up to 135 credits






