The Numbers
Gorgias ran a blind benchmark across 13 AI customer support vendors, testing them on 212 live mid-market ecommerce stores with real inventory and real customer messages. No sandboxes, no vendor demos. Each agent got the same questions. A judge scored responses against 26 binary checks, with factual claims verified programmatically against live store data.
The top five vendors resolve 64% to 75% of conversations with no human involved. The median vendor resolves 48%. Here is the full field, with conversation volume in parentheses:
- Rep AI: 75% (120)
- Decagon: 73% (34)
- Yuma: 71% (118)
- Ada: 68% (420)
- Gorgias: 64% (388)
- DigitalGenius: 50% (561)
- Sierra: 48% (408)
- Siena: 46% (300)
- Kodif: 44% (188)
- Intercom: 42% (307)
- Zendesk: 37% (467)
- Envive: 22% (499)
- Klaviyo: 21% (309)
The AI bundled with your existing helpdesk is one of the weakest options available. Zendesk, Intercom, and Klaviyo all resolve under 42%.
Automation Rate Alone Is Not Enough
Automation rate without quality data will point you to the wrong vendor. Three vendors clear both 64% automation and 65 quality score: Yuma (71% automation, 72 quality), Decagon (73% automation, 70 quality), and Gorgias (64% automation, 66 quality).
Rep AI hits 75% automation but scores 55 on quality. Ada closes 68% of conversations but scores 39 on answer quality, 27 points below Gorgias. A ticket marked resolved that gives the customer the wrong return window creates a second contact that costs more than the first.
The Rate You See in the Pilot Is Not the Rate You Will Have Next Month
Nine of ten vendors with trend data lost automation in the last four weeks. Siena dropped 23 points. Kodif dropped 13. Klaviyo, Sierra, and DigitalGenius all dropped 10 or more. Quality moved the same way: Ada down 17 points, Zendesk down 15, Sierra down 14.
The benchmark does not explain the declines. Model updates, store configuration drift, or a harder question set could all contribute. Whatever the cause, you need to re-check agents on a schedule.
What Sales Teams Need to Know
Gorgias charges $0.90 per resolved conversation. Get each vendor's definition of a billable resolution in writing before you sign. In Gorgias's customer-facing reporting, an interaction counts as automated once the AI resolves it and 72 hours pass without a human agent helping. A customer who gave up and never came back counts as a resolution there. In this benchmark, that customer counts as a failure.
The benchmark counts a conversation as automated only when the AI handled it with zero human touch, no handover, and no deflection out of the channel. If more than half of an agent's replies push the customer out of the channel, the conversation is scored as unresolved.
For sales leaders evaluating AI support tools: the vendor with the highest automation rate is not necessarily the vendor that will reduce your support costs. Check quality scores. Check trend data. Check the definition of resolved. And check it again next quarter.