Built a customer-support AI agent that deflected ~60% of tickets
A support team was drowning in repetitive tickets that the help docs already answered. An AI agent over those docs and past tickets, with order-lookup tools and a clean human handoff, took ~60% of volume off the queue.
The problem
A growing ecommerce team was answering the same questions all day. Roughly two thirds of inbound tickets were "where is my order", "how do I return this", and policy questions the help center already covered, but customers wanted an answer in chat, not a search box. Agents burned out on copy-paste while genuinely hard tickets waited.
The approach
- ▸Indexed the help center plus two years of resolved tickets into pgvector, so the agent answered from real wording customers understood, not just official docs.
- ▸Built the agent in LangGraph with tool-calling to hit the order API and returns system, so it could look up a specific order status instead of giving a generic reply.
- ▸Added a confidence threshold and a clean human handoff that passed the full transcript, so the agent escalated instead of guessing on anything it could not ground.
- ▸Ran it in shadow mode against live tickets for two weeks, scored its answers, and tuned retrieval and prompts before it ever talked to a customer.
The result
- ✓~60% of tickets resolved end to end without a human
- ✓First-response time dropped from ~4 hours to under 30 seconds for deflected chats
- ✓Human agents reclaimed roughly 25 hours a week for complex work
- ✓CSAT on agent-handled chats held within 3 points of human-handled ones
stack: LangGraph · RAG · OpenAI · pgvector · tool-calling
Deflection is a retrieval problem before it is an agent problem
The temptation is to make the agent clever. The actual win was making it accurate and honest about its limits. Most of the volume was a handful of question shapes, so the real work went into retrieval (indexing past tickets, not just docs, because customers ask in their own words) and into the handoff. An agent that confidently invents a refund policy is worse than no agent, so I set a confidence floor and let it say it was escalating. Customers do not mind being handed to a person. They mind waiting four hours for a one-line answer the bot could have given instantly.
Done by Harshit Luthra, an independent infrastructure and AI engineering consultant. Bring me a similar problem →
Questions about this work
How much support volume can an AI agent realistically deflect?+
Here it was ~60%, because that share of tickets was a handful of repeated question shapes the help center already covered (order status, returns, policy). The deflectable share depends on how repetitive your queue is, but most support teams have more of it than they think.
How do you stop a support agent from making up answers?+
A confidence threshold and a clean human handoff. The agent answered only what it could ground in the docs and past tickets, and escalated everything else with the full transcript. An agent that confidently invents a refund policy is worse than no agent, so it was built to say it was escalating instead.
How do you ship a support agent safely?+
Shadow mode. It ran against live tickets for two weeks, its answers were scored, and retrieval and prompts were tuned before it ever talked to a customer. Indexing two years of resolved tickets (not just official docs) is what let it answer in wording customers actually understood.