Automated a manual ops workflow and saved ~20 hours a week
An ops team hand-sorted every incoming request, reading, categorizing, and routing each one by hand. An LLM pipeline that triages and routes with a human review step gave back ~20 hours a week.
The problem
An operations team handled every incoming request by hand. Someone read each one, figured out the category, attached the right context, and routed it to the correct queue. It was steady, error-prone work that scaled only by hiring, and the backlog grew whenever someone took a day off.
The approach
- ▸Mapped the existing manual process first, including the edge cases people handled by instinct, so the automation matched how the team actually worked.
- ▸Built an LLM pipeline that reads each request, classifies it, extracts the key fields, and proposes a route, instead of replacing judgment with a black box.
- ▸Kept a human-in-the-loop review step for low-confidence and high-stakes cases, so the team approved the hard ones while routine items flowed through.
- ▸Logged every decision with its confidence and reasoning, so the team could audit the pipeline and tune the rules over time.
The result
- ✓~20 hours a week of manual triage eliminated
- ✓Roughly 80% of requests routed automatically, with the rest flagged for human review
- ✓Average routing time fell from ~10 minutes per request to seconds
- ✓Misrouted requests dropped because classification stopped depending on who was on shift
stack: LangGraph · OpenAI · workflow automation · human-in-the-loop
Automate the routine, keep humans on the judgment calls
The mistake would have been trying to automate the whole thing. About 80% of the requests were routine and followed clear patterns, so the pipeline handled those cleanly. The other 20% were the cases that needed actual judgment, and forcing those through a model would have traded saved time for new mistakes. So the design kept a human review step for anything low-confidence or high-stakes, and logged every decision so the team could see why the pipeline did what it did. The hours came back from the routine work. The trust came from not pretending the model could handle the cases it could not.
Done by Harshit Luthra, an independent infrastructure and AI engineering consultant. Bring me a similar problem →
Questions about this work
Which ops workflows are worth automating with an LLM?+
High-volume work that follows loose rules and involves reading or writing text, like triage, routing, and field extraction. Here about 80% of requests were routine and the pipeline handled them cleanly. The 20% that needed real judgment kept a human, because forcing those through a model trades saved time for new mistakes.
How do you keep an automated pipeline from making bad calls?+
A human-in-the-loop review step for low-confidence and high-stakes cases, plus a log of every decision with its confidence and reasoning so the team can audit and tune it. Routine items flow through, the hard ones get a person. That's where the trust comes from.
How much time can automating triage actually save?+
This one gave back ~20 hours a week, with routing time per request dropping from ~10 minutes to seconds. The savings come from the routine 80%, not from pretending the model can handle the judgment cases.