Retail · 2026
A WhatsApp agent that knows when to call a human
Consultative support over WhatsApp: understands the need, recommends from a proprietary catalogue, answers conceptual questions through retrieval, and hands off to a specialist with a structured brief.
My role · My project
From graph design to deploy, including the guardrails and the evaluation suite.
- 3
- clients in production on the same architecture2026 · tuned per operation, not rewritten
- 14
- evaluation scenarios running in CIsince Jun 2026
- 9
- nodes in the conversation graphcurrent architecture
This case describes architecture and decisions without exposing proprietary code, credentials, clients or operational data.
The problem
Consultative support over WhatsApp breaks the naive message → model → reply loop. People write in bursts, the channel redelivers the event if you are slow, and the conversation has to remember yesterday. Any one of the three, on its own, produces support that feels broken.
The real constraints
- People send three messages in eight seconds. That is one thought — replying to each produces three disjointed answers.
- The webhook cannot wait for the model. If the reply is slow, the WhatsApp API redelivers the event and the conversation duplicates.
- The conversation is long and has to remember. State does not fit in a prompt and cannot live in process memory, which dies on the first redeploy.
- The agent must not invent a price or promise a delivery date. In consultative sales, getting that wrong costs more than not answering.
Architecture
- load — Hydrates state from the Postgres checkpointer and the accumulated memory.
- understand — Extracts intent, entities and what is still unknown about the person on the other side.
- route — Decides: keep talking, book a slot, or hand off to a human.
- retrieve — Resolves the canonical catalogue and, for conceptual questions, queries the vector index.
- schedule — Talks to the real calendar and returns slots that actually exist.
- compose — Writes the reply using only the context that was retrieved.
- guard — Output barrier: no invented prices, no promised dates, nothing outside the catalogue. Runs on the finished text.
- persist — Stores memory and accumulated understanding for the next turn.
- handoff — Builds the structured brief and delivers it to the human specialist.
Decisions and what they cost
The guard runs after composition, not inside the prompt
- Chose
- Check the finished text against explicit rules, as the last step before persisting.
- Rejected
- Trusting a prompt instruction to stop the model inventing prices and dates.
- Cost
- One extra pass per reply and a rule set to maintain. In exchange, the failure mode stops being probabilistic: asking in the prompt reduces the chance, checking the output removes the case.
Typed state instead of a loose dictionary
- Chose
- An explicit contract between graph nodes, checkable before anything runs.
- Rejected
- Passing a free-form dictionary from node to node, which is faster to write.
- Cost
- More ceremony to add a field. In exchange, renaming something breaks at import time rather than at two in the morning in production.
Two engines, selected by environment variable
- Chose
- Keep the old imperative orchestrator alive next to the new graph, switchable without a redeploy.
- Rejected
- Migrating in one go and deleting the old path.
- Cost
- Duplicated code and two implementations to maintain during the transition. In exchange, rollback is an environment variable rather than a long night.
Phone numbers pseudonymised before leaving the process
- Chose
- Salted hash applied before any log line or observability trace.
- Rejected
- Logging the raw number, which is easier to debug when something breaks.
- Cost
- Debugging a specific case means recomputing the hash. In exchange, no trace hands back anyone's phone number.
Outcome
- The agent carries the whole conversation, recommends from the catalogue and hands the specialist a structured brief instead of a raw transcript.
- Automated evaluation runs with a model as judge and returns a binary exit code, so it sits in CI like any other test. The scenarios cover what actually hurts: invented prices, promised dates, answers outside the catalogue, and losing the thread between turns.
- Engine rollback without a redeploy throughout the migration to the graph.
What I took from it
A guardrail inside the prompt is a request; a guardrail outside it is a guarantee. I started treating model output as untrusted input — the same posture everyone has towards user input, and which somehow gets dropped when the text comes from an LLM.
Stack
- LangGraph
- Python
- FastAPI
- PostgreSQL
- RAG
- LLM
- Langfuse
- Docker