What does AI agent performance optimization include? +
Profiling from traces, a baseline eval set built from your cases, and changes made one at a time: trimmed prompts and context, caching, model routing per step, batching and fewer tool round trips, plain code in place of steps that never needed a model, and retrieval tuning through chunking, hybrid search and reranking. Each change is rerun on the same eval set. The eval set, routing rules and cost dashboards stay in your repository.
When does an agent need optimization rather than a rebuild? +
When its answers are already useful and the problem is the bill, the wait or the share of answers that miss. If the answers are wrong because the process or the data behind them is wrong, tuning will not fix that, and the profile says so before any work starts.
Will cutting cost lower accuracy? +
It can, so every change runs against the baseline eval set; a change that lowers the score is rolled back. The exception is a trade-off you decide to keep, with the numbers on both sides in front of you.
Do we have to switch models or providers? +
No. Routing sends each step to the smallest model that passes its cases, and that may be the model you use today. The system stays model-agnostic, so hosted and open-weight models can be compared on the same eval set.
What do you need from us to start? +
Access to the agent's code, traces or logs from real runs, and a sample of cases with the answers you expect. If the agent has no tracing yet, we add it first and record a sample.
How do you measure latency and cost? +
From traces: duration, token counts and model for every step, plus the tool calls each step makes. Cost per run is computed from those token counts, split into cached, uncached and reasoning tokens, at your current provider prices, so before and after are priced the same way.
Does it work with any agent framework? +
Yes. Profiling, caching, routing and retrieval tuning apply to agents built with LangChain, LangGraph, LlamaIndex, Pydantic AI or plain code. We tune the framework you already run.
Can our team keep tuning after the engagement? +
Yes. The eval set, the routing rules and the dashboards live in your repository, with a runbook for testing a new model or prompt against the baseline.