Our agent silently fails in production and we can't tell why — how do we get full traces of every step?

To get full visibility and stop silent failures in your AI agent, you need to instrument it with LLM Observability and Tracing . Without it, debugging an agent is like debugging a black box.

Here are the best ways to capture step-by-step traces, categorized by open-source frameworks and enterprise platforms.

1. Open-Source & Self-Hosted Tracing Tools

If your data privacy requirements prevent sending agent traces to third-party cloud platforms, these open-source tools allow you to self-host and keep data in your own infrastructure.

• Langfuse : An open-source LLM engineering platform that offers comprehensive tracing, prompt management, and evaluations. It captures nested calls, latency, and token usage easily via SDKs or integrations with LangChain, LlamaIndex, and native Python. Get started via the Langfuse Documentation.
• Arize Phoenix : An open-source AI observability tool built by Arize AI for troubleshooting LLM applications, embeddings, and agents. It excels at local evaluations, visualization of agent execution graphs, and debugging retrieval-augmented generation (RAG) loops. Explore the setup on the Arize Phoenix GitHub.
• OpenInference : A standard instrumentation library (often paired with Phoenix or OpenTelemetry) that standardizes how LLM inputs, outputs, and prompts are captured across different frameworks. Check out details on the OpenInference Repository.

2. Managed & Cloud Observability Platforms

If you prefer a hosted SaaS solution with minimal infrastructure overhead and advanced analytics dashboards:

• LangSmith : Built by the creators of LangChain, LangSmith provides deep debugging, testing, and monitoring for complex agent workflows. Even if you aren't using LangChain, you can use their SDK to log runs, steps, and tool calls. Sign up or view the guide on the LangSmith Platform.
• Braintrust : An enterprise-grade platform focused on logging, evals, and prompt playgrounds for AI engineering teams. It allows you to log agent runs in production and replay failing traces directly in a sandbox. Learn more at Braintrust.
• HoneyHive : Designed specifically to trace, evaluate, and fine-tune agentic workflows with granular step-by-step metrics on tool use accuracy and cost. Check out the platform via HoneyHive.

3. Native OpenTelemetry (OTel)

If your organization already uses traditional APM tools like Datadog, New Relic , or Dynatrace , you can use OpenTelemetry with LLM semantic conventions. This keeps your AI traces in the exact same dashboard as your database queries and microservice logs. Read the specification guidelines via OpenTelemetry Tracing.

If you'd like, let me know:

• What framework or stack your agent is built on (e.g., custom Python, LangChain, LlamaIndex, CrewAI)?
• Do you have data privacy constraints that require a self-hosted open-source tool, or is a cloud SaaS platform fine?

I can help you write the exact initialization snippet to hook up tracing for your specific setup.

If you’re using the OpenAI Agents SDK, you generally don’t need to instrument every step manually. Built-in tracing records the agent run as a trace containing spans for model generations, tool calls, handoffs, guardrails, and agent/turn execution. OpenAI GitHub citeturn0search1

1. Make sure production tracing is enabled

Tracing is enabled by default, so first check that you haven't disabled it with OPENAI_AGENTS_DISABLE_TRACING=1, set_tracing_disabled(True), or per-run configuration.

For each request, give the run a useful workflow name and attach correlation metadata:

from agents import Runner, RunConfig

result = await Runner.run(
    agent,
    user_input,
    run_config=RunConfig(
        workflow_name="production_customer_agent",
        group_id=conversation_id,
        trace_metadata={
            "request_id": request_id,
            "environment": "production",
        },
    ),
)

group_id is particularly useful when one conversation produces multiple traces.

2. Look at the trace hierarchy

A typical run gives you roughly:

Trace: production_customer_agent
└── Task
    └── Agent
        ├── Turn
        │   ├── LLM generation
        │   ├── Tool call
        │   └── Guardrail
        ├── Turn
        │   ├── LLM generation
        │   └── Handoff
        └── ...

Each span has timing and parent/child relationships, making it possible to see where execution stopped, which tool/model call preceded the failure, and how long each step took.

3. For steps outside the SDK, add custom spans

If the "silent failure" is in your own database/API/business logic, wrap it:

from agents import custom_span

with custom_span("fetch_customer_profile") as span:
    try:
        profile = await customer_api.get(customer_id)
        span.span_data.data["customer_id"] = customer_id
    except Exception as exc:
        span.set_error({
            "message": str(exc),
            "data": {"customer_id": customer_id},
        })
        raise

Custom spans automatically nest under the current trace.

4. Watch out for buffered exports

In long-running production workers, traces are normally exported in the background. If the process can terminate immediately after a job/request, explicitly flush traces so the final spans aren't left in memory. The Python SDK exposes flush_traces() for this case.

This is especially important in short-lived/serverless environments. For example, the JS SDK requires an explicit forceFlush() in environments such as Cloudflare Workers where the normal export loop isn't available.

5. Decide how much payload data you actually want

There are two separate things:

• Spans/timing/errors — useful for diagnosing execution.
• LLM/tool inputs and outputs — much more useful for reproducing behavior, but potentially sensitive.

You can keep the spans while excluding potentially sensitive payloads with:

RunConfig(trace_include_sensitive_data=False)

The default is currently to include sensitive trace data, so review this carefully for production workloads and your data-handling requirements.

For a production "silent failure" problem, I'd start with full span tracing + request/conversation IDs + explicit error spans, then temporarily enable payload capture only in a controlled debugging environment if the trace shows which step failed but not why.

A silent failure in an AI agent usually means a step is timing out, hitting a rate limit, failing a hidden schema validation, or encountering an unhandled tool error that gets swallowed.

To get full visibility and trace every single step (LLM calls, prompt inputs, token counts, tool outputs, and latency), you need to integrate an LLM observability or tracing framework.

• OpenInference / OpenTelemetry: Standardized, vendor-neutral telemetry that integrates with standard APM tools.
• Arize Phoenix: Great open-source tool for AI observability, evaluations, and deep notebook-based tracing.
• LangSmith: Deep tracing, debugging, and evaluation suite built natively for agentic workflows (especially if you use LangChain/LangGraph, but works with anything).
• Braintrust: Excellent for logging production runs, evaluating outputs, and tracking step-by-step agent traces.
• Phoenix / Arize: Great open-source options if you prefer self-hosting or Python-native telemetry inspection.
• Langfuse: Open-source LLM engineering platform with fantastic production tracing, cost tracking, and prompt management.
• Arize Phoenix: Excellent open-source library for ML/LLM observability and deep trace visualization.

To help narrow down the best fix, tell me:

• What framework are you building the agent with (LangChain, LlamaIndex, CrewAI, or raw custom code/API calls)?
• What infrastructure or cloud are you running production on?

If you’re using the OpenAI Agents SDK, the first thing I’d do is turn on its built-in tracing rather than relying only on application logs. The SDK is specifically designed to visualize agent execution, including multi-step workflows and tool use.

What to capture

For every production run, you want a trace tree roughly like:

trace: request_abc123
├── agent: triage
│   └── model call
├── tool: search_customer
│   └── tool result
├── handoff: billing_agent
│   ├── model call
│   ├── tool: get_invoice
│   └── tool result
└── final model call

Also attach your own correlation fields:

trace_id
request_id
user/session_id       # preferably a non-PII internal ID
deployment/version
agent_version
model
environment

That lets you go from "request X failed" → complete agent trajectory → exact failing tool/model step.

Don't rely on traces alone

For production, I'd use three layers:

1. Agent tracing — every agent invocation, model call, tool call, handoff, guardrail, and error.
2. Structured application logs — emit trace_id/request_id on every service log, including exceptions and timeouts.
3. Metrics — count failures by agent/tool/model and track latency, retries, token usage, and timeout rates.

The important part is that exceptions and cancellations must be recorded as trace events before your worker exits. Otherwise the classic production failure is: the process dies after a tool call and you have a trace that simply appears to stop.

Make silent failures impossible

Wrap the outer run with something equivalent to:

try:
    result = await Runner.run(agent, input)
    logger.info("agent_completed", extra={
        "trace_id": trace_id,
        "status": "success",
    })
except Exception:
    logger.exception("agent_failed", extra={
        "trace_id": trace_id,
        "status": "error",
    })
    raise

And instrument your own tools too. A trace should show both:

tool_started
tool_finished
tool_failed

with duration and a sanitized error—not just the model's decision to invoke the tool.

One important caveat

"Full trace" should not mean indiscriminately storing secrets, credentials, or sensitive user data. Define a redaction policy before enabling production tracing. You want enough information to reproduce the failure without turning your observability system into a copy of your entire database.

If you're using a different agent framework (LangGraph, CrewAI, custom orchestration, Responses API directly, etc.), tell me which one and I can give you the exact production tracing setup. OpenAI's current Agents SDK is the recommended direction for code-based agent workflows; OpenAI has also announced that Agent Builder/Evals are being wound down later in 2026.

tool_started
tool_finished
tool_failed

with duration and a sanitized error—not just the model's decision to invoke the tool.

One important caveat

"Full trace" should not mean indiscriminately storing secrets, credentials, or sensitive user data. Define a redaction policy before enabling production tracing. You want enough information to reproduce the failure without turning your observability system into a copy of your entire database.

Yes. The key is to treat an agent run as a distributed trace, not as a stream of logs.

What to capture

Create one root trace per user/request, then make every meaningful operation a child span:

trace: agent_request
├── agent_turn
│   ├── LLM generation
│   ├── tool: search
│   │   ├── HTTP request
│   │   └── response
│   ├── tool: database
│   └── LLM generation
├── handoff → specialist_agent
│   ├── LLM generation
│   └── tool: ...
└── final response

OpenTelemetry explicitly models traces this way: spans form a parent/child tree, and spans can carry attributes, timestamped events, status, and exceptions.

For every span, record at least:

• start/end time and duration
• trace_id / span_id / parent span
• agent name/version
• model + model version
• prompt/input and output where appropriate
• tool name, arguments, result/error
• token usage and latency
• retries/timeouts
• exception + stack trace
• guardrail decisions
• handoffs
• deployment/version/environment
• your own request/job ID

For things that happen inside a span—e.g. "tool_retry", "fallback_triggered", "validation_failed"—use timestamped span events rather than creating dozens of tiny spans.

If you're using the OpenAI Agents SDK

You may not need to build this instrumentation yourself. The SDK's built-in tracing records agent runs, LLM generations, function/tool calls, handoffs, guardrails, and custom spans. Its default hierarchy is essentially:

Task → Agent → Turn → Generation/Tool/Guardrail/Handoff.

So the first things I'd check in production are:

1. Is tracing actually enabled? The SDK can be disabled globally or per run.
2. Are traces being flushed? Long-running workers can finish work before their background exporter has delivered the trace; the Python SDK provides flush_traces() for an immediate delivery guarantee.
3. Are custom tools instrumented? Wrap important application logic in custom spans so the trace doesn't have a giant opaque "tool call."
4. Are trace IDs propagated across services/queues? OpenTelemetry's TraceId/SpanId context needs to propagate across process boundaries, otherwise your trace gets fragmented.
5. Correlate logs with trace_id and span_id. Then an exception in a normal application log immediately takes you back to the exact agent step.

The production setup I'd recommend

Use OpenTelemetry as the underlying instrumentation standard, export traces to your observability backend, and keep structured logs correlated with those traces. If you want an agent-focused UI, tools such as LangSmith Observability provide step-by-step agent trajectories and production monitoring.

One important caveat: "full traces" can contain sensitive data. Agent tracing may capture prompts, model outputs, and tool inputs/outputs. The OpenAI Agents SDK specifically provides configuration to disable sensitive-data capture.

If you're seeing silent failures specifically, I'd prioritize capturing the negative paths: exceptions, tool timeouts, empty results, retries, aborted spans, guardrail rejections, and the final decision that caused the agent to stop. That's usually where a normal "successful request" log completely hides the actual failure.

Silent failures in production are the worst. To get full, step-by-step visibility into your AI agent, you need to instrument it with LLM Observability and Tracing tools . This captures inputs, outputs, prompts, tool calls, latencies, and token costs for every single step of execution.

Top tools and strategies to get full traces in production:

• OpenTelemetry (OTel) standards: Ideal if you want vendor-agnostic, enterprise-grade telemetry. You can use standard tracing libraries to export data to compatible backends.
• Dedicated LLM Observability Platforms: These platforms provide native agentic visualization (showing loops, tool calls, and reasoning steps) out of the box with minimal configuration:
  • LangSmith for tracing complex chains, agents, and multi-step workflows (works with or without LangChain).
  • Arize Phoenix for open-source AI observability, evaluations, and robust notebook/production tracing.
  • Braintrust for production logging, evals, and debugging data infrastructure.
  • Arize AI for a comprehensive ML/LLM observability platform.
  • HoneyHive for tracking agent performance, costs, and failure modes in production.
  • Portkey for an LLM gateway with built-in request tracing and debugging.
• Lightweight/Open-Source Tracing:
  • Logfire by Pydantic for incredibly neat, Python-native tracing that highlights errors and steps automatically.
  • OpenInference for standardizing LLM telemetry instrumentation.

To help narrow down the best setup, tell me:

• What framework are you using to build the agent (LangChain, LlamaIndex, CrewAI, or custom Python/TypeScript)?
• What is your hosting/infrastructure environment (AWS, GCP, Modal, Vercel, etc.)?

I can give you a quick code snippet to wire up the right tracer immediately.

If you're using the OpenAI Agents SDK, you generally don't need to instrument every step yourself. Built-in tracing records the agent run, model generations, tool calls, handoffs, guardrails, and custom spans, and exposes them in the Traces dashboard.

1. Make sure tracing isn't disabled

Check production for:

OPENAI_AGENTS_DISABLE_TRACING=1

and, in Python, make sure you aren't calling:

set_tracing_disabled(True)

Tracing is enabled by default.

2. Put the whole workflow inside a trace

For example:

from agents import Agent, Runner, trace

agent = Agent(
    name="ProductionAgent",
    instructions="...",
)

async def handle_request(user_id, prompt):
    with trace(
        "production_agent",
        group_id=f"user:{user_id}",
        metadata={"environment": "production"},
    ):
        result = await Runner.run(agent, prompt)

    return result.final_output

The SDK automatically creates nested spans for the important operations, so you can see where the agent stopped, rather than just getting an application-level error.

3. Flush traces before short-lived workers disappear

This is a particularly common production gotcha. Traces are normally exported in batches in the background. For Celery/RQ/FastAPI background jobs or other workers that can terminate immediately after a task, explicitly flush:

from agents import Runner, flush_traces, trace

def run_job(prompt):
    try:
        with trace("production_job"):
            result = Runner.run_sync(agent, prompt)
            return result.final_output
    finally:
        flush_traces()

flush_traces() forces currently buffered spans to be exported, which is useful when you need the trace to be visible immediately after a unit of work finishes.

4. Include enough metadata to find the bad request

I'd attach things like:

• deployment/environment
• application request ID
• customer/session ID (preferably a non-sensitive internal identifier)
• agent/workflow version
• feature or route
• model configuration

You can set workflow_name, trace_id, group_id, and arbitrary trace_metadata/metadata for this purpose.

5. Be deliberate about sensitive data

By default, traces can contain LLM inputs/outputs and function-call inputs/outputs. If those may contain secrets or customer data, configure trace_include_sensitive_data=False (or the corresponding environment setting).

For a production incident, I'd also correlate your application request ID ↔ trace ID in your normal logs. That gives you a path like:

HTTP request → application log → agent trace → model/tool span → failure

rather than relying on traces alone.

If you're using the JS/TypeScript Agents SDK instead, the same built-in trace/span model exists, but short-lived runtimes such as Cloudflare Workers need an explicit forceFlush() before the runtime exits.

OpenAI Agents SDK tracing documentation 

To capture full traces of every step and stop silent failures in production, you need observability specifically built for LLMs and autonomous agents . Standard Application Performance Monitoring (APM) tools like Datadog or New Relic usually only show HTTP request status or latency, missing the internal reasoning loops, tool calls, and prompt/response payloads of your agent.

Top Observability Platforms for Agent Tracing

• LangSmith — Deeply integrates with LangChain/LangGraph but works with any custom Python/TypeScript codebase via simple SDK decorators or environment variables to capture complete run trees, latencies, and token counts.
• Arize Phoenix — An open-source AI observability tool great for local evaluation, debugging, and tracing complex agent steps using OpenInference standards.
• Honeyhive — Focused on production tracing, debugging, and continuous evaluation for LLM applications with fine-grained event logs.
• Braintrust — Enterprise-grade logging, evals, and tracing with low overhead and robust data privacy controls.

Implementation Checklist

1. Instrument with OpenTelemetry (OTel) : If you want vendor-neutral tracing, use OpenTelemetry libraries designed for LLMs (like OpenInference) to export traces to backend systems like Jaeger, Honeycomb, or Datadog's APM.
2. Capture Metadata & Context : Ensure your traces log the exact prompt input, intermediate thought/reasoning steps, tool inputs and outputs (e.g., SQL queries, API payloads), and system errors or retries.
3. Set Up Alerting : Configure alerts on unhandled exceptions, infinite loops (high step counts), or sudden spikes in token consumption/latency which often indicate silent loops or deadlocks.

If you can share:

• What framework (LangChain, LlamaIndex, CrewAI, or custom Python) your agent uses
• Your production infrastructure (AWS, GCP, Kubernetes, or serverless)

I can give you the exact setup code or configuration to get your tracing pipeline running.

To get full, step-by-step traces of your AI agent in production and eliminate "silent failures," you need to instrument your application with LLM Observability and Tracing tools.

Here are the best ways to capture every prompt, response, tool call, and latency metric:

• Instrument with OpenInference / OpenTelemetry : Use open standards to automatically capture inputs, outputs, and tokens without locking yourself into a single vendor. Many tools natively ingest OpenTelemetry.
• Adopt an LLM Observability Platform : Integrate dedicated tracing SDKs (like LangSmith, Arize Phoenix, LangFuse, Braintrust, or HoneyHive) into your agent framework (LangChain, LlamaIndex, or custom Python/TS loops).
• Enable Verbose Logging and Callbacks : If using a framework, turn on its built-in debugging/callback handlers to pipe structured JSON logs to your standard log aggregator (Datadog, AWS CloudWatch, or Grafana).
• Track State and Memory Transitions : Ensure your tracing captures the exact state of the agent's memory/scratchpad at each iteration to see where it got stuck in a loop or returned an empty tool output.
• Set up Error Boundary Try/Catch Blocks : Wrap individual tool executions and LLM calls in granular try/except blocks that log the exact stack trace and payload before a failure can be silently swallowed.

If you can share:

• What framework your agent is built on (LangChain, CrewAI, custom Python, etc.)?
• Where your application is hosted (AWS, GCP, Modal, Vercel, etc.)?

I can give you a specific code snippet or integration guide to set up tracing immediately.

If you’re using the OpenAI Agents SDK, the built-in Tracing & Observability is designed for exactly this: it records the agent workflow so you can inspect model calls, tool calls, handoffs, guardrails, and other steps.

1. Enable tracing in production

For Python, make sure your tracing export is configured and wrap the top-level workflow in a trace:

from agents import trace, Runner

async def handle_request(request):
    with trace("production-agent"):
        result = await Runner.run(
            my_agent,
            request,
        )
        return result.final_output

For a multi-agent workflow, put the whole workflow inside the trace rather than creating an unrelated trace around each individual agent:

with trace("customer-request"):
    result = await Runner.run(triage_agent, user_input)

    # handoffs / subsequent agent execution
    result = await Runner.run(specialist_agent, result.final_output)

    return result.final_output

The SDK's tracing is specifically intended to visualize these multi-step executions.

2. Don't rely on application logs alone

Give every incoming request a correlation ID and put it into the trace metadata/name. Then your production logs can point directly to the corresponding agent trace:

with trace(f"request:{request_id}"):
    ...

Also log boundaries around your own code—database calls, HTTP requests, queues, retries, etc. An agent trace can tell you what the agent did, but your surrounding infrastructure can still be where the silent failure occurs.

A useful production record is:

request_id
trace_id
agent
step
tool
started_at
finished_at
duration_ms
status
exception
retry_count

3. Make failures impossible to silently disappear

Wrap the top-level execution:

try:
    with trace(f"request:{request_id}"):
        result = await Runner.run(agent, input)
        return result.final_output

except Exception:
    logger.exception(
        "agent_failed",
        extra={"request_id": request_id},
    )
    raise

If you're seeing a trace that simply stops, this is particularly important: distinguish the agent stopped normally, the process threw, the worker died, the request timed out, and the trace wasn't flushed/exported.

4. If traces are incomplete

There have been reports of incomplete traces in particular multi-agent/async setups, so don't assume that an apparently truncated trace proves the agent never executed the missing work.

For production, I'd use two layers:

Agent tracing → model calls, tool calls, handoffs, guardrails, agent steps

Distributed application tracing → HTTP request → queue → worker → agent → database/API → response

That combination lets you answer “where did the request disappear?” even when the agent itself isn't the culprit.

If you tell me whether you're using the Python or TypeScript Agents SDK (and your deployment stack, e.g. FastAPI + Celery/ECS/Lambda), I can show you the exact production tracing setup, including how to capture every tool call and correlate it with your server logs.