Debugging

For AI agents: see llms.txt for the complete documentation index. Markdown versions are available by adding .md to a page URL or requesting Accept: text/markdown.

Debugging in the Playground

Generally the Playground gives a lot of information about what's happening, but when that is insufficient, you have other options.

Logging the raw request and response from LLM calls

You can provide a rawRequestResponseHandler to the agent to log the raw request and response from the LLM.

You could use this to log the request and response to a table, or use console logs with Log Streaming to allow debugging and searching through Axiom or another logging service.

const supportAgent = new Agent(components.agent, {
  ...
  rawRequestResponseHandler: async (ctx, { request, response }) => {
    console.log("request", request);
    console.log("response", response);
  },
});

Logging the context messages via the contextHandler

You can log the context messages via the contextHandler, if you're curious what exactly the LLM is receiving.

const supportAgent = new Agent(components.agent, {
  ...
  contextHandler: async (ctx, { allMessages }) => {
    console.log("context", allMessages);
    return allMessages;
  },
});

Tracing LLM calls with OpenTelemetry

The AI SDK can emit OpenTelemetry spans with the model, token usage, prompt, and response for generate and stream calls. Install @ai-sdk/otel, @opentelemetry/api, @opentelemetry/exporter-trace-otlp-http, @opentelemetry/resources, and @opentelemetry/sdk-trace-base, then register the AI SDK's OpenTelemetry integration to enable them.

To export them to an OTLP backend, register a tracer provider and the AI SDK integration:

import { OpenTelemetry } from "@ai-sdk/otel";
import { trace } from "@opentelemetry/api";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { resourceFromAttributes } from "@opentelemetry/resources";
import {
  BasicTracerProvider,
  SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-base";
import { registerTelemetry } from "ai";

const tracerProvider = new BasicTracerProvider({
  resource: resourceFromAttributes({ "service.name": "convex-agent" }),
  spanProcessors: [
    new SimpleSpanProcessor(
      new OTLPTraceExporter({
        url: process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT,
        headers: {
          Authorization: `Bearer ${process.env.OTEL_EXPORTER_OTLP_TRACES_TOKEN}`,
        },
      }),
    ),
  ],
});
trace.setGlobalTracerProvider(tracerProvider);
registerTelemetry(new OpenTelemetry());

Then add metadata for the call, and flush the spans before the handler returns so their exports are awaited:

export const generateTextWithTelemetry = action({
  args: { prompt: v.string(), threadId: v.string() },
  handler: async (ctx, { prompt, threadId }) => {
    await authorizeThreadAccess(ctx, threadId);
    const result = await agent.generateText(
      ctx,
      { threadId },
      {
        prompt,
        telemetry: {
          functionId: "debugging/telemetry",
        },
      },
    );
    await tracerProvider.forceFlush();
    return result.text;
  },
});

See the full example in debugging/telemetry.ts.

Inspecting the database in the dashboard

You can go to the Data tab in the dashboard and select the agent component above the table list to see the Agent data. The organization of the tables matches the schema. The most useful tables are:

• threads has one row per thread
• messages has a separate row for each ModelMessage - e.g. a user message, assistant tool call, tool result, assistant message, etc. The most important fields are agentName for which agent it's associated with, status, order and stepOrder which are used to order the messages, and message which is roughly what is passed to the LLM.
• streamingMessages has an entry for each streamed message, until it's cleaned up. You can take the ID to look at the associated streamDeltas table.
• files captures the files tracked by the Agent from content that was sent in a message that got stored in File Storage.

Troubleshooting

Type errors on components.agent

If you get type errors about components.agent, ensure you've run npx convex dev to generate code for the component. The types expected by the library are in the npm library, and the types for components.agent currently come from generated code in your project (via npx convex dev).

dynamic module import unsupported from an unrelated query

Queries and mutations run in the Convex runtime, which cannot load Node.js built-in modules or packages that dynamically import them. If a tool's execute imports such a package, the whole module that defines the agent fails to load there, and the error surfaces from whichever query or mutation imports that module, typically the one listing thread messages.

Keep the Node-only work in its own file marked "use node" as an internalAction, and have the tool call it with ctx.runAction. The agent module then stays loadable from the Convex runtime. To find an import of a Node built-in such as fs, run this (it requires --once):

npx convex dev --once --debug-node-apis

It re-bundles each non-"use node" entry point and names the first unresolvable import. A package that bundles but calls import() at runtime will not show up here.

Circular dependencies

Having the return value of workflows depend on other Convex functions can lead to circular dependencies due to the internal.foo.bar way of specifying functions. The way to fix this is to explicitly type the return value of the workflow. When in doubt, add return types to more handler functions, like this:

export const supportAgentWorkflow = workflow.define({
  args: { prompt: v.string(), userId: v.string(), threadId: v.string() },
  handler: async (step, { prompt, userId, threadId }): Promise<string> => {
    // ...
  },
});

// And regular functions too:
export const myFunction = action({
  args: { prompt: v.string() },
  handler: async (ctx, { prompt }): Promise<string> => {
    // ...
  },
});