Getting started

For AI agents: see llms.txt for the complete documentation index. Markdown versions are available by adding .md to a page URL or requesting Accept: text/markdown.

Call models from an action. The gateway authenticates with a short-lived token from getServiceToken("ai-gateway"), which needs convex 1.45+. Model names use the provider/model form in Models.

OpenAI SDK

The gateway is OpenAI-compatible. Set baseURL to the gateway and pass getServiceToken as the API key:

npm install openai

import { action } from "./_generated/server";
import { v } from "convex/values";
import OpenAI from "openai";
import { getServiceToken } from "convex/server";

export const chat = action({
  args: { prompt: v.string() },
  handler: async (ctx, { prompt }) => {
    const openai = new OpenAI({
      baseURL: "https://ai-gateway.convex.dev/v1",
      apiKey: () => getServiceToken("ai-gateway"),
    });
    const completion = await openai.chat.completions.create({
      model: "openai/gpt-4o-mini",
      messages: [{ role: "user", content: prompt }],
    });
    return completion.choices[0].message.content;
  },
});

Agent component

Pass convexGateway as languageModel on @convex-dev/agent. It obtains the token for you:

npm install @convex-dev/agent @convex-dev/ai-sdk-provider

import { Agent } from "@convex-dev/agent";
import { convexGateway } from "@convex-dev/ai-sdk-provider";
import { components } from "./_generated/api";

const agent = new Agent(components.agent, {
  name: "Support agent",
  languageModel: convexGateway("openai/gpt-4o-mini"),
  instructions: "You answer questions about our product.",
});

Workflows that call agent.generateText use that same model. The RAG component still needs its own embedding provider: the gateway does not serve /v1/embeddings yet.

Vercel AI SDK

convexGateway works with generateText and streamText from the Vercel AI SDK. It needs convex 1.45+ and AI SDK 7, and runs in the default runtime and Node.js actions:

npm install @convex-dev/ai-sdk-provider ai

import { action } from "./_generated/server";
import { v } from "convex/values";
import { generateText } from "ai";
import { convexGateway } from "@convex-dev/ai-sdk-provider";

export const chat = action({
  args: { prompt: v.string() },
  handler: async (ctx, { prompt }) => {
    const { text } = await generateText({
      model: convexGateway("openai/gpt-4o-mini"),
      prompt,
    });
    return text;
  },
});

Stream with streamText:

import { streamText } from "ai";
import { convexGateway } from "@convex-dev/ai-sdk-provider";

const result = streamText({
  model: convexGateway("openai/gpt-4o-mini"),
  prompt,
});
for await (const chunk of result.textStream) {
  console.log(chunk);
}

Manual fetch

Use fetch when you want the HTTP API without an SDK. Mint a token, then send it as Authorization: Bearer <token>:

import { action } from "./_generated/server";
import { v } from "convex/values";
import { getServiceToken } from "convex/server";

export const chat = action({
  args: { prompt: v.string() },
  handler: async (ctx, { prompt }) => {
    const token = await getServiceToken("ai-gateway");
    const response = await fetch(
      "https://ai-gateway.convex.dev/v1/chat/completions",
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${token}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: "openai/gpt-4o-mini",
          messages: [{ role: "user", content: prompt }],
        }),
      },
    );
    return await response.json();
  },
});

See HTTP API for request and response shapes.

Token and timeouts

getServiceToken("ai-gateway") returns a short-lived token scoped to your deployment. It only works inside a running action, and repeated calls in the same action reuse one token. Keep the token inside your action: don't return it to clients or store it in environment variables.

The request runs inside your action, so a long completion can hit the action timeout (30 minutes in the Convex runtime, 10 minutes in Node) and fail as an action error rather than a gateway error.

If getting a token fails with AiGatewayDisabled or AiGatewayUnavailable, see Who can use it.