3: The platform in action

You are here

Convex Tour

1. The reactor
2. Convex and your app
3. The platform in action

   Learn about harnessing the broader backend platform capabilities to connect the Reactor to external resources like third-party APIs

In Parts 1 and 2 you iterated on a fullstack chat app, using query and mutation functions to implement new business logic and the Convex React client to invoke those functions from the frontend.

So far, all the app's data and functions have been self-contained within the Convex platform: a pure, deterministic oasis where you enjoy end-to-end type-safety and transactional guarantees that your data will never be incorrect or inconsistent.

But what happens when you need to interact with the "real" world outside of Convex? How do you call external APIs, access data stored elsewhere, or perform any other "side effects" your specific app might need?

To find out, in this module you'll modify the chat app to integrate an AI chat agent powered by a third-party API! Along the way, you'll learn how to:

• Write Convex actions to connect the Router to arbitrary external resources
• Set needed environment variables to securely access e.g. API keys from your Convex functions
• Access an API from a Convex action
• Use the Convex scheduler to invoke an action from a mutation

Ready to get your AI on? Let's go!

Screenshot of chat app with an AI agent embedded in the chat

Before you begin: Get a TogetherAI Account & API Key

The AI agent in the chat will be powered by the Together AI API.

In order to access this API, you'll need to create a free Together AI account and copy your API key.

AI API FTW!

Let's create an AI agent that will be able to answer users' questions from the chat. We'll use Together's Chat Completions API to request "completions" (responses) to user input, specifying one of the available language models to generate the response text.

Connect your project to Together AI

Exercise 3.1

Set your Together API key

On the Convex dashboard (npx convex dashboard), navigate to your deployment Settings page. The Environment Variables tab lets you add new variables to your environment, which is the safest way to access a secret like an API key from your functions.

Create an environment variable named TOGETHER_API_KEY and set its value to the key you copied earlier from your Together account.

Now your Convex functions have secure access to your token, available in your function code as process.env.TOGETHER_API_KEY.

Write your first action

Fire up Fetch

Query and mutation functions run deterministically, enabling transactional guarantees that keep your data consistent, correct, and automatically reactive. For this reason they cannot call third-party APIs or otherwise interact with the outside world. Actions are the escape hatch you need for those use cases.

Like queries and mutations, actions live in TypeScript modules within the convex/ directory in your project's root.

Exercise 3.2

Create a module for your AI action

Create a new file convex/ai.ts where you'll eventually make a request to Together's API and parse the response to create a new message from the AI agent. To start, grab the API key from the environment variable you set in your project.

convex/ai.ts

TS

const TOGETHER_API_KEY = process.env.TOGETHER_API_KEY!;

Start up the development server with npm run dev, if it's not running already.

What are the differences in how code runs in query/mutation functions vs. actions?

Actions run by default in the same Default Convex Runtime as queries and mutations.

For cases where you need libraries or features the default runtime doesn't support, Convex actions can be configured to run in a Node runtime using the "use node" directive.

Read about the advantages and disadvantages of each runtime in detail here: Function Runtimes

A little less conversation, a little more action

OK, now you're ready to actually get your AI on!

Analogous to query() and mutation(), Convex provides an action() constructor that defines an action function, which accepts an object defining the function's args and handler.

Exercise 3.3

Get ready for action

In convex/ai.ts, import the action() constructor and export a new action called chat that accepts a messageBody string as its argument. Similar to mutations, action handlers accept two arguments: an ActionContext ctx and an arguments object as defined in args.

Solution

convex/ai.ts

TS

import { action } from "./_generated/server";
import { v } from "convex/values";

// Grab the API key from the set environment variable
const TOGETHER_API_KEY = process.env.TOGETHER_API_KEY!;

export const chat = action({
  args: {
    messageBody: v.string(),
  },
  handler: async (ctx, args) => {
    // TODO
  },
});

In the action's handler, you'll write the body of your Action function to call TogetherAI's Chat Completions API and send a new message with the AI-generated response.

Looking at the API documentation, in the request body it expects a language model name (see here for a list of models you can use with Together's API) and a messages array that gives the model the context of the chat to be completed.

Exercise 3.4

Complete the chat

Complete the handler function body to get a response from Together's chat completions API.

Pass meta-llama/Llama-3-8b-chat-hf as the model name (or another available model, if you choose), and in the messages array provide two message objects:

• one 'system' message that tells the AI agent how to respond, and
• one 'user' message that contains the message content to respond to.

Then, extract the text content of the response, which is a nested object of the form (simplified):

{
  choices: [{ message: { content: "This is the magical AI response text" } }];
}

Solution

convex/ai.ts

TS

  handler: async (ctx, args) => {
    const response = await fetch(
      "https://api.together.xyz/v1/chat/completions",
      {
        method: "POST",
        headers: {
          // Set the Authorization header with your API key
          Authorization: `Bearer ${TOGETHER_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: "meta-llama/Llama-3-8b-chat-hf",
          messages: [
            {
              // Provide a 'system' message giving context about how to respond
              role: "system",
              content:
                "You are a terse bot in a group chat responding to questions with 1-sentence answers.",
            },
            {
              // Pass on the chat user's message to the AI agent
              role: "user",
              content: args.messageBody,
            },
          ],
        }),
      },
    );

    // Pull the message content out of the response
    const json = await response.json();
    const messageContent = json.choices[0].message?.content;
  },

Almost there! Time to go back to the first argument of the action's handler function. When the action runs, Convex will pass an ActionContext as the first argument to the handler, which includes the utility method runMutation (along with runQuery and runAction, which you don't need right now). This gives actions the opportunity to invoke other Convex functions as needed.

Similar to the useMutation hook on the frontend, runMutation accepts a Convex function belonging to the api Convex generates from your codebase.

Exercise 3.5

Send the AI response as a new message

Import the generated Convex api into your ai.ts module:

import { api } from "./_generated/api";

In the action's handler, use ctx.runMutation to execute the existing api.messages.send mutation to add a new message to the chat, passing through the chat completion response you received from Together (or a fallback string in case the response didn't have any content, for whatever reason):

Solution

Your convex/ai.ts module should now look something like this:

convex/ai.ts

TS

import { action } from "./_generated/server";
import { api } from "./_generated/api";
import { v } from "convex/values";

const TOGETHER_API_KEY = process.env.TOGETHER_API_KEY!;

export const chat = action({
  args: {
    messageBody: v.string(),
  },
  handler: async (ctx, args) => {
    const res = await fetch("https://api.together.xyz/v1/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${TOGETHER_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "meta-llama/Llama-3-8b-chat-hf",
        messages: [
          {
            // Provide a 'system' message to add context about how to respond
            // (feel free to change this to give your AI agent personality!)
            role: "system",
            content:
              "You are a terse bot in a group chat responding to questions with 1-sentence answers.",
          },
          {
            // Pass on the chat user's message to the AI agent
            role: "user",
            content: args.messageBody,
          },
        ],
      }),
    });

    const json = await res.json();
    // Pull the message content out of the response
    const messageContent = json.choices[0].message?.content;

    // Send AI's response as a new message
    await ctx.runMutation(api.messages.send, {
      author: "AI Agent",
      body: messageContent || "Sorry, I don't have an answer for that.",
    });
  },
});

Put your action into action

To make sure the chat action works as intended, you can test-run it in the Dashboard's "Functions" tab, or with the CLI using the convex run command (substituting in your own question, of course!):

npx convex run ai:chat '{"messageBody":"What is a serverless function?"}'

If all went well, you'll see a new document in the messages table, and in the chat itself!

From mutation to action

At this point your action still isn't connected to the UI, so there is no way to trigger it from the chat. Time to fix that!

Right on schedule

As mentioned earlier, query and mutation functions always run deterministically thanks to the Convex Runtime, whereas actions can be nondeterministic and interact with the outside world. If a deterministic mutation called a nondeterministic action directly, that determinism would be lost!

However, the Convex scheduler provides a safe way for mutations to indirectly invoke other functions (whether queries, mutations, or actions). Using the scheduler, a mutation can "queue up" an action to run after the mutation has successfully executed, which allows Convex to make sure that the mutation did not encounter errors before trying to run the action.

Exercise 3.6

Schedule the Chat action after the send mutation

In convex/messages.ts, edit the send mutation to schedule the ai:chat action after sending the new message.

To do this, you'll need to use the scheduler object from the ctx MutationContext that the handler receives as its first argument.

Within the handler body, add a call to ctx.scheduler.runAfter to run the api.ai.chat action after the mutation has completed. The asynchronous runAfter method takes 3 arguments:

• duration in milliseconds the scheduler should wait to run the scheduled function (0 milliseconds makes sense in this case)
• the function to schedule
• an arguments object to pass through to the scheduled function (in this case, the message body)

You probably don't want the AI agent to respond to every message in the chat, so wrap the scheduled action in a conditional so that it will only respond to messages starting with @ai and check the author so it won't respond to itself.

Solution

convex/messages.ts

TS

import { api } from "./_generated/api";

// ...

export const send = mutation({
  args: { body: v.string(), author: v.string() },
  handler: async (ctx, args) => {
    const { body, author } = args;
    // Send a new message.
    await ctx.db.insert("messages", { body, author });

    if (body.startsWith("@ai") && author !== "AI") {
      // Schedule the chat action to run immediately
      await ctx.scheduler.runAfter(0, api.ai.chat, {
        messageBody: body,
      });
    }
  },
});

Recap

• Convex actions let you access arbitrary external resources, such as third-party libraries and APIs
• Environment Variables can be added to your Convex project to securely save secrets (such as API keys) which can be read by your functions
• Deterministic mutation functions can't call actions directly, but they can indirectly invoke them using the scheduler

Go forth and Convex!

You've now completed all 3 parts of the Convex Tour, and built an AI-enabled chat app in the process - amazing work!

But we hope this is just the beginning of your Convex journey, so on the next page we've collected some resources you might want to explore next. Choose your own adventure!