Skip to main content

Bundling

Bundling is the process of gathering, optimizing and transpiling the JS/TS source code of functions and their dependencies. During development and when deploying, the code is transformed to a format that Convex runtimes can directly and efficiently execute.

Convex currently bundles all dependencies automatically, but for the Node.js runtime you can disable bundling certain packages via the external packages config.

Bundling for Convex​

When you push code either via npx convex dev or npx convex deploy, the Convex CLI uses esbuild to traverse your convex/ folder and bundle your functions and all of their used dependencies into a source code bundle. This bundle is then sent to the server.

Thanks to bundling you can write your code using both modern ECMAScript Modules (ESM) or the older CommonJS (CJS) syntax.

ESM vs. CJS

ESM

  • Is the standard for browser Javascript
  • Uses static imports via the import and export keywords (not functions) at the global scope
  • Also supports dynamic imports via the asynchronous import function

CJS

  • Was previously the standard module system for Node.js
  • Relies on dynamic imports via the require and asynchronous import functions for fetching external modules
  • Uses the module.exports object for exports

Bundling limitations​

The nature of bundling comes with a few limitations.

Code size limits​

The total size of your bundled function code in your convex/ folder is limited to 32MiB (~33.55MB). Other platform limits can be found here.

While this limit in itself is quite high for just source code, certain dependencies can quickly make your bundle size cross over this limit, particularly if they are not effectively tree-shakeable (such as aws-sdk or snowflake-sdk)

Dynamic dependencies​

Some libraries rely on dynamic imports (via import/require calls) to avoid always including their dependencies. These imports are not supported by the default Convex runtime and will throw an error at runtime.

Additionally, some libraries rely on local files, which cannot be bundled by esbuild. If bundling is used, irrespective of the choice of runtime, these imports will always fail in Convex.

Examples of libraries with dynamic dependencies

Consider the following examples of packages relying on dynamic dependencies:

  • langchain relying on the presence of peer dependencies that it can dynamically import. These dependencies are not statically imported so will not be bundled by esbuild.
  • sharp relying on the presence of libvips binaries for image-processing operations
  • pdf-parse relies on being dynamically imported with require() in order to detect if it is being run in test mode. Bundling can eliminate these require() calls, making pdf-parse assume it is running in test mode.
  • tiktoken relying on local WASM files

External packages​

As a workaround for the bundling limitations above, Convex provides an escape hatch within your project's convex.json file: external packages. This feature is currently exclusive to Convex's Node.js runtime.

External packages use esbuild's facility for marking a dependency as external. This tells esbuild to not bundle the external dependency at all and to leave the import as a dynamic runtime import using require() or import(). Thus, your Convex modules will rely on the underlying system having that dependency made available at execution-time.

External Packages are in beta

External Packages are currently a beta feature. If you have feedback or feature requests, let us know on Discord!

Package installation on the server​

Packages marked as external are installed from npm the first time you push code that uses them. The version installed matches the version installed in the node_modules folder on your local machine.

While this comes with a latency penalty the first time you push external packages, your packages are cached and this install step only ever needs to rerun if your external packages change. Once cached, pushes can actually be faster due to smaller source code bundles being sent to the server during pushes!

Specifying external packages​

You can set the node.externalPackages = ["*"] field of your convex.json to mark all used dependencies within your Node actions as external:

{
  "node": {
    "externalPackages": ["*"]
  }
}

Alternatively, you can explicitly specify which packages to install on the server:

{
  "node": {
    "externalPackages": ["aws-sdk", "sharp"]
  }
}

The package identifiers should match the string used in import/require in your Node.js action.

Troubleshooting External Package versions

In order for an exact version for an external package to be found, the Convex CLI searches for the installed package version within your local node_modules directory.

Thus, if you change an external dependency version in your project, the change will only will only be pushed to the server after you've updated the package version installed in your local node_modules folder.

Limitations​

The total size of your source code bundle and external packages cannot exceed the following:

  • 45MB zipped
  • 240MB unzipped

Packages that are known not to work at this time:

If there is a package that you would like working in your Convex functions, let us know.