Bundling is the process of gathering, optimizing and transpiling the JS/TS source code of functions and their dependencies. During development and when deploying, the code is transformed to a format that Convex runtimes can directly and efficiently execute.
Convex currently bundles all dependencies automatically, but for the Node.js runtime you can disable bundling certain packages via the external packages config.
Bundling for Convex
When you push code either via
npx convex dev or
npx convex deploy, the
Convex CLI uses esbuild to traverse your
folder and bundle your functions and all of their used dependencies into a
source code bundle. This bundle is then sent to the server.
Thanks to bundling you can write your code using both modern ECMAScript Modules (ESM) or the older CommonJS (CJS) syntax.
ESM vs. CJS
- Uses static imports via the
exportkeywords (not functions) at the global scope
- Also supports dynamic imports via the asynchronous
- Was previously the standard module system for Node.js
- Relies on dynamic imports via the
importfunctions for fetching external modules
- Uses the
module.exportsobject for exports
The nature of bundling comes with a few limitations.
Code size limits
The total size of your bundled function code in your
convex/ folder is
limited to 32MiB (~33.55MB).
While this limit in itself is quite high for just source code, certain dependencies can quickly make your bundle size cross over this limit, particularly if they are not effectively tree-shakeable (such as aws-sdk or snowflake-sdk)
Some libraries rely on dynamic imports (via
require calls) to avoid
always including their dependencies. These imports are not supported by the
default Convex runtime
and will throw an error at runtime.
Additionally, some libraries rely on local files, which cannot be bundled by esbuild. If bundling is used, irrespective of the choice of runtime, these imports will always fail in Convex.
Examples of libraries with dynamic dependencies
Consider the following examples of packages relying on dynamic dependencies:
- langchain relying on the presence
of peer dependencies that it can dynamically import. These dependencies are
imported so will not be bundled by
- sharp relying on the presence of
libvipsbinaries for image-processing operations
- pdf-parse relies on being
dynamically imported with
require()in order to detect if it is being run in test mode. Bundling can eliminate these
pdf-parseassume it is running in test mode.
- tiktoken relying on local WASM files
As a workaround for the bundling limitations above, Convex provides an escape
hatch within your project's
convex.json file: external packages. This
feature is currently exclusive to Convex's
External packages use
esbuild's facility for marking a dependency as external.
esbuild to not bundle the external dependency at all and to leave
the import as a dynamic runtime import using
your Convex modules will rely on the underlying system having that dependency
made available at execution-time.
Package installation on the server
Packages marked as external are installed from npm the
first time you push code that uses them. The version installed matches the
version installed in the
node_modules folder on your local machine.
While this comes with a latency penalty the first time you push external packages, your packages are cached and this install step only ever needs to rerun if your external packages change. Once cached, pushes can actually be faster due to smaller source code bundles being sent to the server during pushes!
Specifying external packages
You can set the
node.externalPackages = ["*"] field of your
mark all used dependencies within your Node actions as external:
Alternatively, you can explicitly specify which packages to install on the server:
"externalPackages": ["aws-sdk", "sharp"]
The package identifiers should match the string used in
Troubleshooting External Package versions
In order for an exact version for an external package to be found, the Convex CLI searches for the
installed package version within your local
Thus, if you change an external dependency version in your project, the change will only will only be
pushed to the server after you've updated the package version installed in your local
The total size of your source code bundle and external packages cannot exceed the following:
- 45MB zipped
- 240MB unzipped
Packages that are known not to work at this time:
- Puppeteer - browser binary installation exceeds the size limit
- @ffmpeg.wasm - since 0.12.0, no longer supports Node environments
If there is a package that you would like working in your Convex functions, let us know.