Skip to main content

Importing Data Snapshots

You can import data into Convex from a local file using the command line.

npx convex import
Snapshot import is in beta

Snapshot import is currently a beta feature. If you have feedback or feature requests, let us know on Discord!

Use --help to see all options. The most common flows are described here.

Single table import​

npx convex import --table <tableName> <path>

Import a CSV, JSON, or JSONLines file into a Convex table.

  • .csv files must have a header, and each row's entries are interpreted either as a (floating point) number or a string.
  • .jsonl files must have a JSON object per line.
  • .json files must be an array of JSON objects.
    • JSON arrays have a size limit of 8MiB. To import more data, use CSV or JSONLines. You can convert json to jsonl with a command like jq -c '.[]' data.json > data.jsonl

Imports into a table with existing data will fail by default, but you can specify --append to append the imported rows to the table or --replace to replace existing data in the table with your import.

The default is to import into your dev deployment. Use --prod to import to your production deployment or --preview-name to import into a preview deployment.

Import data from a snapshot ZIP file​

npx convex import <path>.zip

Import a ZIP file into a Convex deployment, where the ZIP file has been downloaded from Snapshot Export in the dashboard. Documents will retain their _id and _creationTime fields so references between tables are maintained.

Imports where tables have existing data will fail by default, but you can specify --replace to replace existing data in tables mentioned in the ZIP file.

Use cases​

  1. Restore a deployment from backup. Download a snapshot export when tables are in a good state, and restore from this backup if needed.
npx convex import --prod --replace backup_snapshot.zip
  1. Seed dev deployments with sample data, exported from prod or another dev deployment.
npx convex import seed_snapshot.zip
  1. Seed preview deployments with sample data, exported from prod, dev, or another preview deployment. Example for Vercel, seeding data from seed_snapshot.zip committed in the root of the repo.
npx convex deploy --cmd 'npm run build' &&
if [ "$VERCEL_ENV" == "preview" ]; then
npx convex import --preview-name "$VERCEL_GIT_COMMIT_REF" seed_snapshot.zip;
fi
  1. Clear a table efficiently with an empty snapshot import.
touch empty_file.jsonl
npx convex import --replace --table <tableNameToClear> empty_file.jsonl

Features​

  • Snapshot import is the only way to create documents with pre-existing _id and _creationTime fields.
    • The _id field must match Convex's ID format.
    • If _id or _creationTime are not provided, new values are chosen during import.
  • Snapshot import creates and replaces tables atomically (except when using --append).
    • Queries and mutations will not view intermediate states where partial data is imported.
    • Indexes and schemas will work on the new data without needing time for re-backfilling or re-validating.
  • Snapshot import only affects tables that are mentioned in the import, either by --table or as entries in the ZIP file.
  • While JSON and JSONLines can import arbitrary JSON values, ZIP imports can additionally import other Convex values: Int64, Bytes, etc. Types are preserved in the ZIP file through the generated_schema.jsonl file.
  • Snapshot import of ZIP files that include file storage import the files and preserve _storage documents, including their _id, _creationTime, and contentType fields.

Warnings​

  • Streaming Export (Fivetran or Airbyte) does not understand snapshot imports, similar to table deletion and creation and some schema changes. We recommend resetting streaming export sync after a snapshot import.
  • Avoid changing the ZIP file between downloading it from Snapshot Export and importing it with npx convex import. Some manual changes of the ZIP file may be possible, but remain undocumented. Please share your use case and check with the Convex team in Discord.
  • Snapshot import is not always supported when importing into a deployment that was created before Convex version 1.7.
    • The import may work, especially when importing a ZIP snapshot from a deployment created around the same time as the target deployment. As a special case, snapshot import can always restore a deployment from its own backup.
    • Reach out in Discord if you encounter issues, as there may be a workaround.

Snapshot import uses database bandwidth to write all documents, and file bandwidth if the export includes file storage. You can observe this bandwidth in the usage dashboard as function name _cli/import and associated cost in the limits docs.