Docs

Upgrades, backups, retention

How to upgrade a self-hosted engine without losing data, what to back up and how to restore it, and how to choose how long each kind of data is kept.

Applies to Engine At launch

View as Markdown
On this page
  1. Upgrades
  2. What an upgrade may change
  3. Upgrade
  4. Read what changed
  5. Back up the database
  6. Get the new image and restart the engine
  7. Check
  8. Go back
  9. Backups
  10. What needs one
  11. Back up Postgres
  12. Restore
  13. Stop the engine
  14. Restore the dump
  15. Start the engine and check
  16. Process again what arrived after the backup
  17. If the bucket is lost
  18. Retention
  19. Change it
  20. How to choose
  21. What retention does not cover
  22. Related

Engine This page is for whoever keeps a self-hosted engine running after the first day. On Sypnia Cloud all three jobs are ours: we upgrade, we back up, and your plan sets the retention.

Upgrades

What an upgrade may change

The engine is built so that an upgrade inside version 1 is boring. These rules are part of its contract:

RuleWhat it means for you
Database migrations only add.An older engine keeps working against a newer schema. You can migrate first, replace the engine after, and go back without touching the database.
The API inside /v1 only adds.Scripts, the CLI, the MCP server and your coding agent keep working.
Raw events keep fields the engine does not know.A newer SDK never breaks an older engine. Upgrade the engine and your apps in any order.
An unknown SYPNIA_… variable is a warning in sypnia doctor, not an error.A setting that was renamed or removed does not stop an upgrade. The changelog names it.
Processing is safe to repeat.A worker killed halfway through an upgrade loses nothing and counts nothing twice.

A change that breaks one of these would be version 2, announced in the changelog, never a surprise inside version 1.

Which version you run: sypnia version, or the Sypnia-Version header on any API answer. The image tag ghcr.io/sypnia/sypnia:1 follows the latest release of version 1. To decide yourself when each upgrade happens, use an exact tag such as ghcr.io/sypnia/sypnia:1.0.0 and change it by hand.

Upgrade

At the end you will have: the new version running, the same data, and a backup taken just before.

AWS

Follow Update the engine on the AWS page: read the changelog, update the command and the template, run sypnia migrate, deploy. Take a snapshot of the database first:

Terminal
aws rds create-db-snapshot \
  --db-instance-identifier <DatabaseIdentifier> \
  --db-snapshot-identifier sypnia-before-upgrade-$(date +%F)

Ingest does not stop: the old functions answer until the new ones are ready.

Go back

Start the previous version again: the previous image tag, or the previous template. Leave the database as it is. The older engine works against the newer schema, and what arrived in between is kept.

Do not restore a database backup to undo an upgrade. A restore is for a lost or damaged database, and it discards everything people did since the backup.

Backups

What needs one

DataWhereCan it be rebuilt?Back it up?
Projects, ingest keys, API tokens, project settings, alert rules, webhooks, uptime checksPostgresNoYes
What people did: problems resolved or ignored, notes, alerts acknowledgedPostgresNoYes
The agent’s analysesPostgresOnly by running the agent again, which costs model callsYes
Problems’ counts, the session index, performance numbers, the trace indexPostgresYes, from events/ in the bucket, for as long as the events are keptComes with the same backup
Raw events, per-session files, replays, page snapshotsBucketNo. They are the sourceYour choice, see below
Source mapsBucketYes: your CI uploads them again on the next buildNo
The engine’s settings and the root tokenYour .env file, or the stack’s parametersNoYes, in a password manager or a secret store

So the rule is short: back up Postgres every day, and keep the settings somewhere safe. A database backup is small, because raw events are never in it.

The bucket is a different decision. A hosted store such as Amazon S3 or Cloudflare R2 already keeps several copies of every object; most people stop there. A MinIO on one disk, as in the Compose file, is one disk: if it dies, the raw events and the replays are gone, and Sypnia keeps only what Postgres holds. If that matters to you, copy the bucket somewhere else on a schedule, or use a hosted store.

Back up Postgres

AWS

The database the template creates is backed up by AWS every day and each backup is kept 7 days. You can also go back to any minute of those 7 days. Nothing to set up. To check:

Terminal
aws rds describe-db-instances --db-instance-identifier <DatabaseIdentifier> \
  --query "DBInstances[0].[BackupRetentionPeriod,LatestRestorableTime]"
You should see
7 and a time a few minutes ago.

If you brought your own Postgres, its backups are your provider’s or yours.

Restore

At the end you will have: the database as it was at the backup, plus everything that can be rebuilt from the events that arrived since.

  1. Stop the engine

    Terminal
    docker compose stop sypnia

    SDKs keep retrying for a short while; what they cannot hold is lost. A restore is not a routine operation.

  2. Restore the dump

    Terminal
    docker compose exec -T db pg_restore -U sypnia -d sypnia --clean --if-exists < backups/sypnia-2026-03-12.dump
    You should see
    No errors. On AWS, restore the snapshot or the point in time to a new instance instead, and give its address to the stack as DatabaseUrl.
  3. Start the engine and check

    Terminal
    docker compose start sypnia
    docker compose exec sypnia sypnia doctor --skip ai
    You should see
    all checks passed. If the backup is older than the engine, the engine migrates it as it starts.
  4. Process again what arrived after the backup

    The bucket still holds every event. Tell the worker to read them again from the time of the backup:

    Terminal
    docker compose exec sypnia sypnia aggregate --from 2026-03-12T03:00:00Z
    You should see
    Text
    shop  events 48211 -> 361 sessions, 4 problems (1 new), 22 routes   9.4 s
    done in 11.0 s, data through 2026-03-12T16:20:05Z

    It lists the bucket for that range, so it also finds events the restored database has never heard of. Running it twice is safe.

What a restore cannot bring back is what people wrote after the backup:

  • A problem resolved or ignored since then is open again.
  • Ingest keys and API tokens created since then no longer exist. Apps that use them are refused: create them again and deploy.
  • A problem first seen since then is new again, so it alerts again, and your webhook receives it a second time.
  • Analyses written since then are run again when their problem grows, or when you ask.

Try a restore once, on a copy, before the day you need it.

If the bucket is lost

Postgres still has every problem, its sentence and its counts, and all the numbers. Sessions stay in the list and answer 410 expired when opened, because their files are gone. Point the engine at a new, empty bucket, run sypnia doctor, and new data flows. Source maps come back with your next build.

Retention

The engine deletes old data itself, once a day. You choose the number of days for each kind; the bucket needs no rule for it.

DataDefaultSetting
Raw events, sessions with their timelines, traces, the daily Parquet files90 daysSYPNIA_RETENTION_EVENTS_DAYS (1 to 3650)
Replay recordings30 daysSYPNIA_RETENTION_REPLAYS_DAYS (not more than the events)
Page snapshots90 daysSYPNIA_RETENTION_SNAPSHOTS_DAYS
Hourly and daily numbers: performance, click maps, usage, uptime400 daysSYPNIA_RETENTION_NUMBERS_DAYS
Source maps90 days after the release was last seenSYPNIA_RETENTION_SOURCEMAPS_DAYS
Problems: the sentence, the analysis, the status, the daily counts and one exampleWhile the project existsNone
Projects, keys, tokens, settingsUntil you delete themNone

How it works:

  • A session’s age is counted from its last event. Data is deleted within 24 hours of passing its age.
  • The settings are for the whole engine. A self-hosted engine has no retention per project; GET /v1/usage returns the retention that applies.
  • A problem outlives the sessions it hit. That is what lets Sypnia say «came back» months later. Its linked sessions, replays and traces expire on their own schedule.
  • After that, asking for expired data answers 410 expired, not an empty result.

Change it

Set the variable and restart the engine. On AWS, add the line to the template as in Change a setting.

Terminal
SYPNIA_RETENTION_EVENTS_DAYS=30
SYPNIA_RETENTION_REPLAYS_DAYS=14
  • Shorter: everything older than the new number is deleted at the next daily cleanup. It cannot be undone, except from a backup.
  • Longer: data is kept longer from now on. What was already deleted does not come back.

To run the cleanup now instead of waiting for the daily one:

Terminal
sypnia aggregate --task retention

How to choose

  • Replays cost the most space and age the fastest: a recording is useful while the problem is fresh. 14 to 30 days is enough for most teams. See Storage layout.
  • Events are what lets you rebuild the database and open an old session’s timeline. Keep them at least as long as your oldest database backup.
  • Numbers are small and hold nothing about people. Keeping them a year lets you compare this month with the same month last year.
  • Personal data. Replays, sessions and the names your app sent with identify() are personal data. Keep them only as long as you can explain. To delete one person or one session sooner, see Deleting data.

What retention does not cover

  • Your backups. A database dump keeps what was deleted afterwards until the dump itself is deleted. Do not keep backups longer than the data they contain is meant to live.
  • Logs. The engine writes logs to standard output and never logs bodies, keys or tokens. How long they are kept is up to whatever collects them: Docker, CloudWatch (30 days in the AWS template), your own collector. The logs of the proxy in front of the engine hold IP addresses.
  • Bucket versions. If you turned on versioning, old versions outlive the engine’s deletes until a rule of yours removes them.

Sections

Start hereWhat Sypnia is, how it works, what is open source and what is the paid cloud, and the two ways to get your first problem in plain words.ConceptsThe ten ideas Sypnia is made of, each defined precisely, from a project and its keys to problems, traces and alerts.SDKsHow your app sends data to Sypnia. One SDK for the browser, one for Node, one for Next.js, and the OpenTelemetry intake for every other backend.The agentHow Sypnia analyses a problem, what it writes, what it sends to a model and what it never does.MCP and skillsConnect your coding agent to Sypnia so it can ask what is failing in production, read what it needs to fix it, and set Sypnia up for you.API referenceThe Sypnia HTTP API: one API for the open-source engine and for Sypnia Cloud, with every endpoint, parameter, response and error.CLI referenceThe sypnia command: how to install it, the variables it reads, what its exit codes mean, and where each command is documented.Self-hostingWhat you run when you host the Sypnia engine yourself, what you need, what it costs, and what you do not get compared with Sypnia Cloud.How the engine is builtThe whole Sypnia engine on one page, from a click in your app to the sentence your coding agent reads, with the design rules and the code that holds them.Sypnia CloudHow the paid service works, from organizations and roles to plans, limits, billing, where your data lives, the GitHub App and support.Privacy and securityWhat Sypnia collects from your app and its users, what it never collects, how masking works, how to delete data, and how the service is protected.LicensingWhich licence covers each part of Sypnia, what you may do with it, how the name may be used, and the terms of a contribution.GuidesStep-by-step recipes for the common ways to use Sypnia, from apps made with AI builders and Next.js on Vercel to Supabase, source maps in CI, alert e-mails from your own engine, and moving from Sentry.ReferenceThe pages you look things up in, from the event schema and the glossary to what is planned, the changelog and the common questions.
↑↓ to move↵ to openEsc to close