Docs

Scaling and cost

What a self-hosted engine costs each month for a small and a medium project, with the arithmetic, what makes the bill move, the first limits you meet, and when to go from one process to several or to functions.

Applies to Engine At launch

View as Markdown
On this page
  1. What makes the bill move
  2. Two example projects
  3. On AWS
  4. On one server with Docker
  5. Which shape costs less
  6. The first limits you meet
  7. Database connections
  8. From one process to several
  9. Stop migrating at start
  10. Start one process per role
  11. Send each path to its process
  12. From processes to functions
  13. What is not known yet
  14. Related

Engine This page is for whoever pays for a self-hosted engine, or must say what it will cost before it exists. On Sypnia Cloud you pay a plan instead, and capacity is our job: see Plans and limits.

What makes the bill move

In the order you will feel them:

DriverGrows withWhat you can change
The databaseAlmost nothing: it is a fixed monthly price, and at small sizes the largest line.Bring a Postgres you already pay for. Any Postgres 15 or later works, with a database of its own.
Writes to the bucketRecorded sessions (one write per 10 seconds of recording), sessions (their files are written again as they grow), and deploys (one write per source map). Not events: those are grouped into segments.SYPNIA_REPLAY_SAMPLE_RATE for new projects, or each project’s own settings.replay.sample_rate. Upload source maps only for environments someone looks at.
Storage in the bucketEvents × days kept, and recordings × days kept. Recordings are about three quarters of it.SYPNIA_RETENTION_REPLAYS_DAYS, SYPNIA_RETENTION_EVENTS_DAYS.
ComputeOn AWS, requests and the worker’s seconds. On a server, nothing until the server is full.The shape you deploy.
The modelProblems that are new, grow or come back. Not traffic.SYPNIA_AI_MAX_ANALYSES_PER_DAY, SYPNIA_AI_WEEKLY_SUGGESTIONS, and the model you choose.
LogsRequests, if you log each one.Keep SYPNIA_LOG_LEVEL at info.
Data leaving your providerWatching replays: each one is downloaded by the browser that plays it.Nothing to set. It is the size of the replays you watch.

Two of these come from experience, measured on the system this engine comes from:

  • Writes cost more than storage. Of USD 3.81 of object storage in 12 days, USD 2.73 was writes: 552 thousand of them, most from source maps uploaded on every deploy of a development environment. The same account stored 93 GB for about USD 2 a month.
  • One deploy of a large app is 965 source maps and 109 MB. Twenty deploys a month are 19,300 writes and 2 GB.

To cap what a project can cost you whatever it sends, set SYPNIA_QUOTA_EVENTS_PER_MONTH and SYPNIA_QUOTA_REPLAYS_PER_MONTH. At the limit the SDKs are told to send only failures, an alert is created at 80 % and at 100 %, and nothing is dropped silently. See Sampling and quotas.

Two example projects

A monthSmallMedium
Sessions30,000 (1,000 a day)300,000
Events4 million (130,000 a day)40 million
Batches sent to ingest (30 a session)0.9 million9 million
Replay chunks (15 % of sessions recorded × 22 chunks)0.1 million1 million
In the bucket once retention has filledAbout 8 GBAbout 80 GB
In PostgresAbout 1.5 GBAbout 2.5 GB

Small is the size measured on the system this engine comes from: 110 to 150 thousand records and about a thousand sessions a day. Medium is ten times that. The sizes come from Storage layout.

On AWS

The line-by-line arithmetic, with the prices used, is in Deployment shapes. Summed by kind, in USD a month:

SmallMediumWhat it is
Functions3.3115.26Requests to ingest, and the seconds ingest and worker run
Queue0.604.00One send per batch, and the worker’s receives
Bucket: writes, reads, storage3.5214.34Writes are 2.10 and 10.50 of it
Logs0.502.00Successes sampled at 1 %
The address0.000.00Inside the provider’s free allowance of 10 million requests a month
The engine, without Postgres≈ 8≈ 36
Postgres≈ 1425 to 50The smallest managed instance with 1 GB of memory and 20 GB of disk; 2 to 4 GB of memory at medium
Total≈ 22≈ 60 to 85

Read three things from it:

  1. At small sizes you are paying for a database. If you already have a Postgres with room, the engine is about USD 8 a month.
  2. Receiving is nearly free. A million batches cost less than 40 cents to accept.
  3. An idle engine costs almost nothing but the database. That makes this shape a good fit for a staging engine, or for traffic that comes in bursts.

AWS’s free allowance for functions, 400,000 GB-seconds and a million requests a month, covers almost all the function lines of the small project.

Not in these totals:

ExtraAboutWhen
The modelYour provider’s price for a few thousand tokens in and at most two thousand out, per analysis. At most 100 analyses a day per project by default.When you connect one. Measured on the system this comes from: a weekly run of suggestions cost USD 0.08 to 0.20.
A public address for the databaseUSD 3.65 a month (0.005 an hour × 730)When the template creates the database.
Data leaving AWSUSD 0.09 per GB after the first 100 GB a monthWhen you watch replays or read a lot through the API.
A NAT gatewayUSD 33 a month, plus trafficOnly if you move the functions into a private network yourself. The template does not.
A web firewall on the addressFrom USD 6 a monthOnly if you add one to protect the Console.

For reference, the design of the system this engine comes from estimated its own bill at USD 15 to 25 a month at one million events a day, with a database it shared with its app, so the database is not in that number.

On one server with Docker

A server has no price per request. You pay for its size, and the question is what size.

What needs memoryBudget
The engine: two projects being processed (2 × 256 MB), the ingest buffer (32 MiB), the source-map cache (200 MB)1 GB is comfortable, 2 GB leaves room
Postgres1 GB for the small project, 2 to 4 GB for the medium one
MinIO1 GB
The system and Caddy0.5 GB

One CPU core is enough for the engine at the small size: its work is waiting on the bucket and the database, not computing.

SmallMedium
Server2 vCPU, 4 GB of memory4 vCPU, 8 GB of memory
Disk40 GB: 8 GB of bucket, 1.5 GB of database, source maps, room to grow160 GB: 80 GB of bucket, 2.5 GB of database, room to grow
Example price, on demand in us-east-1An arm64 instance of that size at USD 0.0336 an hour × 730 = 24.50, plus 40 GB of disk × 0.08 = 3.20USD 0.0672 an hour × 730 = 49.10, plus 160 GB × 0.08 = 12.80
Total≈ USD 28≈ USD 62

Providers that sell fixed-price servers are often cheaper for the same size; a commitment of a year lowers the on-demand price by about a third. With a hosted bucket instead of MinIO, take the bucket line of the AWS table (USD 3.52 or 14.34) and a smaller disk.

One server is also one point of failure. If it stops, SDKs retry for a short time and then drop what they hold, and nothing alerts you, because the engine that would is the one that stopped. Watch /healthz from somewhere else.

Which shape costs less

Per million events, the AWS engine lines come to about USD 2 at the small size (8 ÷ 4) and USD 0.90 at the medium one (36 ÷ 40), and stay near that as traffic grows. A server’s cost per event falls until the server is full, then jumps to the next size.

  • Small or bursty traffic: functions. You pay for what arrives.
  • Large and steady traffic: servers, with a hosted bucket.
  • In between: the two totals above are within a few dollars of each other. Choose by what you would rather operate, not by price.

We have no measured crossover. It will be published here once the engine has run at both sizes.

The first limits you meet

Each limit has a sign you can see without guessing.

SignWhat it meansWhat to do
Ingest answers 503 with Retry-After: 2More than 32 MiB of events are waiting to be written: the bucket is slow, or the process has no CPU left.A faster disk under MinIO, or a hosted bucket. Then run ingest apart.
data_through in GET /v1/summary is more than 10 minutes old, or an «Engine failing» alertThe worker cannot keep up. Problems still appear on time for a while; numbers fall behind first.Raise SYPNIA_PROCESSING_CONCURRENCY (each step up needs 256 MB more), or run a second worker.
The API answers 503 with Retry-AfterThe database has no free connection.See Database connections.
A project’s hour is marked failed and named in an «Engine failing» alertOne project sends more in an hour than a pass may hold in memory.Give the worker more memory, or set a quota for that project.
The disk fillsRecordings, almost always.SYPNIA_RETENTION_REPLAYS_DAYS, then sypnia aggregate --task retention.
Everything is slow at the same moments each dayThe daily job (Parquet, retention, partitions) at 02:30 UTC shares the machine.More memory, or a separate worker.

Budgets the design must meet, not measurements: a busy hour of 60,000 events and 1,000 sessions for one project is processed in about 40 MB of memory, and ten times that, 600,000 events and 10,000 sessions, in about 110 MB. The arithmetic is in Aggregation. On AWS, 40 copies of the ingest function at 50 ms a request accept about 800 batches a second; the medium project sends 3.5 a second.

Database connections

This is the limit people meet first with functions, and the system this engine comes from met it too: 13 refused connections in two minutes, when every copy of a function opened its own pool.

The total is SYPNIA_DATABASE_MAX_CONNECTIONS times the number of processes, or copies of a function, that run at once.

ShapeArithmeticTotal
One process10, the default10
Three processes by role3 × 1030
The AWS templateingest: 40 copies × 1. api: 20 copies × 2. worker: 2 copies × 3At most 86

A managed Postgres with 1 GB of memory accepts about 100 connections, so the template’s defaults fit the smallest instance with little to spare. When the API starts answering 503:

  • give the database more memory, which raises its limit, or
  • put a connection pooler in front of it, or
  • lower the number of copies in the template, if you would rather have a slower API than a larger database.

When the database refuses a connection, the engine answers 503 with Retry-After at once. It does not queue and it does not fall over.

From one process to several

At the end you will have: ingest, the API and the worker as separate processes against the same database and bucket, so each can be sized and restarted alone.

  1. Stop migrating at start

    Several processes must not migrate at the same moment. Set this for all of them, and run sypnia migrate once in each deploy, before they start.

    Terminal
    SYPNIA_AUTO_MIGRATE=false
  2. Start one process per role

    The same image and the same settings, with a different role each:

    Terminal
    sypnia serve --role ingest --listen 0.0.0.0:8081
    sypnia serve --role api    --listen 0.0.0.0:8082
    sypnia serve --role worker
    You should see
    Each process prints its role on its first line, such as sypnia 1.0.0 role=ingest listen=0.0.0.0:8081, and ends with ready.
  3. Send each path to its process

    In your proxy, under the one base URL:

    PathGoes to
    /v1/ingest/*, /v1/otlp/*, /v1/sdk.jsThe ingest processes
    Everything else: /v1/*, /mcp, /consoleThe api process
    You should see
    curl -i <your base URL>/healthz answers 200, and sypnia doctor passes when run next to any of the processes.

What changes when you split:

  • Run as many ingest processes as you need. They share nothing but the bucket.
  • Two workers are safe. They never count anything twice; the second one mostly shortens the wait after a restart.
  • Rate limits are counted per process, so the real limit is the setting times the number of processes.
  • Timing. With ingest and the worker apart, the worker finds a new segment on its next pass, at most 30 seconds later, not at once. A new problem still appears within 2 minutes, and numbers within 10.

From processes to functions

Move to the AWS template when traffic comes in bursts that a server sized for the average cannot take, when you would rather not patch and watch servers, or when most of the day the engine has nothing to do.

Both shapes read the same bucket layout and the same tables, so moving is pointing the other shape at the same data: deploy the template with DatabaseUrl set to your database, copy the bucket to the one the stack created, then change the DNS record. The way back is the same.

What is not known yet

  • How long the functions really run. The function lines above assume 50 ms for an ingest request and 0.5 to 2 seconds for a worker wake-up.
  • How large the Parquet files are. They are budgeted at the size of the raw events.
  • How much one process takes before it must be split. The signs above tell you when; the number will be published once measured.

Sections

Start hereWhat Sypnia is, how it works, what is open source and what is the paid cloud, and the two ways to get your first problem in plain words.ConceptsThe ten ideas Sypnia is made of, each defined precisely, from a project and its keys to problems, traces and alerts.SDKsHow your app sends data to Sypnia. One SDK for the browser, one for Node, one for Next.js, and the OpenTelemetry intake for every other backend.The agentHow Sypnia analyses a problem, what it writes, what it sends to a model and what it never does.MCP and skillsConnect your coding agent to Sypnia so it can ask what is failing in production, read what it needs to fix it, and set Sypnia up for you.API referenceThe Sypnia HTTP API: one API for the open-source engine and for Sypnia Cloud, with every endpoint, parameter, response and error.CLI referenceThe sypnia command: how to install it, the variables it reads, what its exit codes mean, and where each command is documented.Self-hostingWhat you run when you host the Sypnia engine yourself, what you need, what it costs, and what you do not get compared with Sypnia Cloud.How the engine is builtThe whole Sypnia engine on one page, from a click in your app to the sentence your coding agent reads, with the design rules and the code that holds them.Sypnia CloudHow the paid service works, from organizations and roles to plans, limits, billing, where your data lives, the GitHub App and support.Privacy and securityWhat Sypnia collects from your app and its users, what it never collects, how masking works, how to delete data, and how the service is protected.LicensingWhich licence covers each part of Sypnia, what you may do with it, how the name may be used, and the terms of a contribution.GuidesStep-by-step recipes for the common ways to use Sypnia, from apps made with AI builders and Next.js on Vercel to Supabase, source maps in CI, alert e-mails from your own engine, and moving from Sentry.ReferenceThe pages you look things up in, from the event schema and the glossary to what is planned, the changelog and the common questions.
↑↓ to move↵ to openEsc to close