Acta. Durable execution for .NET

Benchmark

Acta 1.0.0-rc.10: benchmark and release validation

The 1.0.0-rc.10 release evidence, measured by the project on its own machine: one Acta Verify release run on d20e7c77e on PostgreSQL, SQL Server, and SQLite. Every correctness, fault, and chaos scenario passed, every trial's ledger passed the ledger checks, and every performance floor held in the run, with no rerun.

Acta Verify: PASS

The release suite on PostgreSQL, SQL Server, SQLite, commit d20e7c77e, started 2026-11-11 19:12 UTC, 7 h 15 min, on Intel(R) Core(TM) i9-14900K.

PostgreSQLSQL ServerSQLite
Correctnesspasspasspass
Crash-recovery chaospass, 1,000,000 jobs, 84 busy kills reclaimedpass, 1,000,000 jobs, 81 busy kills reclaimedpass, 10,000 jobs, 35 busy kills reclaimed
Ledger checkspass, 157 trials checkedpass, 148 trials checkedpass, 125 trials checked
Performance floorsheldheldheld
Drain83,966 jobs/s50,774 jobs/s8,290 jobs/s
Throughput14,857 jobs/s5,001 jobs/s250 jobs/s
Throughput, one worker7,500 jobs/s4,999 jobs/snot written
14,857 fully executed, durably completed jobs per second on PostgreSQL: 4 workers polling for work, one job per enqueue call, held 3 × 30 s and 120 s at 15,000/s offered, p99 1.6 s, Intel(R) Core(TM) i9-14900K.
5,001 fully executed, durably completed jobs per second on SQL Server: 4 workers polling for work, one job per enqueue call, held 3 × 30 s and 120 s at 5,000/s offered, p99 931 ms, Intel(R) Core(TM) i9-14900K.
250 fully executed, durably completed jobs per second on SQLite: 1 worker, woken in process, one job per enqueue call, held 3 × 30 s and 120 s at 250/s offered, p99 109 ms, Intel(R) Core(TM) i9-14900K.
7,500 fully executed, durably completed jobs per second on PostgreSQL: 1 worker, woken in process, one job per enqueue call, held 3 × 30 s and 120 s at 7,500/s offered, p99 61 ms, Intel(R) Core(TM) i9-14900K.
4,999 fully executed, durably completed jobs per second on SQL Server: 1 worker, woken in process, one job per enqueue call, held 3 × 30 s and 120 s at 5,000/s offered, p99 132 ms, Intel(R) Core(TM) i9-14900K.

Rates to raise

These written rates ask less than the database ran; they decide nothing in this run.

Performance

Each scenario's first written rate is its floor, which this suite requires to hold.

Drain

How fast do eight workers over two namespaces run and durably complete a backlog enqueued before they start?

Setup: workers: 8 on PostgreSQL, 8 on SQL Server, 1 on SQLite; namespaces: 2 on PostgreSQL, 2 on SQL Server, 1 on SQLite; executors per worker: the product default; enqueue calls in flight: 8; jobs per call: 1,000.

DatabasePassJobsSteady rate, database clockWhole cohortDisk write, mean / p95Result
PostgreSQL11,000,00083,966 jobs/s13.5 s0.499 / 2.82 msheld
PostgreSQL21,000,00070,795 jobs/s15 s15.9 / 82.3 msheld
PostgreSQL31,000,00088,711 jobs/s12.3 s7.15 / 41.8 msheld
SQL Server11,000,00036,028 jobs/s28.2 s0.157 / 0.307 msheld
SQL Server21,000,00050,774 jobs/s23.2 s1.32 / 2.42 msheld
SQL Server31,000,00052,189 jobs/s19.8 s0.152 / 0.925 msheld
SQLite1100,0008,359 jobs/s12.3 s0.0389 / 0.117 msheld
SQLite2100,0008,188 jobs/s12.2 s0.0327 / 0.0438 msheld
SQLite3100,0008,290 jobs/s12.2 s0.0405 / 0.12 msheld

Sustained throughput

How many jobs per second, one per enqueue call, does Acta enqueue, run and durably complete with a flat backlog and p99 under 2 s?

Setup: workers: 4 on PostgreSQL, 4 on SQL Server, 1 on SQLite; namespaces: 2 on PostgreSQL, 2 on SQL Server, 1 on SQLite; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: by polling on PostgreSQL, by polling on SQL Server, in process on SQLite; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL10,000110,1611.0 s0.0498 / 0.0517 msheld
PostgreSQL10,00029,8271.0 s0.0385 / 0.0456 msheld
PostgreSQL10,000310,1661.0 s0.0513 / 0.0526 msheld
PostgreSQL12,500112,5621.0 s0.0428 / 0.0565 msheld
PostgreSQL12,500212,573999 ms0.0675 / 0.479 msheld
PostgreSQL12,500312,2621.0 s0.0519 / 0.196 msheld
PostgreSQL15,000114,5481.0 s0.0395 / 0.0426 msheld
PostgreSQL15,000215,0511.0 s0.0404 / 0.0451 msheld
PostgreSQL15,000315,1151.0 s0.0513 / 0.198 msheld
PostgreSQL17,500116,4312.8 s0.0569 / 0.0663 msmissed: backlog grew 880 jobs/s, 175 allowed
PostgreSQL17,500215,8523.8 s0.0583 / 0.0659 msmissed: backlog grew 1,578 jobs/s, 175 allowed
PostgreSQL17,500316,3912.8 s0.0571 / 0.0661 msmissed: backlog grew 1,148 jobs/s, 175 allowed
PostgreSQL20,000not run by rule
PostgreSQL25,000not run by rule
PostgreSQL30,000not run by rule
PostgreSQL15,000confirm, 120 s14,8571.6 s0.455 / 0.407 msheld
SQL Server5,00014,995967 ms0.0329 / 0.131 msheld
SQL Server5,00024,997936 ms0.799 / 12 msheld
SQL Server5,00034,953817 ms0.279 / 5.8 msheld
SQL Server7,50016,6265.2 s0.104 / 0.467 msmissed: backlog grew 859 jobs/s, 75 allowed
SQL Server7,50026,8893.7 s0.628 / 0.688 msmissed: backlog grew 650 jobs/s, 75 allowed
SQL Server7,50036,5955.3 s0.0626 / 0.303 msmissed: backlog grew 911 jobs/s, 75 allowed
SQL Server10,000not run by rule
SQL Server12,500not run by rule
SQL Server15,000not run by rule
SQL Server5,000confirm, 120 s5,001931 ms0.32 / 1.57 msheld
SQLite250125082 ms0.0293 / 0.0304 msheld
SQLite2502250106 ms0.0311 / 0.044 msheld
SQLite2503250103 ms0.0303 / 0.0409 msheld
SQLite500150085 ms0.0243 / 0.0402 msheld
SQLite50024256.1 s0.0256 / 0.0301 msmissed: backlog grew 73.2 jobs/s, 5 allowed
SQLite500339711.9 s0.0284 / 0.0304 msmissed: backlog grew 126 jobs/s, 5 allowed
SQLite1,000168828.2 s0.0259 / 0.0312 msmissed: backlog grew 298 jobs/s, 10 allowed
SQLite2,000not run by rule
SQLite250confirm, 120 s250109 ms0.03 / 0.0309 msheld

Highest rate held in every pass: PostgreSQL 15,000/s; SQL Server 5,000/s; SQLite 250/s.

Sustained throughput, woken through Redis

With Redis carrying each enqueue's wake to the worker processes, how many jobs per second, one per enqueue call, does Acta enqueue, run and durably complete with a flat backlog and p99 under 2 s?

Setup: workers: 4 on PostgreSQL, 4 on SQL Server, 1 on SQLite; namespaces: 2 on PostgreSQL, 2 on SQL Server, 1 on SQLite; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: through Redis on PostgreSQL, through Redis on SQL Server, in process on SQLite; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL10,000110,006103 ms0.04 / 0.049 msheld
PostgreSQL12,500112,50396 ms0.116 / 1.44 msheld
PostgreSQL15,000114,970129 ms0.0541 / 0.0657 msheld
PostgreSQL17,500116,8791.7 s0.613 / 7.96 msmissed: backlog grew 547 jobs/s, 175 allowed
PostgreSQL20,000not run by rule
PostgreSQL25,000not run by rule
PostgreSQL30,000not run by rule
SQL Server5,00015,003198 ms0.0804 / 0.476 msheld
SQL Server7,50016,8354.1 s0.134 / 0.897 msmissed: backlog grew 583 jobs/s, 75 allowed
SQL Server10,000not run by rule
SQL Server12,500not run by rule
SQL Server15,000not run by rule
SQLiteSQLite runs its one worker in the controller's process, where an enqueue wakes it without a transport.

Highest rate held in every pass: PostgreSQL 15,000/s; SQL Server 5,000/s.

Sustained throughput, one worker

How many jobs per second does one worker of default executors, in the process that enqueues, run and durably complete, one job per enqueue call, with a flat backlog and p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00052 ms0.0391 / 0.26 msheld
PostgreSQL5,00025,00152 ms0.0339 / 0.0354 msheld
PostgreSQL5,00035,00159 ms0.0375 / 0.039 msheld
PostgreSQL7,50017,49553 ms0.0339 / 0.0377 msheld
PostgreSQL7,50027,49741 ms0.043 / 0.177 msheld
PostgreSQL7,50037,501187 ms0.0403 / 0.0706 msheld
PostgreSQL10,00019,6982.6 s0.0459 / 0.0586 msmissed: backlog grew 580 jobs/s, 100 allowed
PostgreSQL10,00028,7683.9 s0.0956 / 0.986 msmissed: backlog grew 1,211 jobs/s, 100 allowed
PostgreSQL10,00039,0203.5 s0.0461 / 0.0489 msmissed: backlog grew 920 jobs/s, 100 allowed
PostgreSQL12,500not run by rule
PostgreSQL15,000not run by rule
PostgreSQL7,500confirm, 120 s7,50061 ms0.0476 / 0.0647 msheld
SQL Server2,50012,50289 ms0.296 / 10.9 msheld
SQL Server2,50022,49891 ms0.258 / 6.59 msheld
SQL Server2,50032,50493 ms0.0258 / 0.0403 msheld
SQL Server5,00014,998817 ms0.199 / 3.31 msheld
SQL Server5,00025,00197 ms0.232 / 13.9 msheld
SQL Server5,00035,001161 ms0.269 / 2.99 msheld
SQL Server7,50016,0868.1 s0.157 / 0.551 msmissed: backlog grew 1,212 jobs/s, 75 allowed
SQL Server7,50026,3837.4 s0.133 / 0.65 msmissed: backlog grew 1,005 jobs/s, 75 allowed
SQL Server7,50036,3157.5 s0.249 / 1.04 msmissed: backlog grew 1,062 jobs/s, 75 allowed
SQL Server10,000not run by rule
SQL Server5,000confirm, 120 s4,999132 ms0.199 / 0.669 msheld
SQLiteThe scenario writes no rate for sqlite.

Highest rate held in every pass: PostgreSQL 7,500/s; SQL Server 5,000/s.

Enqueue, one job per call

How fast are single-job enqueue calls acknowledged, each in its own transaction, before any worker runs?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: by polling; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL10,00010210 ms0.0384 / 0.0465 msheld
SQL Server5,0001029 ms0.0816 / 0.804 msheld
SQLite2001022 ms0.0414 / 0.0438 msheld

Highest rate held in every pass: PostgreSQL 10,000/s, every written rate, so the ceiling is above the ladder; SQL Server 5,000/s, every written rate, so the ceiling is above the ladder; SQLite 200/s, every written rate, so the ceiling is above the ladder.

Enqueue in batches of 100

How fast are batches of 100 jobs acknowledged, before any worker runs?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 100; workers learn of new work: by polling; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL10,0001044 ms0.132 / 0.0377 msheld
SQL Server5,0001048 ms3.48 / 0.0431 msheld
SQLite5,0001028 ms0.0354 / 0.0381 msheld

Highest rate held in every pass: PostgreSQL 10,000/s, every written rate, so the ceiling is above the ladder; SQL Server 5,000/s, every written rate, so the ceiling is above the ladder; SQLite 5,000/s, every written rate, so the ceiling is above the ladder.

Enqueue in batches of 1,000

How fast are batches of 1,000 jobs acknowledged, before any worker runs?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1,000; workers learn of new work: by polling; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL20,00010202 ms0.233 / 0.407 msheld
SQL Server10,00010186 ms1.48 / 0.0436 msheld
SQLite10,0001079 ms0.0599 / 0.139 msheld

Highest rate held in every pass: PostgreSQL 20,000/s, every written rate, so the ceiling is above the ladder; SQL Server 10,000/s, every written rate, so the ceiling is above the ladder; SQLite 10,000/s, every written rate, so the ceiling is above the ladder.

Drain, one worker

How fast does one worker run and durably complete a backlog enqueued before it starts?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 8; jobs per call: 1,000.

DatabasePassJobsSteady rate, database clockWhole cohortDisk write, mean / p95Result
PostgreSQL1200,00017,260 jobs/s11.5 s0.26 / 0.932 msheld
SQL Server1200,00012,314 jobs/s15.9 s1.89 / 19.2 msheld
SQLite150,0006,378 jobs/s7.94 s0.0479 / 0.0876 msheld

Drain, four workers

How fast do four workers over two namespaces drain a backlog enqueued before they start?

Setup: workers: 4; namespaces: 2; executors per worker: the product default; enqueue calls in flight: 8; jobs per call: 1,000.

DatabasePassJobsSteady rate, database clockWhole cohortDisk write, mean / p95Result
PostgreSQL1200,00042,844 jobs/s4.72 s0.581 / 1.62 msheld
SQL Server1200,00038,961 jobs/s5.05 s0.185 / 0.942 msheld
SQLiteThe scenario writes no cohort for sqlite.

Drain, sixteen workers

How fast do sixteen workers over two namespaces drain a backlog enqueued before they start?

Setup: workers: 16; namespaces: 2; executors per worker: the product default; enqueue calls in flight: 8; jobs per call: 1,000.

DatabasePassJobsSteady rate, database clockWhole cohortDisk write, mean / p95Result
PostgreSQL11,000,00074,503 jobs/s14.3 s10.7 / 44.8 msheld
SQL Server11,000,00045,701 jobs/s22.9 s1.49 / 0.849 msheld
SQLiteThe scenario writes no cohort for sqlite.

Throughput, sixteen workers

Do sixteen worker processes over two namespaces hold a high offered rate with p99 under 2 s?

Setup: workers: 16; namespaces: 2; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: by polling; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL15,000115,4701.1 s0.059 / 0.0648 msheld
SQL Server4,00014,2441.3 s0.0674 / 0.573 msheld
SQLiteThe scenario writes no rate for sqlite.

Highest rate held in every pass: PostgreSQL 15,000/s, every written rate, so the ceiling is above the ladder; SQL Server 4,000/s, every written rate, so the ceiling is above the ladder.

Throughput, sixteen workers, woken through Redis

With Redis carrying each enqueue's wake, do sixteen worker processes over two namespaces hold a high offered rate with p99 under 2 s?

Setup: workers: 16; namespaces: 2; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: through Redis; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL12,500112,656387 ms0.0763 / 0.433 msheld
SQL Server4,00014,008353 ms0.0312 / 0.0527 msheld
SQLiteSQLite is one process's database; sixteen worker processes are not a SQLite topology.

Highest rate held in every pass: PostgreSQL 12,500/s, every written rate, so the ceiling is above the ladder; SQL Server 4,000/s, every written rate, so the ceiling is above the ladder.

Throughput, sixteen workers on one namespace

Do sixteen worker processes sharing one namespace hold a high offered rate with p99 under 2 s?

Setup: workers: 16; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: by polling; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL15,000115,0971.5 s0.0704 / 0.216 msheld
SQL Server4,00014,0111.9 s0.446 / 7.23 msheld
SQLiteThe scenario writes no rate for sqlite.

Highest rate held in every pass: PostgreSQL 15,000/s, every written rate, so the ceiling is above the ladder; SQL Server 4,000/s, every written rate, so the ceiling is above the ladder.

Throughput, 32 executors

Does one worker with 32 executors hold the stream its default configuration holds?

Setup: workers: 1; namespaces: 1; executors per worker: 32; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00151 ms0.0315 / 0.0325 msheld
SQL Server2,50012,507116 ms0.0747 / 3.43 msheld
SQLite160116095 ms0.0353 / 0.139 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Three persisted steps

Do jobs that commit three checkpoints each hold their offered rate with p99 under 5 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 5.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL1,00011,00195 ms0.0246 / 0.0439 msheld
SQL Server1,00011,000117 ms0.0452 / 0.839 msheld
SQLite4014053 ms0.0335 / 0.041 msheld

Highest rate held in every pass: PostgreSQL 1,000/s, every written rate, so the ceiling is above the ladder; SQL Server 1,000/s, every written rate, so the ceiling is above the ladder; SQLite 40/s, every written rate, so the ceiling is above the ladder.

Transient failure and retry

With one job in four failing twice before it succeeds, does the stream hold its rate with p99 under 10 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 10.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00014,9972.1 s0.0375 / 0.0444 msheld
SQL Server2,50012,5022.2 s0.145 / 1.67 msheld
SQLite16011602.3 s0.0341 / 0.052 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Fixed CPU work

Do jobs that each hash for a fixed number of rounds hold their offered rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL4,00014,00160 ms0.034 / 0.0563 msheld
SQL Server2,50012,49995 ms0.0631 / 2.53 msheld
SQLite160116080 ms0.0327 / 0.0565 msheld

Highest rate held in every pass: PostgreSQL 4,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Payloads of 256 B

Do jobs carrying a 256-byte payload hold their offered rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL4,00014,00357 ms0.0371 / 0.0387 msheld
SQL Server2,50012,49890 ms0.117 / 0.97 msheld
SQLite120112084 ms0.0367 / 0.0556 msheld

Highest rate held in every pass: PostgreSQL 4,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 120/s, every written rate, so the ceiling is above the ladder.

Payloads of 4 KB

Do jobs carrying a 4 KB payload hold their offered rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00087 ms0.738 / 9.07 msheld
SQL Server2,50012,498107 ms0.505 / 6.4 msheld
SQLite120112082 ms0.0369 / 0.0543 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 120/s, every written rate, so the ceiling is above the ladder.

Payloads of 256 B, 4 KB and 64 KB mixed

Do jobs carrying payloads of three sizes in turn hold their offered rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL2,50012,500519 ms0.0458 / 0.0699 msheld
SQL Server1,0001999106 ms2.69 / 23.7 msheld
SQLite160116086 ms0.0346 / 0.0516 msheld

Highest rate held in every pass: PostgreSQL 2,500/s, every written rate, so the ceiling is above the ladder; SQL Server 1,000/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Payloads of 64 KB

Do jobs carrying a 64 KB payload hold their offered rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL1,0001999305 ms5.99 / 83.2 msheld
SQL Server5001500108 ms1.98 / 24.3 msheld
SQLite4014023 ms0.0422 / 0.124 msheld

Highest rate held in every pass: PostgreSQL 1,000/s, every written rate, so the ceiling is above the ladder; SQL Server 500/s, every written rate, so the ceiling is above the ladder; SQLite 40/s, every written rate, so the ceiling is above the ladder.

A thousand lanes

Spread over a thousand ordered lanes, does the stream hold its rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL4,00013,99960 ms0.271 / 3.49 msheld
SQL Server1,0001998102 ms2.47 / 11.7 msheld
SQLite120112089 ms0.0374 / 0.0588 msheld

Highest rate held in every pass: PostgreSQL 4,000/s, every written rate, so the ceiling is above the ladder; SQL Server 1,000/s, every written rate, so the ceiling is above the ladder; SQLite 120/s, every written rate, so the ceiling is above the ladder.

Operator queries during processing

While the stream runs, do the operator list queries answer quickly, and does the stream keep its rate?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00158 ms0.259 / 3.46 msheld
SQL Server2,50012,50085 ms0.0253 / 0.046 msheld
SQLite120112082 ms0.0372 / 0.0631 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 2,500/s, every written rate, so the ceiling is above the ladder; SQLite 120/s, every written rate, so the ceiling is above the ladder.

Parked work alongside

With a large population of jobs parked on signals, does the stream hold its rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00153 ms0.107 / 0.821 msheld
SQL Server500150095 ms0.0252 / 0.0297 msheld
SQLite160116080 ms0.0339 / 0.0518 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 500/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Future high-priority work alongside

With a large population of high-priority jobs due tomorrow, does the stream hold its rate with p99 under 2 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL5,00015,00156 ms0.0606 / 0.303 msheld
SQL Server3201319106 ms1.76 / 15.6 msheld
SQLite160116081 ms0.0336 / 0.225 msheld

Highest rate held in every pass: PostgreSQL 5,000/s, every written rate, so the ceiling is above the ladder; SQL Server 320/s, every written rate, so the ceiling is above the ladder; SQLite 160/s, every written rate, so the ceiling is above the ladder.

Ten children and a join

Do parents that fan out ten children and join them hold their offered rate with p99 under 10 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 10.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL2501251303 ms0.0276 / 0.0436 msheld
SQL Server2001199342 ms0.0513 / 0.325 msheld
SQLite818269 ms0.0317 / 0.0697 msheld

Highest rate held in every pass: PostgreSQL 250/s, every written rate, so the ceiling is above the ladder; SQL Server 200/s, every written rate, so the ceiling is above the ladder; SQLite 8/s, every written rate, so the ceiling is above the ladder.

Controlled I/O through an idempotent sink

Do jobs that each call an external idempotent sink for 5 ms hold their offered rate with p99 under 5 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 5.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL2501251135 ms0.022 / 0.0442 msheld
SQL Server2001200219 ms0.0249 / 0.0742 msheld
SQLite80180.3338 ms0.0344 / 0.0508 msheld

Highest rate held in every pass: PostgreSQL 250/s, every written rate, so the ceiling is above the ladder; SQL Server 200/s, every written rate, so the ceiling is above the ladder; SQLite 80/s, every written rate, so the ceiling is above the ladder.

Short and long work mixed

Do no-op, CPU, sink, timer and fan-out jobs mixed in turn hold their offered rate with p99 under 5 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 5.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL60016011.1 s0.0588 / 0.607 msheld
SQL Server50015011.1 s0.0505 / 1.76 msheld
SQLite80180.21.1 s0.0295 / 0.0436 msheld

Highest rate held in every pass: PostgreSQL 600/s, every written rate, so the ceiling is above the ladder; SQL Server 500/s, every written rate, so the ceiling is above the ladder; SQLite 80/s, every written rate, so the ceiling is above the ladder.

Throughput, one executor

Does a worker with a single executor keep up with a modest stream, so the result shows its serial cost?

Setup: workers: 1; namespaces: 1; executors per worker: 1; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL150115027 ms0.022 / 0.0439 msheld
SQL Server150115095 ms1.11 / 9.12 msheld
SQLite100110026 ms0.0332 / 0.0925 msheld

Highest rate held in every pass: PostgreSQL 150/s, every written rate, so the ceiling is above the ladder; SQL Server 150/s, every written rate, so the ceiling is above the ladder; SQLite 100/s, every written rate, so the ceiling is above the ladder.

One hot lane

How fast does one ordered lane hand its jobs over, each waiting for the previous completion to commit?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL6016025 ms0.0252 / 0.0494 msheld
SQL Server100110061 ms0.124 / 1.64 msheld
SQLite8018042 ms0.0342 / 0.147 msheld

Highest rate held in every pass: PostgreSQL 60/s, every written rate, so the ceiling is above the ladder; SQL Server 100/s, every written rate, so the ceiling is above the ladder; SQLite 80/s, every written rate, so the ceiling is above the ladder.

One hot concurrency key

With every job sharing one concurrency key of limit one, does a stream of one a second keep a flat backlog with p99 under 30 s?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 30.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL11141 ms0.701 / 2.54 msheld
SQL Server11143 ms0.409 / 3.42 msheld
SQLite41428 ms0.122 / 0.866 msheld

Highest rate held in every pass: PostgreSQL 1/s, every written rate, so the ceiling is above the ladder; SQL Server 1/s, every written rate, so the ceiling is above the ladder; SQLite 4/s, every written rate, so the ceiling is above the ladder.

Five-minute mixed soak

Does the mixed workload hold a steady rate for five minutes with p99 under 5 s and a flat backlog?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; workers learn of new work: in process; p99 target: 5.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL70017001.2 s0.0894 / 0.589 msheld
SQL Server35013501.1 s0.0925 / 0.604 msheld
SQLite561561.1 s0.0341 / 0.0425 msheld

Highest rate held in every pass: PostgreSQL 700/s, every written rate, so the ceiling is above the ladder; SQL Server 350/s, every written rate, so the ceiling is above the ladder; SQLite 56/s, every written rate, so the ceiling is above the ladder.

Overload, then recovery

Offered more than it can run, does the backlog grow, and does it fall back once the offered rate drops?

Setup: workers: 1; namespaces: 1; executors per worker: the product default; enqueue calls in flight: 256 on PostgreSQL, 256 on SQL Server, 8 on SQLite; jobs per call: 1; p99 target: 2.0 s.

DatabaseOffered/sPassCompleted/sp99Disk write, mean / p95Result
PostgreSQL13,000, then 2,50017,75012.3 s0.14 / 0.0506 mspass: the backlog grew, then fell back
SQL Server8,000, then 1,00014,50010.2 s0.063 / 0.322 mspass: the backlog grew, then fell back
SQLiteSQLite's one writer serves the enqueues and the worker alike, so an offer past what it absorbs queues in the caller, not in the database.

Correctness

ScenarioPostgreSQLSQL ServerSQLite
Checkpoint depths of 1, 10 and 100 stepspasspasspass
Wide fan-out of 1000 childrenpasspasspass
Nested tree five levels deeppasspasspass
One failing child among tenpasspasspass
Children in another namespacepasspasspass
Signal sent before the waitpasspasspass
Signal sent after the waitpasspasspass
Same signal sent twicepasspasspass
Poison jobs beside healthy onespasspasspass
Order across 1000 lanespasspasspass
Order in one hot lanepasspasspass
One-slot concurrency keypasspasspass
Priority across tenantspasspasspass
Rate limit shared by two workerspasspasspass
New work beside retained historypasspasspass
Retention purges during trafficpasspasspass
Retention of a waiting parent's childrenpasspasspass
Enqueue in a committed transactionpasspasspass
Enqueue in a rolled-back transactionpasspasspass
Compatible step result changepasspasspass
Incompatible input type changepasspasspass
Schema upgrade from a supported releaseskippedskippedskipped

Faults

ScenarioPostgreSQLSQL ServerSQLite
Crash after a checkpoint commitspasspasspass
Crash after an effect, before its checkpointpasspasspass
Crash inside an at-most-once steppasspasspass
Parent dies during fan-outpasspasspass
Last child against the joinpasspasspass
Cancelling a parent during fan-outpasspasspass
Durable timer across a worker restartpasspasspass
Signal sent during wait registrationpasspasspass
Signal against a wait timeoutpasspasspass
Timed-out attempt that keeps runningpasspasspass
Kill a busy workerpasspasspass
Kill every workerpasspasspass
Worker paused past its leasepasspasspass
Database connections cutpasspassskipped
Database process crashpasspassskipped
Lost reply to a completionpasspasspass
Lost reply to an enqueuepasspasspass
Wake signals droppedpasspasspass
Outbox relay crashpasspasspass
Non-idempotent effect, crash before completionpasspasspass
Idempotent effect, crash before its checkpointpasspasspass
Rolling restartpasspasspass

Chaos

ScenarioPostgreSQLSQL ServerSQLite
Worker kills under mixed loadpasspasspass

How this run was made

Disk, by phase and driveWindows counters
before | 0 c:write 2.59 ms mean, 6.53 ms p95, 9 writes/s, 99% idle, 14 samples
before | 1 d:write 4.53 ms mean, 7.57 ms p95, 42 writes/s, 99% idle, 14 samples
chaos mssql | 0 c:write 3.21 ms mean, 8.13 ms p95, 7 writes/s, 99% idle, 1454 samples
chaos mssql | 1 d:write 0.29 ms mean, 1.17 ms p95, 574 writes/s, 73% idle, 1454 samples
chaos pg | 0 c:write 3.11 ms mean, 8.16 ms p95, 5 writes/s, 99% idle, 612 samples
chaos pg | 1 d:write 0.07 ms mean, 0.08 ms p95, 789 writes/s, 54% idle, 612 samples
chaos sqlite | 0 c:write 2.48 ms mean, 7.61 ms p95, 12 writes/s, 94% idle, 371 samples
chaos sqlite | 1 d:write 1.85 ms mean, 7.54 ms p95, 691 writes/s, 80% idle, 371 samples
correctness | 0 c:write 2.41 ms mean, 8.11 ms p95, 11 writes/s, 99% idle, 322 samples
correctness | 1 d:write 0.32 ms mean, 1.54 ms p95, 513 writes/s, 81% idle, 322 samples
mssql | 0 c:write 2.70 ms mean, 7.71 ms p95, 8 writes/s, 99% idle, 3797 samples
mssql | 1 d:write 1.13 ms mean, 6.42 ms p95, 1522 writes/s, 64% idle, 3797 samples
pg | 0 c:write 2.52 ms mean, 7.60 ms p95, 10 writes/s, 99% idle, 4025 samples
pg | 1 d:write 1.35 ms mean, 6.70 ms p95, 699 writes/s, 69% idle, 4025 samples
sqlite | 0 c:write 2.65 ms mean, 7.93 ms p95, 8 writes/s, 99% idle, 2391 samples
sqlite | 1 d:write 0.63 ms mean, 5.94 ms p95, 748 writes/s, 72% idle, 2391 samples
after | 0 c:write 4.03 ms mean, 8.58 ms p95, 5 writes/s, 99% idle, 15 samples
after | 1 d:write 4.69 ms mean, 7.37 ms p95, 7 writes/s, 100% idle, 15 samples