Skip to main content
This page gives architects the numbers needed to size a Databunker Pro deployment. Every figure below is measured — we loaded a 120-field user-profile vault to 10, 50, and 100 million records against a dedicated AWS RDS PostgreSQL database and recorded throughput, latency, storage, and resource utilisation. The methodology is documented so you can reproduce it with your own payloads.

​
At a glance

  • Storage is linear and predictable: ~5.7 KB per encrypted 120-field record. Multiply by your record count to size the volume (10 M ≈ 59 GB, 100 M ≈ 585 GB).
  • Index size scales with how many fields you make searchable — plan for your field set. This benchmark indexed only email (plus the always-present token and retention indexes) → ~139 bytes/record. Each additional searchable field you index (phone, login, custom) adds another index: budget roughly +40–50 bytes/record and more write CPU per field. Size the database RAM to cache all your indexes (with email only, 100 M ≈ 14 GB; cache-hit ≥ 97%).
  • Detokenisation is fast: ~14 ms per single-record UserGet, regardless of vault size.
  • Write speed depends on scale. At small scale the Databunker Pro app is the limit and one instance does ~5,800 records/sec. At tens of millions of rows the database becomes the limit and write speed tops out around ~12,000 records/sec — so you size the database for large vaults, not the app tier.
  • Rule of thumb: pick your database and application instances from the sizing guide by record count. For most workloads a single Databunker Pro instance is already more than enough.
  • Running on Oracle instead? A comparable 10 M run on RDS Oracle SE2 is included below: similar (slightly faster) read latency, ~2.4× slower writes at peak (PostgreSQL’s multi-row INSERT is more efficient than Oracle’s INSERT ALL), and roughly 3× the on-disk size (CLOB storage). See Oracle Database backend (10 M).

​
What we measured

Three runs, same workload, increasing scale. In every run Databunker Pro (stateless, in Docker) ran on EC2 with the load generator co-located, writing to a dedicated RDS PostgreSQL 16 instance in the same VPC (eu-central-1). Multiple app instances share one database and one wrapping key. Common workload
  • Profile: 120 fields per record (mixed PII + custom application fields).
  • Write path: UserCreateBulk at 4,000 records/request, 8 concurrent workers per instance.
  • Storage & encryption: gp3 volumes; AES-256 at rest, TLS in transit (defaults).
  • Indexes per record: every user always has a unique token index (the detokenisation lookup) and a retention (finaltime) index. Search fields add one salted-hash index each — this benchmark sets only email, giving three indexes/record (~139 bytes total). Databunker Pro can also index phone, login, and custom (a hashed index is written only when the field is present); setting those adds per-insert database CPU and lowers sustained write throughput at scale. name and the other profile fields are encrypted but not indexed.
  • Placement: loaders co-located with Databunker Pro (no client-side network noise); the only network hop measured is App → RDS.
Redis is used only for transient session state and is not exercised by bulk writes; a production deployment points all instances at a shared Redis cluster. A separate 5 M run with PostgreSQL co-located on the app host managed only ~1,700 rec/s — moving to a dedicated database roughly tripled write throughput, which is why every run here uses dedicated RDS.

​
Results

All runs completed with zero failed batches. Per-record storage is essentially constant across scale — storage planning is simple linear math.

​
The key insight: the bottleneck moves with scale

This is the single most important thing to understand when sizing Databunker Pro. At small scale, the application tier is the limit. Encrypting and hash-indexing each record is CPU work done by Databunker Pro. At 10 M records one instance used ~3.2 of its 8 cores while the database sat at just ~20% CPU. Here, adding Databunker Pro instances raises throughput — the stateless app tier scales cleanly (we verified throughput grew almost proportionally from 1 to 6 instances on a small vault), and the database has plenty of spare capacity. At large scale, the database CPU is the limit. As the table grows, every insert costs the database more (deeper B-tree indexes, page splits, WAL, autovacuum). By tens of millions of rows this dominates: Write speed tops out around ~12,000 records/sec. Note that 50 M and 100 M delivered almost the same throughput even though the 100 M run used twice the database vCPU (32 vs 16). Doubling the database didn’t help because the per-insert cost had also doubled with the larger index. To go faster than this, database vCPU has to grow faster than the table. What this means for you:
  • Steady-state and real-time workloads are nowhere near these limits — one Databunker Pro instance on a modest database handles them comfortably.
  • Bulk backfills of very large vaults are limited by the database and take hours (100 M ≈ ~2.3 h here). Size the database for the row count, and don’t expect more app instances to speed it up once the database is at full CPU.

​
How to size your deployment

Pick the row you’re targeting. Storage scales at ~5.7 KB/record. Index RAM depends on how many fields you index — the numbers below assume the benchmark’s set (token + retention + one search index, email) ≈ ~139 bytes/record; add ~40–50 bytes/record for each extra searchable field you index (phone, login, custom). The database is what limits speed at scale, so it’s sized for vCPU (write speed) and RAM (index cache). Application nodes are stateless m6i.2xlarge (Databunker Pro uses ~3 vCPU at full load) — add them to speed up bulk loads only until the database reaches ~70–80% CPU. The index column is for the benchmark’s index set only (token + retention + the single email search index). Note the large spare capacity — 14 GB of indexes on a 128 GB instance — because these instances are chosen for write vCPU, not index RAM, so they hold several more indexes comfortably. If you index more fields, recompute: ~139 bytes/record (email only) + ~40–50 bytes/record per additional searchable field (phone, login, custom), × your record count, and confirm it still fits in RAM. Storage and write vCPU are unaffected by this — only index RAM and per-insert write cost grow. Three sizing rules
  1. Storage = 5.7 KB × records, plus 30–40% spare space for WAL, bloat, and backups.
  2. RAM must exceed your total index size + hot rows so lookups stay cached. Index storage = token + retention + one per searchable field. With only email indexed this was ~139 bytes/record (100 M ≈ 14 GB; cache-hit ≥ 97%); add ~40–50 bytes/record for every extra field you index (phone, login, custom). Multiply by your record count.
  3. Database vCPU is what limits sustained write speed at scale — grow it (or move to Aurora for storage auto-scaling and read replicas) before adding app instances once RDS passes ~70% CPU.

​
Reads: detokenisation latency

Detokenisation (UserGet) is a hashed-index point lookup by token, measured against the fully-loaded 10 M vault: Latency is dominated by the round trip (HTTP, auth, index lookup, decryption), not index depth — the index cache-hit ratio stayed ≥ 97% at every scale, so lookups are served from memory. Read throughput scales horizontally: add Databunker Pro instances (and, for very large vaults, database read replicas) to serve more concurrent lookups.

​
Storage footprint

Encrypted profile blobs dominate the footprint; indexes are comparatively small (which is why a modest amount of RAM keeps them cached). The two figures that scale with your design:
  • Encrypted profile data grows with payload size — a bigger/smaller profile than our 120-field one moves the ~5.5 KB/record directly.
  • Indexes grow with how many fields you make searchable. The ~139 B/record above is the benchmark’s set only — token (always) + retention + the single email search index. Add ~40–50 B/record for each additional indexed field (phone, login, custom). Storage total, heap, and per-record profile size are unaffected by indexing choices — only the index line (and per-insert write CPU) grows.

​
Write latency: bulk vs single record

The ~0.17 ms per record at 10 M is averaged across a 4,000-record bulk request. Individual UserCreateBulk calls take longer end-to-end (p50 ~4.5 s for a 4,000-record batch at 10 M, growing with table size), and single-record synchronous writes are typically 3–10 ms over a real network because each call carries its own connection, auth, and transaction overhead.
  • Backfills / batch flows: use UserCreateBulk.
  • Real-time API flows: assume single-digit-millisecond single-record writes; production throughput = concurrency ÷ per-call latency.

​
Oracle Database backend (10 M)

Databunker Pro also runs on Oracle Database. We repeated the 10 M user-record benchmark on RDS for Oracle so Oracle shops have a directly comparable number. Why a different instance class. The PostgreSQL runs above use db.m6i.2xlarge. In eu-central-1, RDS Oracle Standard Edition 2 does not offer the m6i family — its current-generation m-class options are m5 and m8i. We therefore used db.m8i.2xlarge, the closest current-generation equivalent at the same 8 vCPU / 32 GB. So this is PostgreSQL on m6i.2xlarge vs Oracle on m8i.2xlarge — matched on vCPU, RAM, storage (400 GB gp3 / 12,000 IOPS), and TLS, one CPU generation apart. (We separately confirmed the instance size doesn’t move Oracle’s result: an identical run on db.r6i.2xlarge with double the RAM, 64 GB, produced the same throughput — the load is limited by disk I/O, not memory.) What the numbers say:
  • At peak, Oracle writes are about 2.4× slower than PostgreSQL (~5,100 rec/s vs ~12,000) — mainly because PostgreSQL’s multi-row INSERT is more efficient than Oracle’s INSERT ALL for writing a batch. On a single instance the two are close (~5,100 vs PostgreSQL’s ~5,800 here), but Oracle’s load was already limited by disk I/O (write IOPS near the 12,000 cap while CPU sat at ~18%), so it stays around 5,100 rec/s — whereas PostgreSQL is limited by the app at 10 M and scales toward ~12,000 rec/s by adding stateless app instances. Pushing Oracle higher needs more provisioned IOPS, not more workers.
  • Reads are faster: ~6 ms detok p50 vs ~14 ms on PostgreSQL — helped by the newer m8i CPU. Detokenisation is a hashed-index point lookup either way.
  • Storage is ~3× heavier: ~16.5 KB/record vs ~5.7 KB. Oracle stores the encrypted profile as a CLOB, which is far less space-efficient than PostgreSQL’s TOAST — budget storage accordingly. Indexes are the same ~140 B/record; the index strategy (token + retention + email) is identical across engines.
This run covers the Oracle SE2 user-record vault path only. Format-preserving tokenisation uses a range-partitioned table that requires Oracle Enterprise Edition, so it is out of scope on Standard Edition 2 — see Oracle Database backend for setup and edition guidance.

​
Reproduce this benchmark

Start from the open-source Python load scripts in the databunkerpro-python SDK repo:
1

Provision

Dedicated managed PostgreSQL (RDS / Aurora / Cloud SQL) plus a separate application host in the same VPC.
2

Match your payload

Edit the profile shape in bulk_user_creator.py to the JSON your application actually sends.
3

Load

Run the loader co-located with Databunker Pro, with multiple concurrent workers (and multiple instances for large loads).
4

Measure

Record sustained records/sec, p50/p95/p99 bulk latency, database CPU and IOPS, and application-tier CPU — whichever is maxed out is your bottleneck.
5

Read test

Run bulk_user_fetcher.py against the same vault to capture UserGet latency and throughput.
Your numbers will vary with payload size, number of search indexes, access-control policy depth, and database instance/IOPS. Contact office@databunkertech.com if you’d like the professional-services team to run a benchmark against your representative payloads.

​
Scope & roadmap

These runs cover the PII vault write path (UserCreateBulk) and the detokenisation read path (UserGet). They do not cover the secure session-storage API, format-preserving tokenisation, or agreement/legal-basis flows — those have different characteristics and are measured separately. Coming next: 150 M–200 M measured runs (to replace the projected sizing rows), secure session-storage API, format-preserving tokenisation throughput, and a MySQL backend comparison.

​
Next steps