Skip to content

PERF.md

Latest commit

 

History

History
615 lines (540 loc) · 19.7 KB

PERF.md

File metadata and controls

615 lines (540 loc) · 19.7 KB
Raw

Performance Benchmarks

Status: Active — Current benchmark results and testing guide.

Test Environment

  • OS: Linux 6.17.0-8-generic x86_64
  • CPU: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (16 cores / 32 threads, 64 MiB L3)
  • RAM: 122 GiB system memory
  • Build: Rust --release profile
  • Storage/DB: SQLite on disk (WAL mode, synchronous=NORMAL)

Running Performance Tests

Direct cargo test commands below assume one harness thread per invocation. That keeps each run memory-bounded without preventing separate workstreams from launching different cargo test commands in parallel. Override per run with RUST_TEST_THREADS=<n> cargo test ... if needed.

# Preferred: strict serial runner (prevents cross-test interference)
# Also auto-updates this doc's "Latest Results" section.
# `full` includes topo cascade at 10k, 50k, and 500k.
python3 scripts/run_perf_serial.py core
python3 scripts/run_perf_serial.py lowmem
python3 scripts/run_perf_serial.py network
python3 scripts/run_perf_serial.py full

# Optional: run serial perf suite without writing docs/PERF.md
WRITE_PERF_MD=0 python3 scripts/run_perf_serial.py core
WRITE_PERF_MD=0 python3 scripts/run_perf_serial.py lowmem
WRITE_PERF_MD=0 python3 scripts/run_perf_serial.py full

# Optional lowmem matrix overrides
# (defaults shown)
PERF_LOWMEM_BASELINE_SMALL=50000 \
PERF_LOWMEM_BASELINE_BRACKET=100000 \
PERF_LOWMEM_BASELINE_LARGE=500000 \
PERF_LOWMEM_DELTA_TARGET=10000 \
PERF_LOWMEM_DELTA_BRACKET=50000 \
PERF_LOWMEM_FILE_BASELINE=50000 \
PERF_LOWMEM_FILE_COUNT=20 \
PERF_LOWMEM_FILE_SIZE_MIB=1 \
PERF_LOWMEM_CGROUP_ENFORCE=1 \
PERF_LOWMEM_CGROUP_LIMIT_KB=22528 \
python3 scripts/run_perf_serial.py lowmem

# Optional slow lowmem message delta lanes (off by default)
# Includes the serial suite's 100k+50k and 500k+10k message-delta cases.
PERF_LOWMEM_RUN_BRACKET=1 \
PERF_LOWMEM_RUN_LARGE_TARGET=1 \
python3 scripts/run_perf_serial.py lowmem

# Same, but folded into the full maintained perf run.
PERF_LOWMEM_RUN_BRACKET=1 \
PERF_LOWMEM_RUN_LARGE_TARGET=1 \
python3 scripts/run_perf_serial.py full

# Optional lowmem proof-of-concept overrides
# (24MB iOS target, 22MB enforced Linux receiver cap)
PERF_LOWMEM_BUDGET_KB=24576 \
PERF_LOWMEM_CGROUP_ENFORCE=1 \
PERF_LOWMEM_CGROUP_LIMIT_KB=22528 \
PERF_LOWMEM_POC_ENABLE=1 \
PERF_LOWMEM_RUN_LARGE_TARGET=1 \
PERF_LOWMEM_RUN_SMALL_BRACKET=1 \
PERF_LOWMEM_POC_MSG_BASELINE=1000000 \
PERF_LOWMEM_POC_MSG_DELTA=10000 \
PERF_LOWMEM_POC_REALISM_FILE_BASELINE=500000 \
PERF_LOWMEM_POC_REALISM_FILE_COUNT=100 \
PERF_LOWMEM_POC_FILE_COUNT=10000 \
PERF_LOWMEM_POC_FILE_SIZE_MIB=1 \
python3 scripts/run_perf_serial.py lowmem

# Core sync benchmarks (real daemons, warm-start timing, per-daemon VmHWM)
cargo test --release --test daemon_perf_test -- --nocapture --test-threads=1
cargo test --release --test daemon_perf_test -- --nocapture --include-ignored --test-threads=1

# Star topology: hub + N leaves (daemon-managed invite acceptance)
env STAR_TOPOLOGY_LEAVES=50 STAR_TOPOLOGY_HUB_MESSAGES=1 STAR_TOPOLOGY_MESSAGES_PER_LEAF=1 \
  cargo test --release --test daemon_perf_test perf_star_topology_capacity -- --ignored --nocapture --test-threads=1

# File attachment throughput
cargo test --release --test file_throughput_test -- --nocapture --include-ignored --test-threads=1

# Topo-sort cascade benchmark
cargo test --release --test topo_cascade_test topo_cascade_10k -- --nocapture --test-threads=1
cargo test --release --test topo_cascade_test topo_cascade_50k -- --nocapture --ignored --test-threads=1
cargo test --release --test topo_cascade_test topo_cascade_500k -- --nocapture --ignored --test-threads=1
cargo test --release --test topo_cascade_test -- --nocapture --include-ignored --test-threads=1

# Topo-sort cascade (low-memory, iOS NSE budget)
LOW_MEM_IOS=1 cargo test --release --test topo_cascade_test topo_cascade_lowmem_10k -- --nocapture --test-threads=1

# Low-memory budget tests
cargo test --release --test low_mem_test -- --nocapture --test-threads=1

# Low-memory RSS-sampling tests (ignored by default)
cargo test --release --test low_mem_test -- --ignored --nocapture --test-threads=1

# Dedicated lowmem delta harness (direct, Linux-only)
# Requires /proc/<pid>/{status,smaps}; cgroup-enforced mode also requires cgroup v2.
# Baseline runs with both daemons normal; only the receiver restarts in lowmem for the delta.
scripts/run_lowmem.sh delta10k
LOWMEM_BASE_EVENTS=100000 LOWMEM_DELTA_EVENTS=50000 scripts/run_lowmem.sh delta10k
LOWMEM_BASE_EVENTS=500000 LOWMEM_DELTA_EVENTS=10000 scripts/run_lowmem.sh delta10k

# Linux tc-based shaper validation (optional, requires root + tc)
# Validates that the userspace UDP traffic shaper produces results
# comparable to kernel htb + netem on loopback.  Skips gracefully
# without root or when PERF_LINUX_TC is unset.
sudo PERF_LINUX_TC=1 cargo +stable test --release \
  --test tc_network_validation_test -- --ignored --nocapture --test-threads=1

# Realistic traffic-shaper network matrix (maintained separately from `full`)
# Includes WAN-style profiles plus optimistic short-range Wi-Fi / Bluetooth /
# BLE max-link references.
python3 scripts/run_perf_serial.py network

# Optional: run only the short-range max-link profiles
PERF_REALISTIC_NETWORK_PROFILES=wifi-max,bluetooth-max,ble-max \
  cargo +stable test --release --test daemon_realistic_network_perf_test \
  perf_sync_2k_realistic_profiles -- --nocapture --ignored --test-threads=1

Traffic Shaper Network Profiles

The maintained traffic-shaper matrix includes two categories:

  • WAN-style access profiles: cable, dsl, mobile, slow-mobile, starlink
  • Optimistic short-range upper bounds: wifi-max, bluetooth-max, ble-max

The short-range profiles are intentionally framed as optimistic ceilings for nearby peers on clean links. They are not claims about every Wi-Fi or Bluetooth device; they are reference points for "best case local radio" throughput and latency in the shaper.

Current 2k realistic-network matrix from:

cargo +stable test --release --test daemon_realistic_network_perf_test \
  perf_sync_2k_realistic_profiles -- --nocapture --ignored --test-threads=1
Profile Bandwidth RTT Loss Wall Time Msgs/s Peak VmHWM
Wi-Fi Max 150 Mbps 8 ms 0.1% 0.82s 2,430 26.0 MiB
Bluetooth Max 2.1 Mbps 20 ms 0.2% 10.34s 193 29.4 MiB
BLE Max 1.4 Mbps 18 ms 0.1% 15.34s 130 26.8 MiB
Cable 35 Mbps 24 ms 0.2% 1.17s 1,714 25.8 MiB
DSL 3 Mbps 44 ms 0.3% 7.69s 260 27.1 MiB
Mobile 15 Mbps 80 ms 0.8% 5.72s 349 25.8 MiB
Slow Mobile 2 Mbps 140 ms 1.5% 19.63s 102 24.1 MiB
Starlink 15 Mbps 70 ms 0.5% 3.24s 617 25.2 MiB

Latest Results

This section is updated by python3 scripts/run_perf_serial.py when WRITE_PERF_MD=1.

Generated by python3 scripts/run_perf_serial.py full on 2026-03-10 02:05:46 UTC. Lowmem message-delta blocks were refreshed on 2026-03-11 from direct lowmem harness runs because an unrelated file_throughput_test regression currently blocks a clean full auto-regeneration.

Core Sync (daemon_perf_test perf_sync_10k)

cargo +stable test --release --test daemon_perf_test perf_sync_10k -- --nocapture --test-threads=1
=== 10k bidirectional sync (daemon, warm) ===
  Wall time:    2.37s
  Messages:     10000
  Msgs/s:       4211
  Peak RSS:     70.4 MiB (max daemon VmHWM)
  Alice peak RSS: 57.0 MiB
  Bob peak RSS:   70.4 MiB

Core Sync (daemon_perf_test perf_continuous_10k)

cargo +stable test --release --test daemon_perf_test perf_continuous_10k -- --nocapture --test-threads=1
=== 10k continuous sync (daemon, warm) ===
  Wall time:    1.85s
  Messages:     10000
  Msgs/s:       5412
  Peak RSS:     66.6 MiB (max daemon VmHWM)
  Alice peak RSS: 55.2 MiB
  Bob peak RSS:   66.6 MiB

File Throughput (file_throughput_test, include ignored)

cargo +stable test --release --test file_throughput_test -- --nocapture --include-ignored --test-threads=1
=== File throughput: 100.0 MB in 400 slices (262144 B/slice) ===
  Wall time:  0.489s
  Throughput: 204.6 MB/s
  Slices/s:   818
=== File throughput: 10.0 MB in 40 slices (262144 B/slice) ===
  Wall time:  0.049s
  Throughput: 206.0 MB/s
  Slices/s:   824
=== File throughput: 1024.0 MB in 4096 slices (262144 B/slice) ===
  Wall time:  4.995s
  Throughput: 205.0 MB/s
  Slices/s:   820
=== File throughput: 0.2 MB in 1 slices (262144 B/slice) ===
  Wall time:  0.001s
  Throughput: 185.6 MB/s
  Slices/s:   742

Topo Cascade (topo_cascade_10k)

cargo +stable test --release --test topo_cascade_test topo_cascade_10k -- --nocapture --test-threads=1
=== Topo cascade (N=10000, deps_per_event=10) ===
  Setup:         0.067s  (10000 events, 99945 dep rows)
  Blocking:      1.614s  (9990 events blocked via project_one)
  Cascade:       1.168s  (9990 events resolved)
  Cascade rate:  8555 events/s
  Total:         2.850s
  Peak RSS:      59.0 MiB

Core Sync (daemon_perf_test perf_sync_50k, ignored)

cargo +stable test --release --test daemon_perf_test perf_sync_50k -- --nocapture --ignored --test-threads=1
=== 50k one-way sync (daemon, warm) ===
  Wall time:    16.13s
  Messages:     50000
  Msgs/s:       3100
  Peak RSS:     131.2 MiB (max daemon VmHWM)
  Alice peak RSS: 89.3 MiB
  Bob peak RSS:   131.2 MiB

Topo Cascade (topo_cascade_50k)

cargo +stable test --release --test topo_cascade_test topo_cascade_50k -- --nocapture --ignored --test-threads=1
=== Topo cascade (N=50000, deps_per_event=10) ===
  Setup:         0.377s  (50000 events, 499945 dep rows)
  Blocking:      10.633s  (49990 events blocked via project_one)
  Cascade:       6.786s  (49990 events resolved)
  Cascade rate:  7366 events/s
  Total:         17.797s
  Peak RSS:      106.0 MiB

Topo Cascade (topo_cascade_200k)

cargo +stable test --release --test topo_cascade_test topo_cascade_200k -- --nocapture --ignored --test-threads=1
=== Topo cascade (N=200000, deps_per_event=10) ===
  Setup:         1.712s  (200000 events, 1999945 dep rows)
  Blocking:      50.224s  (199990 events blocked via project_one)
  Cascade:       29.736s  (199990 events resolved)
  Cascade rate:  6726 events/s
  Total:         81.672s
  Peak RSS:      203.9 MiB

Topo Cascade (topo_cascade_500k)

cargo +stable test --release --test topo_cascade_test topo_cascade_500k -- --nocapture --ignored --test-threads=1
=== Topo cascade (N=500000, deps_per_event=10) ===
  Setup:         4.764s  (500000 events, 4999945 dep rows)
  Blocking:      92.460s  (499990 events blocked via project_one)
  Cascade:       71.961s  (499990 events resolved)
  Cascade rate:  6948 events/s
  Total:         169.185s
  Peak RSS:      399.6 MiB

Low-Memory (low_mem_ios_functional_smoke_2k)

cargo +stable test --release --test low_mem_test low_mem_ios_functional_smoke_2k -- --nocapture --test-threads=1
=== Low-mem functional smoke (2k) ===
  Alice messages: 1000
  Bob messages:   1000
  Current RSS:    13.98 MiB

Low-Memory (low_mem_ios_budget_smoke_10k, ignored)

cargo +stable test --release --test low_mem_test low_mem_ios_budget_smoke_10k -- --nocapture --ignored --test-threads=1
=== Low-mem budget smoke (10k) ===
  Messages/peer:  5000
  Current RSS:    14.51 MiB
  Budget:         48.00 MiB

Large Trust-Set Low-Memory (low_mem_large_trustset_test)

cargo +stable test --release --test low_mem_large_trustset_test -- --nocapture --test-threads=1
=== Large trust-set correctness ===
  Seeded keys:    100000
  Lookups:        4
  Current RSS:    9.80 MiB
=== Large trust-set budget ===
  Seeded keys:    100000
  Lookups:        1001
  RSS before:     9.99 MiB
  RSS after:      9.99 MiB
  RSS delta:      0.00 MiB
  Budget:         2.00 MiB

Lowmem Delta (50000+10000 messages)

env LOWMEM_BASE_EVENTS=50000 LOWMEM_DELTA_EVENTS=10000 LOWMEM_BUDGET_KB=24576 LOWMEM_CGROUP_ENFORCE=1 LOWMEM_CGROUP_LIMIT_KB=22528 scripts/run_lowmem.sh delta10k
RUN_DIR=/tmp/poc-7-lowmem-large-scenarios-docs/target/lowmem/delta-461601_1773182000
SCENARIO=large_delta
DELTA_KIND=messages
BASE_EVENTS=50000
DELTA_EVENTS=10000
DELTA_FILES=0
DELTA_FILE_SIZE_MIB=0
DELTA_FILE_SLICES_EXPECTED=0
PRE_DELTA_MESSAGES=50000
POST_DELTA_MESSAGES=60000
DELTA_MESSAGES_OBSERVED=10000
PRE_DELTA_FILE_SLICES=0
POST_DELTA_FILE_SLICES=0
DELTA_FILE_SLICES_OBSERVED=0
DELTA_MARKER_PREFIX=delta-notify-461601-1773182000
DELTA_MARKERS_EXPECTED=3
DELTA_MARKERS_SYNCED=3
ALICE_PEAK_VMHWM_MIB=7.21
BOB_PEAK_VMHWM_MIB=7.19
BOB_PEAK_RSS_MIB=7.19
MAX_BOB_ANON_KB=172
MAX_BOB_ANON_UNLABELED_KB=60
MAX_BOB_DB_SHM_KB=0
MAX_BOB_DB_WAL_KB=0
MAX_BOB_TOTAL_MIB=7.19
MAX_BOB_TOTAL_KB=7364
LOWMEM_BUDGET_KB=24576
PASS_UNDER_24MB=1
CGROUP_ENFORCED=1
CGROUP_LIMIT_KB=22528
CGROUP_OOM=0
CGROUP_OOM_KILL=0
MEMTRACE_PRESENT=1
MAX_INIT_WANTED=12
MAX_INIT_NEED_QUEUE=8619
MAX_DATA_EVENTS_INGESTED=10730
MAX_DATA_BLOB=1264
MAX_SQLITE_MEM_CUR=1984816
MAX_SQLITE_MEM_HIGH=2099528
MAX_MALL_ARENA=5582848
MAX_MALL_USED=5308288
MAX_MALL_FREE=2240816
MAX_MALL_MMAP=3518464

Lowmem Delta (100000+50000 messages, slow opt-in)

env LOWMEM_BASE_EVENTS=100000 LOWMEM_DELTA_EVENTS=50000 LOWMEM_BUDGET_KB=24576 LOWMEM_CGROUP_ENFORCE=1 LOWMEM_CGROUP_LIMIT_KB=22528 scripts/run_lowmem.sh delta10k
RUN_DIR=/tmp/poc-7-lowmem-large-scenarios-docs/target/lowmem/delta-465285_1773182030
SCENARIO=large_delta
DELTA_KIND=messages
BASE_EVENTS=100000
DELTA_EVENTS=50000
DELTA_FILES=0
DELTA_FILE_SIZE_MIB=0
DELTA_FILE_SLICES_EXPECTED=0
PRE_DELTA_MESSAGES=100000
POST_DELTA_MESSAGES=150000
DELTA_MESSAGES_OBSERVED=50000
PRE_DELTA_FILE_SLICES=0
POST_DELTA_FILE_SLICES=0
DELTA_FILE_SLICES_OBSERVED=0
DELTA_MARKER_PREFIX=delta-notify-465285-1773182030
DELTA_MARKERS_EXPECTED=3
DELTA_MARKERS_SYNCED=3
ALICE_PEAK_VMHWM_MIB=7.23
BOB_PEAK_VMHWM_MIB=7.31
BOB_PEAK_RSS_MIB=7.31
MAX_BOB_ANON_KB=168
MAX_BOB_ANON_UNLABELED_KB=60
MAX_BOB_DB_SHM_KB=0
MAX_BOB_DB_WAL_KB=0
MAX_BOB_TOTAL_MIB=7.31
MAX_BOB_TOTAL_KB=7488
LOWMEM_BUDGET_KB=24576
PASS_UNDER_24MB=1
CGROUP_ENFORCED=1
CGROUP_LIMIT_KB=22528
CGROUP_OOM=0
CGROUP_OOM_KILL=0
MEMTRACE_PRESENT=1
MAX_INIT_WANTED=12
MAX_INIT_NEED_QUEUE=49986
MAX_DATA_EVENTS_INGESTED=52809
MAX_DATA_BLOB=1264
MAX_SQLITE_MEM_CUR=2096392
MAX_SQLITE_MEM_HIGH=2101296
MAX_MALL_ARENA=5816320
MAX_MALL_USED=5487312
MAX_MALL_FREE=2640592
MAX_MALL_MMAP=3518464

Lowmem Delta (500000+10000 messages, slow opt-in)

env LOWMEM_BASE_EVENTS=500000 LOWMEM_DELTA_EVENTS=10000 LOWMEM_BUDGET_KB=24576 scripts/run_lowmem.sh delta10k
RUN_DIR=/home/holmes/.codex/memories/lowmem-runs-fixed12/delta-1625661_1773252423
SCENARIO=large_delta
DELTA_KIND=messages
BASE_EVENTS=500000
DELTA_EVENTS=10000
DELTA_FILES=0
DELTA_FILE_SIZE_MIB=0
DELTA_FILE_SLICES_EXPECTED=0
PRE_DELTA_MESSAGES=500000
POST_DELTA_MESSAGES=510000
DELTA_MESSAGES_OBSERVED=10000
PRE_DELTA_FILE_SLICES=0
POST_DELTA_FILE_SLICES=0
DELTA_FILE_SLICES_OBSERVED=0
DELTA_MARKER_PREFIX=delta-notify-1625661-1773252423
DELTA_MARKERS_EXPECTED=3
DELTA_MARKERS_SYNCED=3
ALICE_PEAK_VMHWM_MIB=2406.01
BOB_PEAK_VMHWM_MIB=17.28
BOB_PEAK_RSS_MIB=17.28
MAX_BOB_ANON_KB=4608
MAX_BOB_ANON_UNLABELED_KB=848
MAX_BOB_DB_SHM_KB=32
MAX_BOB_DB_WAL_KB=0
MAX_BOB_TOTAL_MIB=17.35
MAX_BOB_TOTAL_KB=17768
LOWMEM_BUDGET_KB=24576
PASS_UNDER_24MB=1
CGROUP_ENFORCED=0
CGROUP_LIMIT_KB=0
CGROUP_OOM=0
CGROUP_OOM_KILL=0
MEMTRACE_PRESENT=1
MAX_INIT_WANTED=0
MAX_INIT_NEED_QUEUE=0
MAX_DATA_EVENTS_INGESTED=9543
MAX_DATA_BLOB=1264
MAX_SQLITE_MEM_CUR=2056720
MAX_SQLITE_MEM_HIGH=2101448
MAX_MALL_ARENA=4534272
MAX_MALL_USED=4246208
MAX_MALL_FREE=1739376
MAX_MALL_MMAP=3018752

Lowmem Delta Files (50000+20x1MiB)

env LOWMEM_BASE_EVENTS=50000 LOWMEM_DELTA_FILES=20 LOWMEM_DELTA_FILE_MIB=1 LOWMEM_BUDGET_KB=24576 LOWMEM_CGROUP_ENFORCE=1 LOWMEM_CGROUP_LIMIT_KB=22528 scripts/run_lowmem.sh deltafiles
RUN_DIR=/tmp/poc-7-claude-perf-followup/target/lowmem/delta-1268525_1773108319
SCENARIO=large_delta
DELTA_KIND=files
BASE_EVENTS=50000
DELTA_EVENTS=23
DELTA_FILES=20
DELTA_FILE_SIZE_MIB=1
DELTA_FILE_SLICES_EXPECTED=80
PRE_DELTA_MESSAGES=50000
POST_DELTA_MESSAGES=50023
DELTA_MESSAGES_OBSERVED=23
PRE_DELTA_FILE_SLICES=0
POST_DELTA_FILE_SLICES=80
DELTA_FILE_SLICES_OBSERVED=80
DELTA_MARKER_PREFIX=delta-notify-1268525-1773108319
DELTA_MARKERS_EXPECTED=3
DELTA_MARKERS_SYNCED=3
ALICE_PEAK_VMHWM_MIB=7.14
BOB_PEAK_VMHWM_MIB=6.85
BOB_PEAK_RSS_MIB=6.85
MAX_BOB_ANON_KB=168
MAX_BOB_ANON_UNLABELED_KB=60
MAX_BOB_DB_SHM_KB=0
MAX_BOB_DB_WAL_KB=0
MAX_BOB_TOTAL_MIB=6.85
MAX_BOB_TOTAL_KB=7016
LOWMEM_BUDGET_KB=24576
PASS_UNDER_24MB=1
CGROUP_ENFORCED=1
CGROUP_LIMIT_KB=22528
CGROUP_OOM=0
CGROUP_OOM_KILL=0
MEMTRACE_PRESENT=1
MAX_INIT_WANTED=0
MAX_INIT_NEED_QUEUE=0
MAX_DATA_EVENTS_INGESTED=0
MAX_DATA_BLOB=0
MAX_SQLITE_MEM_CUR=0
MAX_SQLITE_MEM_HIGH=0
MAX_MALL_ARENA=0
MAX_MALL_USED=0
MAX_MALL_FREE=0
MAX_MALL_MMAP=0

Star Topology (perf_star_topology_capacity, 50 leaves)

env STAR_TOPOLOGY_LEAVES=50 STAR_TOPOLOGY_HUB_MESSAGES=1 STAR_TOPOLOGY_MESSAGES_PER_LEAF=1 cargo +stable test --release --test daemon_perf_test perf_star_topology_capacity -- --nocapture --ignored --test-threads=1
=== Star topology capacity probe (daemon, warm) ===
  Leaf daemons:        50
  Total nodes:         51
  Lowmem:              false
  Hub messages:        1
  Messages/leaf:       1
  Total messages:      51
  Total deliveries:    2601
  Wall time:           4.67s
  Messages/s:          11
  Deliveries/s:        557
  Hub RSS current:     117.2 MiB
  Hub RSS peak:        117.2 MiB
  Leaf RSS current:    avg 21.7 MiB / p95 22.5 MiB / max 22.9 MiB
  Leaf RSS peak:       avg 21.7 MiB / p95 22.5 MiB / max 22.9 MiB
  Hub threads:         208
  Leaf threads:        avg 39.6 / max 40
  Hub FDs:             675
  Leaf FDs:            avg 31.2 / max 33
  Hub maps:            891
  Leaf maps:           avg 202.7 / max 206

Key Design Points

  1. No blob prefetch: blobs fetched on demand, not cached in memory
  2. Reconciliation worker thread: neg.reconcile() runs on a dedicated OS thread so egress drain continues during 100-400ms reconciliation calls
  3. Immediate egress deletion: sent items are deleted (not marked) to keep the egress table small during bulk transfers
  4. Batch transactions: BEGIN/COMMIT around event batches
  5. Priority-first ranges: auth/removal-frontier and key ranges are synced before hot message ranges
  6. Inline projection: project in same transaction as store
  7. Coordinated download: sink-driven round-based assignment avoids redundant transfers from overlapping sources

Environment Variables

LOW_MEM_IOS=1  # iOS NSE low-memory mode (target <=24 MiB RSS)
LOW_MEM_IOS_SOAK_EVENTS=1000000  # optional: override soak event count
LOW_MEM_IOS_SOAK_BUDGET_MIB=24   # optional: override soak RSS budget

24 MiB Target (iOS NSE) — Memory Control Levers

LOW_MEM_IOS=1 enables:

  • SQLite cache ~1 MiB per connection
  • temp_store=FILE, mmap_size=0
  • Smaller WAL/journal limits
  • Ingest channel capacity reduced to 1000

Additional tuning options:

  • PRAGMA cache_size = -1024 (1 MiB)
  • Minimize open connections (one read snapshot + one writer)
  • PRAGMA wal_autocheckpoint and PRAGMA journal_size_limit to keep WAL small

CLI Tools for Manual Testing

# Generate test messages
cargo run --release -- generate --count 50000 --db test.db

# Run sync between two terminals
cargo run --release -- sync --bind 127.0.0.1:4433 --db topo.db
cargo run --release -- invite --db topo.db --bootstrap 127.0.0.1:4433
# then on client:
cargo run --release -- accept --db client.db topo://invite/...
cargo run --release -- sync --bind 127.0.0.1:4434 --db client.db

# Check status
cargo run --release -- status --db topo.db