feat: add mmap-backed dumps for groth16 proving keys

[sergeytimoshin]

This PR adds file-backed unsafe dump support for Groth16 proving keys.

ProvingKey.WriteDump / ReadDump already provide an unsafe raw-memory proving-key serialization path, but ReadDump reconstructs the large proving-key slices on the Go heap. This change adds a parallel mmap-backed dump path:

• concrete Groth16 proving keys expose ProvingKey.WriteMmapDump(path);
• concrete Groth16 curve packages expose ReadMmapDump(path);
• the generic backend/groth16 package exposes groth16.WriteMmapDump(pk, path) and groth16.ReadMmapDump(curveID, path) so callers using the top-level Groth16 API do not need to import a concrete curve package;
• ReadMmapDump returns a closeable MmapProvingKey whose large slices are backed by a read-only memory mapping;
• WithMmapDumpNoDomainPrecompute(threshold) optionally disables FFT domain precomputation on load for applications that prefer lower memory usage over precomputed-domain speed.

This is useful for applications that repeatedly load large Groth16 proving keys, run on memory-constrained machines, or run multiple prover processes that can benefit from OS page-cache sharing instead of each process copying the full proving key into Go heap memory.

This does not claim to solve all low-memory proving cases by itself. Serialized R1CS loading can still be a large heap allocation and should be handled in a follow-up change. This PR focuses on the proving-key side because it extends the existing unsafe dump concept without changing constraint-system serialization.

This PR is also intentionally scoped to Groth16 proving keys. It applies to any gnark circuit proven with Groth16 on the supported curves, but it does not add mmap dumps for PLONK proving keys. PLONK has a different proving-key layout and should be reviewed separately if there is demand for the same loading mode there.

The mmap dump format is intentionally treated as unsafe and local-artifact-only. The reader validates the current process against metadata including curve, operating system, architecture, Go compiler/version, endianness, pointer size, relevant type sizes, and gnark / gnark-crypto versions when build info is available. It also checks section bounds/overlap and core proving-key length invariants before constructing unsafe slices. It does not validate curve points or subgroup membership, matching the unsafe dump model.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How has this been tested?

Added generated tests for all Groth16 curve packages:

• bls12-377
• bls12-381
• bn254
• bw6-761

The tests cover:

• writing a mmap dump and proving/verifying with the mapped proving key,
• proving/verifying commitment circuits with mapped commitment proving keys,
• preserving the serialized FFT domain by default,
• using mapped proving keys through the top-level backend/groth16 API,
• skipping mmap runtime tests on unsupported platforms,
• explicit close behavior for MmapProvingKey.

Focused commands run:

go generate ./internal/generator/backend
go test ./backend/groth16 ./backend/groth16/bn254 ./backend/groth16/bls12-377 ./backend/groth16/bls12-381 ./backend/groth16/bw6-761 ./internal/backend/ioutils/mmap
go test -short -timeout=30m ./backend/groth16/... ./internal/backend/ioutils/mmap

CI-equivalent checks also run locally:

gofmt -l .
go tool goimports -l .
go generate ./...
golangci-lint run -v --timeout=5m
go test -short -timeout=30m ./...
go test -short -timeout=30m -tags=release_checks,solccheck .
go test -short -timeout=30m -tags=prover_checks ./test/...
go test -short -timeout=30m -tags=prover_checks ./examples/...
go test -short -timeout=30m -run=NONE -fuzz=FuzzIntcomp -fuzztime=10s ./internal/backend/ioutils

golangci-lint was run with the CI-pinned version, v2.10.1. go generate ./... was rerun after the template changes and produced only the intended mmap-related generated files.

Additional compile-only checks for the touched Groth16/mmap packages:

GOOS=windows GOARCH=amd64 go test -exec=true -run '^$' ./backend/groth16/... ./internal/backend/ioutils/mmap
GOOS=linux GOARCH=amd64 go test -exec=true -run '^$' ./backend/groth16/... ./internal/backend/ioutils/mmap

How has this been benchmarked?

Added generated benchmarks comparing heap-backed unsafe dump loading with mmap-backed dump loading:

go test ./backend/groth16/bn254 -run '^$' -bench '^BenchmarkMmapDumpProvingKeyLoad$' -benchmem -benchtime=1x

Sample result on Apple M4 Max / darwin arm64:

BenchmarkMmapDumpProvingKeyLoad/ReadDump-16
    130042 ns/op    1248400 B/op    116 allocs/op

BenchmarkMmapDumpProvingKeyLoad/ReadMmapDump-16
    179458 ns/op     296832 B/op    114 allocs/op

BenchmarkMmapDumpProvingKeyLoad/ReadMmapDumpNoDomainPrecompute-16
     88792 ns/op       4448 B/op     40 allocs/op

The benchmark is intentionally focused on loading, not proving. It demonstrates the standalone motivation for this PR: mmap-backed proving-key loading avoids most of the heap allocation from ReadDump, even before adding a follow-up R1CS mmap loader.

Main takeaways from this run:

• ReadMmapDump reduces heap allocation from about 1.25 MB/op to about 297 KB/op for this benchmark key, roughly a 76% allocation reduction.
• The default mmap path is not faster in this small benchmark (179 µs/op vs 130 µs/op), so the primary benefit is lower heap pressure and file-backed sharing rather than lower load latency.
• ReadMmapDumpNoDomainPrecompute reduces allocation to about 4.4 KB/op, roughly a 99.6% reduction versus ReadDump. This shows that most remaining allocation in the default mmap path comes from FFT domain precomputation rather than proving-key slice loading.
• Disabling domain precomputation is a memory/speed tradeoff. It minimizes allocations while loading, but later prover work may spend more CPU computing FFT twiddles on demand.

Checklist:

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
  • Exported API additions include Go doc comments. There is no in-repo user documentation page for backend proving-key serialization to update.
• I have added tests that prove my fix is effective or that my feature works
• I did not modify files generated from templates
  • Note: this PR intentionally adds generated Groth16 backend files and their source templates. The generated files were produced from templates with go generate ./internal/generator/backend, and go generate ./... was rerun afterward to verify generated-file cleanliness.
• golangci-lint does not output errors locally
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
  • No downstream dependency changes are required by this PR.

Follow-up

Related follow-ups:

• mmap-backed or sectioned loading for serialized R1CS data. This is intentionally left out of this PR because it touches constraint-system serialization/deserialization internals and should be reviewed separately.
• PLONK proving-key mmap dumps, if there is demand for the same loading mode for PLONK keys.

---

Note

Medium Risk
New unsafe, trusted-local serialization on the proving path: bad or mismatched dumps could corrupt proving behavior, though metadata checks and tests mitigate load-time mistakes.

Overview
Adds a file-backed mmap path for Groth16 proving keys so large G1/G2 slices can be loaded from disk without copying them onto the Go heap, alongside the existing unsafe WriteDump/ReadDump flow.

Per supported curve (BLS12-377/381, BN254, BW6-761), WriteMmapDump writes aligned raw-memory sections plus JSON metadata (platform, versions, section layout), and ReadMmapDump returns a MmapProvingKey that must stay open until Close. Load uses unsafe views into the mapping; metadata and section bounds are checked, but curve points are not. WithMmapDumpNoDomainPrecompute can strip FFT domain precompute on load to cut heap use at prove time.

The top-level backend/groth16 package exposes WriteMmapDump, ReadMmapDump(curveID, …), and Prove now accepts both in-memory and mmap-backed keys. A small internal/backend/ioutils/mmap helper implements Unix mmap with a non-Unix stub. Code is generated from new templates plus tests/benchmarks comparing ReadDump vs mmap load.

^{Reviewed by Cursor Bugbot for commit 937ec00. Bugbot is set up for automated code reviews on this repo. Configure here.}