feat: Add Sigstore bundle support and cosign-witness-archivista integration#651
feat: Add Sigstore bundle support and cosign-witness-archivista integration#651
Conversation
colek42
force-pushed
the
feat/sigstore-bundle-integration
branch
7 times, most recently
from
30c4904 to
7d66f15
Compare
cmd/archivistactl/cmd/bundle.go
Outdated
| } | ||
|
|
||
| // For now, marshal the DSSE envelope as JSON | ||
| // TODO: Implement bundle reconstruction from DSSE metadata |
This commit addresses critical storage integrity and DoS prevention issues identified in security review: 1. **Fix silent base64 decode errors** (HIGH priority) - pkg/sigstorebundle/export.go: Add error handling for base64 decode - Previously ignored errors could result in corrupted signatures - Now returns explicit errors for malformed or empty data 2. **Implement bundle export reconstruction** (HIGH priority) - cmd/archivistactl/cmd/bundle.go: Implement proper bundle export - Removed TODO, now reconstructs valid Sigstore bundle format - Exports include all verification material (certs, timestamps) - Ensures interoperability with Sigstore tooling (cosign, etc.) 3. **Add resource limits for DoS prevention** (MEDIUM priority) - pkg/sigstorebundle/store.go: Add payload size limit (100MB) - pkg/sigstorebundle/store.go: Add signature count limit (100) - Prevents memory exhaustion and resource exhaustion attacks 4. **Add comprehensive testing** - pkg/sigstorebundle/roundtrip_export_test.go: Unit tests for export - test/integration/cosign-witness/test-bundle-roundtrip.sh: E2E test - Verifies byte-for-byte preservation of key bundle fields - Tests interoperability with cosign verification Architecture note: These fixes focus on Archivista's storage integrity responsibilities. Cryptographic verification (certificate validation, timestamp verification) remains the client's responsibility at retrieval time (go-witness, cosign). Related: #651
This commit addresses critical storage integrity and DoS prevention issues identified in security review: 1. **Fix silent base64 decode errors** (HIGH priority) - pkg/sigstorebundle/export.go: Add error handling for base64 decode - Previously ignored errors could result in corrupted signatures - Now returns explicit errors for malformed or empty data 2. **Implement bundle export reconstruction** (HIGH priority) - cmd/archivistactl/cmd/bundle.go: Implement proper bundle export - Removed TODO, now reconstructs valid Sigstore bundle format - Exports include all verification material (certs, timestamps) - Ensures interoperability with Sigstore tooling (cosign, etc.) 3. **Add resource limits for DoS prevention** (MEDIUM priority) - pkg/sigstorebundle/store.go: Add payload size limit (100MB) - pkg/sigstorebundle/store.go: Add signature count limit (100) - Prevents memory exhaustion and resource exhaustion attacks 4. **Add comprehensive testing** - pkg/sigstorebundle/roundtrip_export_test.go: Unit tests for export - test/integration/cosign-witness/test-bundle-roundtrip.sh: E2E test - Verifies byte-for-byte preservation of key bundle fields - Tests interoperability with cosign verification Architecture note: These fixes focus on Archivista's storage integrity responsibilities. Cryptographic verification (certificate validation, timestamp verification) remains the client's responsibility at retrieval time (go-witness, cosign). Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
colek42
force-pushed
the
feat/sigstore-bundle-integration
branch
from
f5009b4 to
4dc04e4
Compare
✅ Security Review Findings - All AddressedAll critical and medium-priority storage integrity issues have been fixed: 🔴 Critical Fixes (Storage Integrity)1. Silent Base64 Decode Errors - FIXED ✅Files: Before: sigBytes, _ := base64.StdEncoding.DecodeString(sig.Signature)
// Errors silently ignored → corrupted dataAfter: sigBytes, err := base64.StdEncoding.DecodeString(sig.Signature)
if err != nil {
return nil, fmt.Errorf("corrupted signature data in database: %w", err)
}
if len(sigBytes) == 0 {
return nil, fmt.Errorf("signature data is empty")
}Impact: Prevents silent data corruption on export, ensures data integrity. 2. Bundle Export TODO - FIXED ✅Files: Before: Exported raw DSSE envelope instead of proper Sigstore bundle format After: Fully implements bundle reconstruction with:
Impact: Exported bundles now work with Sigstore ecosystem tools. 🟡 Medium Priority (DoS Prevention)3. Resource Limits - FIXED ✅Files: Added:
const (
maxPayloadSize = 100 * 1024 * 1024 // 100MB
maxSignaturesPerBundle = 100 // 100 signatures
)Impact: Protects against resource exhaustion attacks. 🧪 Testing Added
✅ All CI Checks Passing
Architecture ClarificationArchivista's Responsibilities (this PR):
Client's Responsibilities (go-witness, cosign):
Verification happens at retrieval time, not storage time. Commit: 4dc04e4 |
Makes payload size and signature count limits configurable via environment variables instead of hardcoded constants. This allows operators to tune limits based on their deployment requirements. Changes: - pkg/config/config.go: Add ARCHIVISTA_MAX_PAYLOAD_SIZE_MB and ARCHIVISTA_MAX_SIGNATURES_PER_BUNDLE config options - pkg/sigstorebundle/store.go: Add BundleLimits struct with defaults - pkg/sigstorebundle/store.go: Update MapBundleToDSSE to accept optional limits parameter (maintains backward compatibility) - pkg/metadatastorage/sqlstore/store.go: Accept and use BundleLimits - pkg/server/services.go: Pass config values to Store constructor Defaults remain the same: - Max payload size: 100MB - Max signatures per bundle: 100 Example configuration: ```bash export ARCHIVISTA_MAX_PAYLOAD_SIZE_MB=200 export ARCHIVISTA_MAX_SIGNATURES_PER_BUNDLE=50 ``` All tests pass with backward-compatible defaults. Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
✅ Resource Limits Now ConfigurableUpdated the hardcoded resource limits to be configurable via environment variables, as requested. ChangesConfig Options Added: ARCHIVISTA_MAX_PAYLOAD_SIZE_MB=100 # Default: 100MB
ARCHIVISTA_MAX_SIGNATURES_PER_BUNDLE=100 # Default: 100Implementation:
Usage: // Use defaults
limits := sigstorebundle.DefaultBundleLimits()
// Or customize from config
limits := &sigstorebundle.BundleLimits{
MaxPayloadSizeMB: cfg.MaxPayloadSizeMB,
MaxSignaturesPerBundle: cfg.MaxSignaturesPerBundle,
}Backward Compatibility: Commit: 5691c1e |
Adds fuzz tests to discover edge cases in Sigstore bundle parsing, validation, and resource limit enforcement. Fuzzing helps find: - JSON parsing edge cases - Base64 decoding issues - Resource exhaustion vectors - Media type validation bugs Tests added: - FuzzIsBundleJSON: Tests bundle detection with malformed JSON - FuzzParseBundle: Tests JSON unmarshaling edge cases - FuzzMapBundleToDSSE: Tests main conversion logic with random data - FuzzBundleMediaType: Tests media type validation (ReDoS prevention) - FuzzBundleWithLargePayload: Tests payload size limit enforcement - FuzzBundleWithManySignatures: Tests signature count limit enforcement Run individual fuzz tests: ```bash go test ./pkg/sigstorebundle -fuzz=FuzzMapBundleToDSSE -fuzztime=1m ``` Run all fuzz tests (longer): ```bash go test ./pkg/sigstorebundle -fuzz=. -fuzztime=5m ``` Initial 10-second runs found 150+ interesting inputs without crashes, indicating robust error handling in the parser. Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
🔍 Fuzz Testing ResultsRan comprehensive fuzz testing for 5 minutes on the bundle parsing code: FuzzMapBundleToDSSE (Main Conversion Logic)
FuzzIsBundleJSON (Bundle Detection)
SummaryBoth fuzzers successfully tested millions of edge cases including:
No crashes, panics, or memory issues discovered. The error handling in the bundle parser is robust. Run Fuzz Tests Locallyfuzz: elapsed: 0s, gathering baseline coverage: 0/322 completed Commit: 3a138b0 |
colek42
force-pushed
the
feat/sigstore-bundle-integration
branch
2 times, most recently
from
625384c to
7c8a7f0
Compare
📊 Known Performance Consideration: Duplicate JSON ParsingCurrent BehaviorThe format handler pattern currently parses bundle JSON twice:
Why This Is Acceptable (For Now)
Future Optimization OptionsIf profiling shows this is a bottleneck, we have several options: Option 1: Cheap Heuristic Detectionfunc (h *Handler) Detect(obj []byte) bool {
// Fast string matching instead of full parse
return bytes.Contains(obj, []byte(\`"mediaType"\`)) &&
bytes.Contains(obj, []byte(\`"verificationMaterial"\`))
}Pros: Simple, no state, ~50% faster Option 2: MemoizationCache the parsed bundle between Detect() and Store() calls using FNV hash + sync.RWMutex. Pros: Eliminates duplicate parse entirely Option 3: Two-Phase Interface (Future)type Handler interface {
QuickCheck(obj []byte) bool
Parse(obj []byte) (ParsedData, error)
Store(ctx, store, data) error
}Pros: Parse exactly once, clean architecture DecisionDeferring optimization until we have:
Added TODO comment in code for future reference. Related: See deep code review comments above for full architectural analysis. |
…fication Implements complete support for Sigstore bundles in Archivista, enabling storage, retrieval, and verification of cosign-generated attestations. Core Features: - Sigstore bundle detection per official spec (RFC) - DSSE envelope extraction with verification material mapping - RFC3161 TSA timestamp preservation - File-based storage backend configuration - Comprehensive 7-step integration test suite Bundle Support (pkg/sigstorebundle): - types.go: Sigstore bundle structure definitions - store.go: Bundle detection (IsBundleJSON), DSSE mapping (MapBundleToDSSE) - export.go: Bundle export with verification material - Complete test coverage with round-trip validation Schema Changes (ent/schema): - sigstorebundle.go: New SigstoreBundle entity - signature.go: Added certificate and intermediates fields - timestamp.go: Added RFC3161 data field for raw timestamp storage - dsse.go: Added SigstoreBundle edge Storage Flow: 1. Upload bundle via POST /upload 2. Detect Sigstore bundle (validates mediaType, verificationMaterial, content) 3. Extract DSSE envelope from bundle.DsseEnvelope 4. Map verification material (certs, intermediates, RFC3161 timestamps) 5. Store attestation with all verification data preserved 6. Download by gitoid returns complete bundle Integration Tests (test/integration/cosign-witness): - 7-step end-to-end test suite - Cosign bundle generation with TSA timestamps - Upload/download round-trip verification - RFC3161 timestamp preservation validation - All tests passing TSA Verification: - RFC3161 timestamps preserved through storage/retrieval - Timestamp data integrity validated - Supports Sigstore TSA (timestamp.sigstore.dev) - Integration test confirms 957-byte timestamp signatures intact Configuration: - FILE storage backend for local development - Eliminates MinIO dependency Related: - Enables: in-toto/go-witness#595 (Standard attestation policy support) All changes are additive and backwards compatible. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Cole Kennedy <cole@testifysec.com>
This commit addresses critical storage integrity and DoS prevention issues identified in security review: 1. **Fix silent base64 decode errors** (HIGH priority) - pkg/sigstorebundle/export.go: Add error handling for base64 decode - Previously ignored errors could result in corrupted signatures - Now returns explicit errors for malformed or empty data 2. **Implement bundle export reconstruction** (HIGH priority) - cmd/archivistactl/cmd/bundle.go: Implement proper bundle export - Removed TODO, now reconstructs valid Sigstore bundle format - Exports include all verification material (certs, timestamps) - Ensures interoperability with Sigstore tooling (cosign, etc.) 3. **Add resource limits for DoS prevention** (MEDIUM priority) - pkg/sigstorebundle/store.go: Add payload size limit (100MB) - pkg/sigstorebundle/store.go: Add signature count limit (100) - Prevents memory exhaustion and resource exhaustion attacks 4. **Add comprehensive testing** - pkg/sigstorebundle/roundtrip_export_test.go: Unit tests for export - test/integration/cosign-witness/test-bundle-roundtrip.sh: E2E test - Verifies byte-for-byte preservation of key bundle fields - Tests interoperability with cosign verification Architecture note: These fixes focus on Archivista's storage integrity responsibilities. Cryptographic verification (certificate validation, timestamp verification) remains the client's responsibility at retrieval time (go-witness, cosign). Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
Makes payload size and signature count limits configurable via environment variables instead of hardcoded constants. This allows operators to tune limits based on their deployment requirements. Changes: - pkg/config/config.go: Add ARCHIVISTA_MAX_PAYLOAD_SIZE_MB and ARCHIVISTA_MAX_SIGNATURES_PER_BUNDLE config options - pkg/sigstorebundle/store.go: Add BundleLimits struct with defaults - pkg/sigstorebundle/store.go: Update MapBundleToDSSE to accept optional limits parameter (maintains backward compatibility) - pkg/metadatastorage/sqlstore/store.go: Accept and use BundleLimits - pkg/server/services.go: Pass config values to Store constructor Defaults remain the same: - Max payload size: 100MB - Max signatures per bundle: 100 Example configuration: ```bash export ARCHIVISTA_MAX_PAYLOAD_SIZE_MB=200 export ARCHIVISTA_MAX_SIGNATURES_PER_BUNDLE=50 ``` All tests pass with backward-compatible defaults. Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
Adds fuzz tests to discover edge cases in Sigstore bundle parsing, validation, and resource limit enforcement. Fuzzing helps find: - JSON parsing edge cases - Base64 decoding issues - Resource exhaustion vectors - Media type validation bugs Tests added: - FuzzIsBundleJSON: Tests bundle detection with malformed JSON - FuzzParseBundle: Tests JSON unmarshaling edge cases - FuzzMapBundleToDSSE: Tests main conversion logic with random data - FuzzBundleMediaType: Tests media type validation (ReDoS prevention) - FuzzBundleWithLargePayload: Tests payload size limit enforcement - FuzzBundleWithManySignatures: Tests signature count limit enforcement Run individual fuzz tests: ```bash go test ./pkg/sigstorebundle -fuzz=FuzzMapBundleToDSSE -fuzztime=1m ``` Run all fuzz tests (longer): ```bash go test ./pkg/sigstorebundle -fuzz=. -fuzztime=5m ``` Initial 10-second runs found 150+ interesting inputs without crashes, indicating robust error handling in the parser. Related: #651 Signed-off-by: Cole Kennedy <cole@testifysec.com>
This commit addresses several critical issues found in security review: ## Critical Fixes: - **Bundle metadata storage**: storeBundle now properly stores SigstoreBundle records in the database, linking them to DSSE entries. Previously it only logged but didn't persist metadata, breaking database schema integrity. ## High Priority Fixes: - **Code duplication**: Extracted ReconstructBundleFromEnvelope() to eliminate 67 lines of duplicate bundle reconstruction logic between export.go and bundle.go CLI command. - **Configuration validation**: Added validation for MaxPayloadSizeMB and MaxSignaturesPerBundle to prevent invalid values (must be 1-1000). - **Multi-signature handling**: Added warning when exporting DSSE envelopes with multiple signatures to Sigstore bundles (which support only one signature), preventing silent data loss. ## Implementation Details: - Modified storeAttestation() to return DSSE UUID for linking - Created ReconstructBundleFromEnvelope() helper for bundle reconstruction - Added config validation in Process() with clear error messages - Added logrus warning for multi-signature scenarios in export path All changes maintain backward compatibility and pass existing tests. Signed-off-by: Claude Code <noreply@anthropic.com> Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
Signed-off-by: Cole Kennedy <cole@testifysec.com>
The tcr.sh script is a development tool and should not be part of the repository. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Cole Kennedy <cole@testifysec.com>
Documents the known performance consideration where bundle data is parsed twice (Detect + Store). Defers optimization to allow discussion of implementation approaches. Signed-off-by: Claude Code <noreply@anthropic.com> Signed-off-by: Cole Kennedy <cole@testifysec.com>
colek42
force-pushed
the
feat/sigstore-bundle-integration
branch
from
7c8a7f0 to
b092229
Compare
| // Message signature bundles are not yet supported (would need separate storage) | ||
| if bundle.MessageSignature != nil { | ||
| logrus.Warnf("message signature bundle received (not stored in attestations): %s", gitoid) | ||
| // For now, we'll skip storing these bundles in the attestation metadata store |
There was a problem hiding this comment.
Do we want to return an error instead of returning without an error? Looks like the user might get a gitoid back, indicating something was stored but the data is actually dropped.
| }, | ||
| } | ||
|
|
||
| var exportBundleCmd = &cobra.Command{ |
There was a problem hiding this comment.
Is the DSSE ID the same as the gitoid? If not, this will likely return a 404 in all cases.
| if vm.X509CertificateChain != nil && len(vm.X509CertificateChain.Certificates) > 0 { | ||
| // First cert is leaf | ||
| if vm.X509CertificateChain.Certificates[0].RawBytes != "" { | ||
| if cert, err := base64.StdEncoding.DecodeString(vm.X509CertificateChain.Certificates[0].RawBytes); err == nil { |
There was a problem hiding this comment.
looks like we are silently dropping when base64 decoding fails. An attacker could corrupt the data mid flight, causing data to be stored without the certificate chains. We should fail the upload when cert chains or timestamps are corrupted.
3 participants
** This is a DRAFT, opening for visibility.
Summary
Complete implementation of Sigstore bundle v0.3 support in Archivista with end-to-end integration testing using cosign, Witness, and file-based storage.
Changes
Bundle Support
API & Storage
Testing & Verification
Key Features
✅ RFC3161 timestamp support (TSA)
✅ Certificate chain handling
✅ Transparency log integration
✅ File-based local storage
✅ GraphQL bundle queries
✅ Full supply chain verification workflow
Test Results