Special/upgrading services

API.md

@@ -5,44 +5,46 @@ This is a simplified manual overview of the REST API implemented by the Spring B
 Authentication notes:
 
 - `/actuator/health`, `/v3/api-docs/**`, and Swagger UI are public.
-- `/api/v1/webauthn/**` is public.
-- `POST /api/v1/organizations` and `POST /api/v1/users` require `PLATFORM_OPERATOR` or `ORG_ADMIN`, commonly via `X-Bootstrap-Token` during setup.
-- `/api/v1/admin/**` and `/api/v1/governance/**` require `PLATFORM_OPERATOR` or `ORG_ADMIN`.
+- WebAuthn login start/finish is public, but only verified credentials can issue sessions.
+- WebAuthn registration start/finish requires `X-Bootstrap-Token`, same-user bearer auth, or bearer admin auth.
+- `POST /api/v1/organizations` requires `X-Bootstrap-Token` or bearer `PLATFORM_OPERATOR`.
+- `POST /api/v1/users`, `POST /api/v1/devices`, and `POST /api/v1/bootstrap/sessions` support the bootstrap onboarding flow.
+- `/api/v1/admin/**` and `/api/v1/governance/**` require bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`; bootstrap is not accepted.
 - Other API routes require a bearer session token unless otherwise noted.
 
 ## Organizations
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/organizations` | Create an organization tenant. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `name`, `jurisdiction`, optional `externalTenantId`. | `id`, `createdAt`. | Intended for controlled bootstrap/admin use. |
+| `POST` | `/api/v1/organizations` | Create an organization tenant. | `X-Bootstrap-Token` or bearer `PLATFORM_OPERATOR`. | `name`, `jurisdiction`, optional `externalTenantId`. | `id`, `createdAt`. | Intended for controlled onboarding/platform use. |
 
 ## Users
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/users` | Register a user in an organization. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `organizationId`, `email`, `displayName`, optional `roles`. | `id`, `createdAt`. | Non-bootstrap actors are restricted to their organization. |
+| `POST` | `/api/v1/users` | Register a user in an organization. | `X-Bootstrap-Token`, bearer `PLATFORM_OPERATOR`, or bearer `ORG_ADMIN`. | `organizationId`, `email`, `displayName`, optional `roles`. | `id`, `createdAt`. | Bearer actors are restricted to their organization unless platform-scoped. |
 
 ## Devices
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/devices` | Register a device and attestation record. | Bearer token. | `userId`, `platform`, `deviceName`, `attestationFormat`, `attestationObjectBase64`, `deviceSigningPublicKeyBase64`. | `id`, `createdAt`. | Attestation is stored as pending external verification. |
+| `POST` | `/api/v1/devices` | Register a device and attestation record. | `X-Bootstrap-Token` or bearer token. | `userId`, `platform`, `deviceName`, `attestationFormat`, `attestationObjectBase64`, `deviceSigningPublicKeyBase64`. | `id`, `createdAt`. | Bootstrap may register onboarding devices; bearer actors are same-user or admin scoped. |
 | `DELETE` | `/api/v1/devices/{deviceId}` | Revoke a device. | Bearer token. | Path `deviceId`. | Empty response. | Revokes device trust state and API sessions for that device. |
 
 ## WebAuthn / Passkeys
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/webauthn/registration/options/{userId}` | Start passkey registration. | Public. | Path `userId`. | Challenge and public key credential options. | Challenge is stored server-side. |
-| `POST` | `/api/v1/webauthn/registration/finish` | Finish passkey registration. | Public. | `userId`, optional `deviceId`, `credentialJson` containing the browser credential response. | Session response. | Validates challenge, origin, ceremony type, and replay state before storing credential material; audited authenticator signature verification remains future Yubico ceremony work. |
-| `POST` | `/api/v1/webauthn/login/options/{userId}` | Start passkey login. | Public. | Path `userId`. | Challenge and public key credential options. | Requires an existing credential record. |
+| `POST` | `/api/v1/webauthn/registration/options/{userId}` | Start passkey registration. | `X-Bootstrap-Token`, same-user bearer, or bearer admin. | Path `userId`. | Challenge and public key credential options. | Persists Yubico creation options with a short-lived challenge. |
+| `POST` | `/api/v1/webauthn/registration/finish` | Finish passkey registration. | `X-Bootstrap-Token`, same-user bearer, or bearer admin. | `userId`, optional `deviceId`, `credentialJson` containing the browser credential response. | Session response. | Uses Yubico ceremony verification and stores only `VERIFIED` credentials for login. |
+| `POST` | `/api/v1/webauthn/login/options/{userId}` | Start passkey login. | Public. | Path `userId`. | Challenge and public key credential options. | Requires an existing `VERIFIED` credential; legacy demo credentials are excluded. |
 | `POST` | `/api/v1/webauthn/login/finish` | Finish passkey login. | Public. | `userId`, optional `deviceId`, `credentialJson` containing the browser assertion response. | Session response. | Validates challenge, origin, ceremony type, replay state, and registered credential membership. |
 
 ## Bootstrap Sessions
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/bootstrap/sessions` | Issue a bearer session for a bootstrapped user/device. | `X-Bootstrap-Token`. | `userId`, `deviceId`. | `userId`, `deviceId`, `token`, `expiresAt`. | Use only for local setup or controlled tenant bootstrap; rotate bootstrap tokens. |
+| `POST` | `/api/v1/bootstrap/sessions` | Issue a bearer session for a bootstrapped user/device. | `X-Bootstrap-Token`. | `userId`, `deviceId`. | `userId`, `deviceId`, `token`, `expiresAt`. | Bootstrap is onboarding-only and does not grant admin/governance access. Rotate after setup. |
 
 ## Keys
 
@@ -84,18 +86,18 @@ Authentication notes:
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/admin/actions` | Record a signed admin action envelope. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw signed action envelope JSON string. | Empty response. | Envelope validation and dual control need hardening. |
-| `POST` | `/api/v1/admin/audit/export` | Request audit export. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `organizationId`, `sinkType`, optional `from`, optional `to`. | Empty response. | SIEM sink is configured out of band. |
-| `GET` | `/api/v1/admin/security/verifier` | Check verifier health. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | None. | Verifier status and mode. | Production readiness fails if verifier configuration is missing. |
-| `GET` | `/api/v1/admin/security/transparency-monitor/{organizationId}` | Check key transparency monitor status. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Path `organizationId`. | Entry count, latest log index, latest entry time, verifier mode. | Operational visibility endpoint. |
-| `POST` | `/api/v1/admin/security/audit/verify/{organizationId}` | Ask verifier to validate audit-chain checkpoint state. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Path `organizationId`. | Verification result and checkpoint ID. | Deeper audit replay remains a future verifier enhancement. |
-| `POST` | `/api/v1/admin/emergency-lockdowns` | Start org or room lockdown. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `organizationId`, optional `roomId`, `reason`, `scope`. | Empty response. | Blocks selected activity by org/room. |
-| `POST` | `/api/v1/admin/emergency-lockdowns/{lockdownId}/end` | End a lockdown. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Optional body with `reason`. | Empty response. | Requires careful audit review. |
+| `POST` | `/api/v1/admin/actions` | Record a signed admin action envelope. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw signed action envelope JSON string. | Empty response. | Envelope validation and dual control need hardening. |
+| `POST` | `/api/v1/admin/audit/export` | Request audit export. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `organizationId`, `sinkType`, optional `from`, optional `to`. | Empty response. | SIEM sink is configured out of band. |
+| `GET` | `/api/v1/admin/security/verifier` | Check verifier health. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | None. | Verifier status and mode. | Production readiness fails if verifier configuration is missing. |
+| `GET` | `/api/v1/admin/security/transparency-monitor/{organizationId}` | Check key transparency monitor status. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Path `organizationId`. | Entry count, latest log index, latest entry time, verifier mode. | Operational visibility endpoint. |
+| `POST` | `/api/v1/admin/security/audit/verify/{organizationId}` | Ask verifier to validate audit-chain checkpoint state. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Path `organizationId`. | Verification result and checkpoint ID. | Deeper audit replay remains a future verifier enhancement. |
+| `POST` | `/api/v1/admin/emergency-lockdowns` | Start org or room lockdown. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `organizationId`, optional `roomId`, `reason`, `scope`. | Empty response. | Blocks selected activity by org/room. |
+| `POST` | `/api/v1/admin/emergency-lockdowns/{lockdownId}/end` | End a lockdown. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Optional body with `reason`. | Empty response. | Requires careful audit review. |
 
 ## Governance
 
 | Method | Path | Purpose | Auth | Request | Response | Security notes |
 | ------ | ---- | ------- | ---- | ------- | -------- | -------------- |
-| `POST` | `/api/v1/governance/cql/parse` | Parse a CQL query. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw query string. | Parsed query object. | Parser is experimental. |
-| `POST` | `/api/v1/governance/cql/execute` | Execute a CQL query. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw query string. | List of allowlisted result rows. | Query execution is table and column allowlisted. |
-| `POST` | `/api/v1/governance/smalltalk/evaluate` | Evaluate a Smalltalk policy script. | `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `script`, `context`. | Evaluation result. | Disabled by default and in production unless explicitly enabled; applies script length and plaintext-shaped output checks. |
+| `POST` | `/api/v1/governance/cql/parse` | Parse a CQL query. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw query string. | Parsed query object. | Bootstrap is rejected; parser is experimental. |
+| `POST` | `/api/v1/governance/cql/execute` | Execute a CQL query. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | Raw query string. | List of allowlisted result rows. | Execution is organization-scoped, table/column allowlisted, parameterized, length-limited, and result-capped. |
+| `POST` | `/api/v1/governance/smalltalk/evaluate` | Evaluate a Smalltalk policy script. | Bearer `PLATFORM_OPERATOR` or `ORG_ADMIN`. | `script`, `context`. | Evaluation result. | Disabled by default and in production unless explicitly enabled; applies script length and recursive plaintext-shaped input/output checks. |

ARCHITECTURE.md

@@ -67,7 +67,7 @@ sequenceDiagram
     API->>DB: Validate token hash, expiry, revocation
 ```
 
-WebAuthn/passkey challenge endpoints exist, but finish verification currently fails closed until real passkey verification is configured.
+Bootstrap authentication is an onboarding-only path for initial tenant, user, device, and bootstrap session creation. Admin, governance, key, room, message, attachment, and device-revocation flows require bearer sessions. WebAuthn/passkey registration and login use persisted Yubico ceremony options and store only verified credentials for login; legacy demo credentials are excluded.
 
 ## Message Envelope Flow
 
@@ -129,8 +129,8 @@ Download responses return metadata-only signed grants with object key, ciphertex
 
 The governance layer includes:
 
-- CQL parsing and execution against approved governance data.
-- Smalltalk-style rule evaluation for policy experiments, disabled by default unless explicitly enabled for a runtime profile.
+- CQL parsing and organization-scoped execution against approved governance data with table/column allowlists and result caps.
+- Smalltalk-style rule evaluation for policy experiments, disabled by default unless explicitly enabled for a runtime profile, with recursive plaintext-shaped input/output checks.
 - Room policy versioning.
 - Admin action recording.
 - Emergency lockdown records.

DEPLOYMENT.md

@@ -35,7 +35,7 @@ curl http://localhost:8080/actuator/health
 | `WEBAUTHN_RP_ID` | WebAuthn relying party ID. |
 | `WEBAUTHN_RP_NAME` | WebAuthn relying party display name. |
 | `WEBAUTHN_ALLOWED_ORIGINS` | Allowed WebAuthn origins. |
-| `BOOTSTRAP_TOKEN` | Temporary bootstrap token for initial setup. |
+| `BOOTSTRAP_TOKEN` | Temporary onboarding-only bootstrap token for initial org/user/device/session setup. |
 | `PORT` | Optional backend port; defaults to `8080`. |
 
 Do not use example values outside local development.
@@ -59,7 +59,7 @@ Before deploying beyond local development:
 - Use TLS at the ingress or load balancer.
 - Use non-default, rotated secrets.
 - Restrict actuator exposure.
-- Disable or rotate bootstrap credentials after initial setup.
+- Disable or rotate bootstrap credentials after initial setup; bootstrap is not an admin or governance credential.
 - Configure structured logging without request bodies.
 - Run dependency and container image scans.
 - Confirm Flyway migrations against staging data.
@@ -105,7 +105,7 @@ These manifests require environment-specific review for ingress class, TLS issue
 
 Do not deploy Crypta to production until:
 
-- WebAuthn/passkey finish verification is complete.
+- WebAuthn/passkey implementation has completed external review, including attestation trust policy.
 - Native mobile secure-key storage is complete.
 - Real direct and group cryptographic providers are integrated and reviewed.
 - Key transparency verification is implemented client-side.
@@ -117,7 +117,7 @@ Do not deploy Crypta to production until:
 ## Secret Management Recommendations
 
 - Store secrets in a dedicated secret manager.
-- Rotate bootstrap tokens immediately after setup.
+- Rotate bootstrap tokens immediately after setup and do not use them for admin, governance, key, message, room, attachment, or device-revocation workflows.
 - Use short-lived credentials where possible.
 - Never commit real secrets, private keys, tokens, credentials, or user data.
 - Keep `.env` local and untracked.

README.md

@@ -29,7 +29,15 @@ For a direct Maven run against the Compose database, export `DATABASE_URL`, `DAT
 mvn spring-boot:run
 ```
 
-Provision organizations, users, and devices with `X-Bootstrap-Token`, then exchange a bootstrapped `userId` and `deviceId` at `POST /api/v1/bootstrap/sessions` for a bearer token. WebAuthn challenge creation and finish endpoints validate challenge/origin/replay state, but full audited authenticator signature verification remains a production-readiness item.
+Provision organizations, users, and devices with `X-Bootstrap-Token`, then exchange a bootstrapped `userId` and `deviceId` at `POST /api/v1/bootstrap/sessions` for a bearer token. The bootstrap token is an onboarding credential only; admin, governance, key, room, message, attachment, and device-revocation paths require bearer sessions. WebAuthn/passkey registration and login use persisted Yubico ceremony options and verified credentials, while external WebAuthn review and production attestation policy remain production-readiness items.
+
+Backend tests can be run with:
+
+```bash
+mvn test
+```
+
+The full `BackendFlowIntegrationTest` uses Testcontainers and runs when Docker is available; without Docker it is skipped by design while unit and service-level hardening tests still run.
 
 ## Documentation
 
@@ -99,7 +107,7 @@ flowchart TD
 
 ### User Authentication & Sessions
 
-The project includes WebAuthn/passkey challenge scaffolding, bootstrap session issuance, and random bearer tokens stored as SHA-256 hashes. WebAuthn credential finish verification is not yet configured.
+The project includes bootstrap onboarding, Yubico-backed WebAuthn/passkey ceremony verification, persisted WebAuthn request options, verified credential storage, and random bearer tokens stored as hashes. Legacy demo credentials are marked separately and excluded from login. Bootstrap is limited to onboarding; normal administration and application use require bearer sessions.
 
 Implemented with:
 
@@ -109,6 +117,7 @@ Implemented with:
 * [ApiAuthenticationFilter](src/main/java/com/sovereigncomm/security/ApiAuthenticationFilter.java)
 * [webauthn_credentials](src/main/resources/db/migration/V1__initial_secure_comm_schema.sql)
 * [webauthn_challenges](src/main/resources/db/migration/V2__production_security_runtime.sql)
+* [webauthn_verification_boundary](src/main/resources/db/migration/V5__webauthn_verification_boundary.sql)
 * [api_sessions](src/main/resources/db/migration/V2__production_security_runtime.sql)
 
 ### Zero-Trust Message Ingestion
@@ -124,7 +133,7 @@ Implemented with:
 
 ### Plaintext Prevention Guard
 
-The project uses strict validation logic to reject JSON metadata payloads containing any keys matching patterns for plaintext, body content, or decrypted parameters.
+The project uses strict validation logic to reject JSON metadata payloads containing any keys matching patterns for plaintext, body content, or decrypted parameters, including forbidden keys nested inside maps, lists, and arrays.
 
 Implemented with:
 
@@ -219,15 +228,15 @@ Implemented with:
 
 The platform integrates a dynamic governance plane for real-time compliance auditing and rule-based policy enforcement:
 
-* **Compliance Query Language (CQL)**: An ANTLR4-parsed, SQL-inspired language designed specifically for secure querying of `AUDIT_EVENTS`, `DEVICES`, and `ROOMS`.
+* **Compliance Query Language (CQL)**: An ANTLR4-parsed, SQL-inspired language designed specifically for bearer-admin querying of `AUDIT_EVENTS`, `DEVICES`, and `ROOMS`. Execution is organization-scoped, table/column allowlisted, parameterized, length-limited, and result-capped.
   * Grammar: [CQL.g4](src/main/antlr4/com/sovereigncomm/cql/CQL.g4)
   * Compiler / Service: [CqlPolicyService](src/main/java/com/sovereigncomm/cql/CqlPolicyService.java)
   * Example Query: `SELECT id, event_type FROM AUDIT_EVENTS WHERE event_type = 'AUDIT_EXPORT_REQUESTED'`
-* **Smalltalk Policy Engine**: A highly flexible, lightweight Smalltalk message-passing engine embedded within the Java policy layer to evaluate compliance rules with block evaluations (`[ :param | ... ]`).
+* **Smalltalk Policy Engine**: A highly flexible, lightweight Smalltalk message-passing engine embedded within the Java policy layer to evaluate compliance rules with block evaluations (`[ :param | ... ]`). It is disabled by default and scans input and returned structures for plaintext-shaped metadata.
   * Engine: [SmalltalkEngine](src/main/java/com/sovereigncomm/smalltalk/SmalltalkEngine.java)
   * Service: [SmalltalkService](src/main/java/com/sovereigncomm/smalltalk/SmalltalkService.java)
   * Example Script: `[ :device | device platform = 'iOS' ]`
 * **Governance REST Endpoints**:
   * `POST /api/v1/governance/cql/parse` - Parse CQL query string to abstract AST representation.
-  * `POST /api/v1/governance/cql/execute` - Execute secure CQL query against database audit tables.
+  * `POST /api/v1/governance/cql/execute` - Execute an organization-scoped CQL query against approved governance tables.
   * `POST /api/v1/governance/smalltalk/evaluate` - Evaluate Smalltalk block against target object contexts dynamically.