Track Symphony adoption before implementation

openspec/changes/colony-symphony-import/design.md

@@ -0,0 +1,40 @@
+# Design: Colony-as-tracker
+
+## Status
+
+Accepted for `colony-symphony-import`.
+
+## Context
+
+Symphony's upstream specification uses Linear as the tracker because the source
+service is a Linear-backed scheduler and runner. Colony already has its own task
+threads, file claims, task plans, handoffs, ready-task routing, and OpenSpec
+integration. Using Linear as the adoption tracker would add an external tracker
+that duplicates Colony's coordination model instead of proving it.
+
+## Decision
+
+Colony adoption uses `Colony-as-tracker`.
+
+Linear-as-tracker is N/A for this port. Symphony tracker responsibilities map
+onto Colony primitives:
+
+| Symphony responsibility | Colony primitive |
+| --- | --- |
+| Candidate issue fetch | `task_ready_for_agent`, `task_plan_claim_subtask` |
+| State refresh | task thread state, task timeline, claim state |
+| Terminal fetch | completed/blocked task state plus OpenSpec evidence |
+| Dispatch claim | task plan subtask claim plus file claims |
+| Tracker handoff | `task_post`, `task_hand_off`, `task_relay` |
+| Operator evidence | task notes, structured logs, OpenSpec tasks, PR evidence |
+
+## Consequences
+
+- Core conformance work must not require Linear credentials, Linear GraphQL, or
+  Linear project state.
+- Symphony Section 18.2 Linear GraphQL behavior remains a RECOMMENDED extension,
+  not a core requirement.
+- Wave 5 reconciliation must treat observed Colony task state as read/stop/cleanup
+  input, not implicit permission to write status transitions.
+- Future tracker adapters can be proposed separately without changing the
+  umbrella import decision.

openspec/changes/colony-symphony-import/proposal.md

@@ -0,0 +1,58 @@
+# Colony Symphony Import
+
+## Why
+
+Colony has a durable Symphony context in `openspec/specs/colony-symphony/context.md`,
+but Waves 2-7 need one umbrella change that turns the upstream checklist into
+claimable Colony work. The change records the adoption phases, the tracker
+decision, and the phase-gated verification ledger before implementation starts.
+
+## What Changes
+
+- Open a seven-wave adoption ledger for Symphony-to-Colony work.
+- Map Symphony Section 18.1 REQUIRED conformance items into Colony-native
+  requirements by wave.
+- Map Symphony Section 18.2 RECOMMENDED extensions into optional follow-up
+  requirements without making them blockers for core conformance.
+- Record `Colony-as-tracker` as the tracker decision for this port.
+
+## Scope
+
+1. Wave 1: reference import, domain mapping, and umbrella ledger.
+2. Wave 2: workflow path, `WORKFLOW.md` loading, front matter, config defaults,
+   env indirection, and reload.
+3. Wave 3: Colony tracker intake, candidate refresh, terminal refresh, and typed
+   issue/task normalization.
+4. Wave 4: workspace manager, lifecycle hooks, Codex app-server launch, prompt
+   rendering, retry queue, and retry backoff.
+5. Wave 5: single-authority orchestration, polling, reconciliation, status-flip
+   handling, and terminal cleanup.
+6. Wave 6: structured logs, runtime snapshots, monitoring surface, and optional
+   HTTP control surface.
+7. Wave 7: safety invariants, production validation, extension boundaries, and
+   archive readiness.
+
+## Non-Goals
+
+- Do not vendor, translate, or depend on upstream Symphony Elixir code under any
+  repo path.
+- Do not make Linear the tracker for Colony adoption. Linear-as-tracker is N/A;
+  Colony is the tracker and coordination substrate.
+- Do not start package, app, or runtime implementation work in this change.
+- Do not mark checklist rows complete without command evidence or merged PR
+  evidence.
+
+## Risks
+
+- Wave 5 can accidentally flip task or run status if reconciliation treats
+  observed Colony state as write intent. Wave 5 must prove read/stop/cleanup
+  behavior before any status-transition automation is accepted.
+- Symphony terminology can leak Linear assumptions into Colony. The spec and
+  design docs must keep tracker behavior Colony-native.
+- Optional extensions can blur conformance. Section 18.2 items stay
+  RECOMMENDED unless a later change explicitly promotes them.
+
+## Impact
+
+Agents 200-229 get one OpenSpec surface for claiming, sequencing, and verifying
+the Symphony adoption phases without starting code from a static prompt pack.

openspec/changes/colony-symphony-import/spec.md

@@ -0,0 +1,111 @@
+# Colony Symphony Import Requirements
+
+## ADDED Requirements
+
+### Requirement: Wave 1 Foundation Ledger
+
+Colony SHALL maintain a docs-only foundation wave before implementation begins.
+
+#### Scenario: foundation artifacts are claimed
+
+- **GIVEN** Agents 200-202 are preparing Symphony adoption inputs
+- **WHEN** the umbrella change is opened
+- **THEN** the context, domain mapping, proposal, design, spec, and tasks ledger are present
+- **AND** no upstream Symphony Elixir implementation is vendored, translated, or copied into Colony
+- **AND** Linear-specific tracker behavior is documented as N/A for Colony adoption
+
+### Requirement: Wave 2 Workflow Contract
+
+Colony SHALL implement Symphony Section 18.1 workflow contract items as REQUIRED
+for conformance when this wave enters implementation.
+
+#### Scenario: workflow contract is loaded
+
+- **GIVEN** a Colony runtime starts from a repository workspace
+- **WHEN** workflow configuration is resolved
+- **THEN** explicit runtime path selection and cwd default selection are REQUIRED
+- **AND** `WORKFLOW.md` loading with YAML front matter plus prompt body split is REQUIRED
+- **AND** typed config defaults and `$` environment indirection are REQUIRED
+- **AND** dynamic `WORKFLOW.md` watch/reload/re-apply for config and prompt is REQUIRED
+
+### Requirement: Wave 3 Colony Tracker Intake
+
+Colony SHALL treat Colony task state as the tracker source for Symphony adoption.
+
+#### Scenario: tracker intake reads Colony state
+
+- **GIVEN** Colony is the tracker and coordination substrate
+- **WHEN** candidate work is fetched for dispatch or reconciliation
+- **THEN** candidate fetch, state refresh, and terminal fetch are REQUIRED
+- **AND** tracker payload normalization into stable issue/task models is REQUIRED
+- **AND** Linear GraphQL behavior from Symphony Section 18.2 is RECOMMENDED only when a future
+  extension intentionally targets Linear interoperability
+- **AND** pluggable tracker adapters from Symphony Section 18.2 are RECOMMENDED, not REQUIRED
+
+### Requirement: Wave 4 Workspace And Agent Attempts
+
+Colony SHALL port the Symphony worker-attempt contract around isolated
+workspaces, hooks, app-server launch, prompts, and retries.
+
+#### Scenario: worker attempt starts
+
+- **GIVEN** a claimable Colony task is selected for execution
+- **WHEN** a worker attempt is prepared
+- **THEN** sanitized per-issue or per-task workspaces are REQUIRED
+- **AND** workspace lifecycle hooks `after_create`, `before_run`, `after_run`, and
+  `before_remove` are REQUIRED
+- **AND** hook timeout config with default `60000` ms is REQUIRED
+- **AND** coding-agent app-server subprocess client JSON line protocol is REQUIRED
+- **AND** configurable Codex launch command with default `codex app-server` is REQUIRED
+- **AND** strict prompt rendering with `issue` and `attempt` variables is REQUIRED
+- **AND** exponential retry queue with continuation retries after normal exit is REQUIRED
+- **AND** retry backoff cap with default five minutes is REQUIRED
+- **AND** persisting retry queue and session metadata across process restarts is RECOMMENDED
+
+### Requirement: Wave 5 Reconciliation And Status Safety
+
+Colony SHALL keep one orchestration authority for polling, reconciliation, and
+terminal cleanup without accidental task status flips.
+
+#### Scenario: reconciliation sees inactive or terminal task state
+
+- **GIVEN** a task or run is no longer active
+- **WHEN** reconciliation observes the state change
+- **THEN** polling orchestration with single-authority mutable state is REQUIRED
+- **AND** claimed and running checks before launch are REQUIRED
+- **AND** reconciliation that stops runs on terminal or non-active tracker states is REQUIRED
+- **AND** workspace cleanup for terminal issues via startup sweep and active transition is REQUIRED
+- **AND** status changes MUST NOT be written unless a later change explicitly defines and verifies
+  the Colony write contract
+
+### Requirement: Wave 6 Observability And Optional Control Surface
+
+Colony SHALL expose operator-visible observability for Symphony adoption.
+
+#### Scenario: operator inspects runtime behavior
+
+- **GIVEN** one or more Colony worker attempts have run
+- **WHEN** runtime evidence is inspected
+- **THEN** structured logs with `issue_id`, `issue_identifier`, and `session_id` are REQUIRED
+- **AND** operator-visible observability through structured logs is REQUIRED
+- **AND** runtime snapshot or status surfaces are RECOMMENDED when shipped
+- **AND** HTTP server behavior from Symphony Section 18.2 is RECOMMENDED only if Colony ships that
+  extension, including CLI `--port` precedence, safe default bind host, and baseline endpoints
+- **AND** observability settings in workflow front matter are RECOMMENDED without prescribing UI
+  implementation details
+
+### Requirement: Wave 7 Safety And Production Validation
+
+Colony SHALL close Symphony adoption with safety validation and extension
+boundary checks before archival.
+
+#### Scenario: adoption is ready to archive
+
+- **GIVEN** Waves 1-6 have completed with evidence
+- **WHEN** the umbrella change is prepared for archival
+- **THEN** production validation against the target host OS and shell environment is RECOMMENDED
+- **AND** hook execution and workflow path resolution validation are RECOMMENDED
+- **AND** optional HTTP server bind and port behavior validation is RECOMMENDED if shipped
+- **AND** first-class tracker write APIs are RECOMMENDED only as a future explicit extension
+- **AND** no checklist item is complete without tests, validation output, PR evidence, or documented
+  blocker evidence

openspec/changes/colony-symphony-import/tasks.md

@@ -0,0 +1,79 @@
+# Tasks
+
+Checklist rows stay unchecked until evidence is recorded inline. Agent ID ranges
+are ownership hints for Waves 2-7; claim live Colony tasks before editing.
+
+| Phase | Wave | Agent IDs | Gate |
+| --- | --- | --- | --- |
+| 1 | Foundation ledger | 200-202 | Docs-only umbrella open |
+| 2 | Workflow contract | 203-206 | Workflow loader + config proof |
+| 3 | Colony tracker intake | 207-210 | Candidate/state/terminal refresh proof |
+| 4 | Workspace and attempts | 211-214 | Workspace, hook, app-server, retry proof |
+| 5 | Reconciliation safety | 215-218 | Stop/cleanup proof without status-flip regression |
+| 6 | Observability/control | 219-222 | Structured log and optional surface proof |
+| 7 | Safety/archive | 223-229 | Production validation and archival evidence |
+
+## 1. Wave 1 - Foundation Ledger
+
+- [ ] Agent 200: import Symphony reference context into `openspec/specs/colony-symphony/context.md`.
+- [ ] Agent 201: record domain mapping, including `Colony-as-tracker` and Linear-as-tracker N/A.
+- [ ] Agent 202: create `proposal.md`, `spec.md`, `tasks.md`, and `design.md` for
+  `openspec/changes/colony-symphony-import/`.
+- [ ] Agent 202: verify Wave 1 with `openspec validate --specs`.
+- [ ] Agent 202: record `ls openspec/changes/colony-symphony-import/` evidence.
+- [ ] Agent 202: record `git diff openspec/changes/colony-symphony-import/` evidence.
+
+## 2. Wave 2 - Workflow Contract
+
+- [ ] Agents 203-204: implement workflow path selection, cwd default selection, and `WORKFLOW.md`
+  YAML front matter plus prompt body loading.
+- [ ] Agents 205-206: implement typed config defaults, `$` env indirection, and dynamic reload.
+- [ ] Agents 203-206: verify workflow contract with focused unit tests.
+- [ ] Agents 203-206: verify OpenSpec with `openspec validate --specs`.
+- [ ] Agents 203-206: record PR URL and merge evidence before checking off the wave.
+
+## 3. Wave 3 - Colony Tracker Intake
+
+- [ ] Agents 207-208: map candidate fetch, state refresh, and terminal fetch onto Colony task state.
+- [ ] Agents 209-210: normalize Colony task payloads into stable issue/task models.
+- [ ] Agents 207-210: keep Linear GraphQL behavior out of core conformance unless a later extension
+  explicitly claims it.
+- [ ] Agents 207-210: verify candidate/state/terminal refresh behavior with focused tests.
+- [ ] Agents 207-210: record PR URL and merge evidence before checking off the wave.
+
+## 4. Wave 4 - Workspace And Attempts
+
+- [ ] Agents 211-212: implement sanitized task workspaces and workspace lifecycle hooks.
+- [ ] Agents 213-214: implement hook timeout config, Codex app-server JSON line launch, strict prompt
+  rendering, retry queue, and retry backoff cap.
+- [ ] Agents 211-214: verify workspace containment and hook timeout behavior.
+- [ ] Agents 211-214: verify retry and continuation behavior with deterministic tests.
+- [ ] Agents 211-214: record PR URL and merge evidence before checking off the wave.
+
+## 5. Wave 5 - Reconciliation Safety
+
+- [ ] Agents 215-216: implement single-authority polling state, claimed checks, and running checks.
+- [ ] Agents 217-218: implement terminal/non-active stop behavior and terminal workspace cleanup.
+- [ ] Agents 215-218: add regression coverage proving reconciliation does not write task status unless
+  an explicit Colony write contract exists.
+- [ ] Agents 215-218: verify startup sweep and active-transition cleanup.
+- [ ] Agents 215-218: record PR URL and merge evidence before checking off the wave.
+
+## 6. Wave 6 - Observability And Optional Control Surface
+
+- [ ] Agents 219-220: implement structured logs with `issue_id`, `issue_identifier`, and `session_id`.
+- [ ] Agents 221-222: add runtime snapshot/status surface only if scoped as a RECOMMENDED extension.
+- [ ] Agents 219-222: verify structured log fields and operator-visible evidence.
+- [ ] Agents 219-222: if an HTTP surface ships, verify CLI `--port` precedence, bind host, endpoints, and
+  error semantics.
+- [ ] Agents 219-222: record PR URL and merge evidence before checking off the wave.
+
+## 7. Wave 7 - Safety And Archive
+
+- [ ] Agents 223-225: verify target host OS/shell workflow path resolution and hook execution.
+- [ ] Agents 226-227: verify no upstream Symphony Elixir code is vendored, translated, or copied into
+  Colony paths.
+- [ ] Agents 228-229: validate optional extension boundaries and unresolved blocker notes.
+- [ ] Agents 223-229: run `openspec validate --specs` and any strict change validation required by the
+  final archival lane.
+- [ ] Agents 223-229: record final PR URL, `MERGED` evidence, sandbox cleanup state, and archive readiness.