ENG-3771: add lab project commands

[mrmoxon]

Linear: ENG-3771

Summary

Adds Lab Project support to the Prime CLI and eval client so users can set up a projectized Lab workspace, create/select/clear/inspect/update projects, and attach artifacts to projects from the terminal.

UX

prime lab setup now folds projects into the normal setup flow. When the user is authenticated and no active workspace project exists, setup creates a default project named after the workspace folder and makes it active in .prime/lab/context.json. Users can bind an existing project with prime lab setup --project <project-id-or-slug>, choose the generated project name with prime lab setup --project-name "Alphabet Sort Baselines", or keep setup local-only with prime lab setup --no-project. If the user is not authenticated or the Projects API is unavailable, local setup still succeeds and prints the follow-up prime project create path.

Projects remain compatible with existing Lab flows. Users who do not create or select a project can keep running prime train, prime eval push, and prime eval run as they do today; without an active project, those commands run without project attachment. Once a workspace has an active project, new training runs and evaluations attach to it by default. Users can override that per command with --project <project-id-or-slug> or opt out explicitly with --no-project. Active project context is local to the workspace and ignored when the CLI team or API base URL does not match the current config.

Active project context follows the active CLI account context. A personal CLI context can only set a personal project as active; a team CLI context can only set a project from that same team as active. --team-id is still supported for explicit one-off team operations such as listing, showing, creating with --no-use, assigning, or removing, but it does not switch future prime train / prime eval commands into that team. Users should run prime switch <team-slug-or-id> or set PRIME_TEAM_ID=<team-id> before making a team project active.

Active project lookup is anchored to the nearest Lab workspace marker, so nested paths such as outputs/evals/... still resolve the workspace project, but nested Lab workspaces do not accidentally inherit a parent workspace's active project.

A typical projectized workspace flow looks like:

mkdir alphabet-sort && cd alphabet-sort
prime lab setup
prime project current

prime train rl.toml
prime eval push outputs/evals/gsm8k--gpt-4/abc123
prime eval run gsm8k

For team-owned projects, switch to the team first so setup creates the default project in that team context:

prime switch <team-slug-or-id>
prime lab setup
prime train rl.toml

Users can still switch projects with prime project use <project-id>, stop default attachment with prime project clear, and organize existing artifacts with prime project assign / prime project remove for runs, evals, and adapters.

Details

• Adds prime project commands for creating/listing/showing/using/checking current/clearing/updating/assigning/removing a project.
• Adds default project creation/binding to prime lab setup, with --project, --project-name, and --no-project controls.
• Keeps local Lab setup non-blocking when the user is unauthenticated or project creation fails.
• Adds active project context resolution for training and evaluation flows.
• Guards active project writes so a project can only become the workspace default when its team scope matches the current CLI account/team context.
• Resolves active context from the nearest Lab workspace marker to avoid crossing nested workspace boundaries.
• Keeps project current --output json stable if API detail fetch fails by returning project: null and separate cached context metadata.
• Adds project IDs to eval push/create/update and hosted eval paths.
• Adds project assignment helpers for hosted training runs and adapters.
• Supports additive project assignment for runs/adapters via assign, and clear-or-specific removal via remove.
• Supports eval project assignment/clear. Targeted removal from one named project is intentionally rejected because evaluations have a single project association.
• Adds regression tests for lab setup project creation/binding/skip/fallback, training run project attachment, hosted eval project attachment, eval push forwarding, eval project clearing, and run/adapter assignment operations.
• Documents projectized Lab setup and project CLI usage in the repo READMEs, plus project assignment in the prime-evals README.

Not in this PR

• Project archive/restore is not implemented in the CLI.
• Project environment aggregation, such as prime project environments, is left for a follow-up backend-backed endpoint.

Validation

• uv run pytest packages/prime/tests/test_lab_setup.py -q
• uv run pytest packages/prime/tests/test_lab_setup.py packages/prime/tests/test_projects_cli.py packages/prime/tests/test_eval_push.py packages/prime/tests/test_hosted_eval.py packages/prime/tests/test_rl_api.py -q
• uv run pytest packages/prime/tests packages/prime-evals/tests -q (797 passed, 4 skipped)
• uv run ruff check packages/prime/src/prime_cli/lab_setup.py packages/prime/src/prime_cli/utils/projects.py packages/prime/src/prime_cli/commands/projects.py packages/prime/tests/test_lab_setup.py
• uv run ruff format --check packages/prime/src/prime_cli/lab_setup.py packages/prime/src/prime_cli/utils/projects.py packages/prime/src/prime_cli/commands/projects.py packages/prime/tests/test_lab_setup.py
• uv run ty check packages/prime/src packages/prime-evals/src
• git diff --check
• Local backend smoke: created a team project with --team-id <team-id> --no-use, confirmed no context file was written, confirmed personal context rejects project use <team-project> --team-id <team-id>, then confirmed matching PRIME_TEAM_ID=<team-id> allows project use and writes a team-scoped .prime/lab/context.json.

---

Note

Medium Risk
Touches default attachment behavior for training and eval commands and team-scoped active project rules; mistakes could attach artifacts to the wrong project, but changes are opt-out via --no-project and heavily tested.

Overview
This PR adds Lab Projects end-to-end: a new prime project command group, workspace-scoped active project context, and optional project attachment across training and eval flows.

CLI & workspace context. New ProjectsClient and prime project commands (create, list, show, use, current, update, clear, assign/remove for runs, evals, adapters). Active project is stored in .prime/lab/context.json (gitignored), resolved from the nearest Lab workspace, and scoped to the current API base URL and team. prime lab setup can create or bind a default project (--project, --project-name, --no-project) without blocking local setup when unauthenticated or the API fails.

Training & deployments. Hosted training runs accept project_id on create and via update_run_project (add/remove/clear, optional adapter moves). Adapters get update_adapter_project with the same operation model.

Evaluations. prime-evals gains project_id on create and assign/clear on update_evaluation (clear_project sends project_id: null). prime eval push / prime eval run (local and hosted) and post-run hub upload honor --project / --no-project and the active project by default.

CI & docs. GitHub Actions bumps checkout to v6 and pins setup-uv to 0.11.15 with cache disabled. READMEs document projectized Lab workflows.

Tests cover setup project flows, CLI project commands, eval/train payload wiring, and scope/boundary rules for active context.

^{Reviewed by Cursor Bugbot for commit 530dca9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[willccbb]

@mrmoxon thoughts on folding this into the setup flow? ie default = single project that shares name with workspace folder, overridable + switchable. a bit worried about abstraction creep, feels like we can have these coupled until users explicitly want to decouple

[mrmoxon]

@mrmoxon thoughts on folding this into the setup flow? ie default = single project that shares name with workspace folder, overridable + switchable. a bit worried about abstraction creep, feels like we can have these coupled until users explicitly want to decouple

Defo agree, am making the lab setup flow more subtle with next commit. Want to minimise user knowledge gradients. Thanks for the suggestion.

[mrmoxon]

Addressed the setup-flow point: prime lab setup now creates an active default project named after the workspace folder when authenticated, with --project to bind an existing project, --project-name to override the generated name, and --no-project for local-only setup. Setup still succeeds without auth/API availability and prints the follow-up project command path.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 530dca9. Configure here.}

[cursor]

Redundant no-op scope validation before project creation

Low Severity

The first ensure_active_project_scope(config.team_id, config, ...) call passes config.team_id as both the project_team_id argument and then compares it against config.team_id inside the function. Since project_team_id == config.team_id is tautologically True, this call can never raise and is dead code. The second call after projects_client.create(...) is the meaningful guard (validating the API-returned project.team_id), making the first one purely redundant.

 

^{Reviewed by Cursor Bugbot for commit 530dca9. Configure here.}