feat: skill asks for fal app+env, deploys, then runs Playwright

"test cloud" should actually test the user's current working tree,
not whatever code happens to be deployed. Previously the skill
documented running Playwright directly against scope-livepeer-emran,
which could silently false-pass against a stale deploy.

Three changes make this work:

- Parametrize `deploy-staging.sh` on SCOPE_FAL_APP_NAME +
  SCOPE_FAL_ENV (+ optional SCOPE_FAL_AUTH), defaulted from
  .env.local. Also make it track its own HERE path so it works when
  called from any cwd.
- Document both vars in .env.example, grouped into client-side and
  deploy-side sections. SCOPE_CLOUD_APP_ID stays — but we now note
  that the skill derives it from the app+env the user confirms at
  test time (fal's URL convention: main has no suffix, other envs
  get --<env>).
- Rewrite the SKILL's "Running the Playwright test" section with an
  explicit flow: ask for app+env → sanity-check secrets → free port
  8000 → deploy → start scope with derived URL → run Playwright.
  This is what agents should follow when a user says "test cloud".

Also add `deploy-staging.sh` to the repo (it was previously an
untracked per-user file); needed so any contributor following the
skill can actually run the deploy step.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: emranemran <emran.mah@gmail.com>

Author: emranemran

.agents/skills/testing-livepeer-fal-deploy/SKILL.md

@@ -78,13 +78,71 @@ Do **not** use this skill for local-only livepeer testing — that's
 
 ## Running the Playwright test (primary)
 
+When the user says "test cloud" (or any trigger in the description),
+**always deploy their current working tree before running Playwright**.
+Otherwise the test runs against whatever stale code was last deployed
+and can false-positive on their change.
+
+### Step 0 — Ask the user where to deploy
+
+Before anything else, confirm the deploy target. Use AskUserQuestion
+(or plain text prompts) and persist answers for the session:
+
+1. **Fal app name** — required. If `SCOPE_FAL_APP_NAME` is set in
+   `.env.local`, show that value and ask the user to confirm or
+   override. Otherwise ask outright (e.g. `scope-livepeer-<name>`).
+2. **Fal env** — defaults to `main`. If `SCOPE_FAL_ENV` is set in
+   `.env.local`, show and offer to override. Non-default envs (e.g.
+   `preview`) change the URL suffix in `SCOPE_CLOUD_APP_ID` — see
+   below.
+
+Once confirmed, export both for the current shell, and derive /
+overwrite `SCOPE_CLOUD_APP_ID`:
+
+| Env | `SCOPE_CLOUD_APP_ID` |
+|---|---|
+| `main` | `daydream/<app>/ws`         (no suffix) |
+| anything else | `daydream/<app>--<env>/ws`  (with suffix) |
+
+This is a fal convention — the default `main` env is exposed without
+a suffix; all other envs include `--<env>` in the URL. Getting this
+wrong produces `did not receive ready message from websocket`.
+
+### Step 1 — Sanity-check `.env.local`
+
+- `SCOPE_CLOUD_API_KEY` must be set (otherwise:
+  `discover_orchestrators requires discovery_url or signer_url`)
+- `SCOPE_USER_ID` must be set (otherwise the runner's
+  `validate_user_access` rejects with `ACCESS_DENIED`)
+
+If either is missing, stop and ask the user before deploying.
+
+### Step 2 — Kill any scope already on :8000
+
+If another scope process is bound to the port, stop it (or ask the
+user) before continuing. The run-app.sh the script starts must be the
+one under test.
+
+### Step 3 — Deploy
+
+```bash
+SCOPE_FAL_APP_NAME=<app> SCOPE_FAL_ENV=<env> ./deploy-staging.sh
+```
+
+Abort with a clear error if this fails — don't run Playwright against
+stale deployed code. Common failure: the `{git-short-sha}-cloud`
+Docker base image isn't built yet (CI for the current commit is still
+running). If that's the case, either wait for CI or have the user
+confirm they want to deploy against an older base image.
+
+### Step 4 — Start scope and run Playwright
+
 ```bash
 # Terminal 1 — scope (port 8000)
-./run-app.sh
+SCOPE_CLOUD_APP_ID=<derived-url> ./run-app.sh
 
 # Terminal 2 — test
-cd e2e
-npx playwright test
+cd e2e && npx playwright test
 ```
 
 Expected on success (≤5 min cold, ~20 s warm):

.env.example

@@ -1,9 +1,13 @@
 # Copy this file to `.env.local` (gitignored) and fill in real values.
-# Used by run-app.sh and test-cloud-connect.sh.
+# Used by run-app.sh, deploy-staging.sh, and test-cloud-connect.sh.
 
-# Required — fal app ID for your livepeer deployment.
+# --- Client-side (the local scope process) ---
+
+# Required — fal app URL for your livepeer deployment.
 # Format: daydream/<app-name>/ws  (no --main suffix for the default env;
-# for non-default envs the URL includes the env, e.g. --preview/ws)
+# for non-default envs the URL includes the env, e.g. --preview/ws).
+# This MUST match SCOPE_FAL_APP_NAME + SCOPE_FAL_ENV below — the skill
+# derives it for you when it asks which app + env to test against.
 export SCOPE_CLOUD_APP_ID=daydream/<your-app>/ws
 
 # Required — daydream cloud API key (used to auth with signer.daydream.live).
@@ -14,7 +18,23 @@ export SCOPE_CLOUD_API_KEY=sk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 # ~/.daydream-scope/logs/scope-logs-*.log after a UI-driven connect.
 export SCOPE_USER_ID=user_xxxxxxxxxxxxxxxxxxxxxxxxx
 
-# Optional — enable DEBUG logs from livepeer_gateway so per-orchestrator
-# rejection reasons appear in scope.log (e.g. "ACCESS_DENIED",
-# "did not receive ready message from websocket").
+# --- Deploy-side (what deploy-staging.sh pushes to) ---
+
+# Optional default app name for deploy-staging.sh. If unset, the skill
+# asks the user. Example: scope-livepeer-<your-name>
+export SCOPE_FAL_APP_NAME=scope-livepeer-<your-name>
+
+# Optional default env for deploy-staging.sh. Defaults to "main". For
+# non-default envs remember that the URL in SCOPE_CLOUD_APP_ID includes
+# a --<env> suffix (e.g. "daydream/scope-livepeer-foo--preview/ws").
+# export SCOPE_FAL_ENV=main
+
+# Optional — auth mode for the fal deploy. Defaults to "public".
+# export SCOPE_FAL_AUTH=public
+
+# --- Optional ---
+
+# Enable DEBUG logs from livepeer_gateway so per-orchestrator rejection
+# reasons appear in scope.log (e.g. "ACCESS_DENIED", "did not receive
+# ready message from websocket").
 # export LIVEPEER_DEBUG=1

deploy-staging.sh

@@ -0,0 +1,51 @@
+#!/bin/bash
+# Deploy the Livepeer fal wrapper to a fal.ai app.
+#
+# Reads from env (typically sourced from .env.local):
+#   SCOPE_FAL_APP_NAME  required, e.g. "scope-livepeer-emran"
+#   SCOPE_FAL_ENV       optional, defaults to "main"
+#   SCOPE_FAL_AUTH      optional, defaults to "public"
+#
+# Exits non-zero on any failure so callers can fail fast.
+
+set -euo pipefail
+
+HERE="$(cd "$(dirname "$0")" && pwd)"
+
+if [ -f "$HERE/.env.local" ]; then
+    # shellcheck disable=SC1091
+    source "$HERE/.env.local"
+fi
+
+: "${SCOPE_FAL_APP_NAME:?Set SCOPE_FAL_APP_NAME in .env.local (see .env.example). Example: scope-livepeer-<your-name>}"
+SCOPE_FAL_ENV="${SCOPE_FAL_ENV:-main}"
+SCOPE_FAL_AUTH="${SCOPE_FAL_AUTH:-public}"
+
+VENV_DIR="$HERE/.venv-fal"
+
+# Ensure a Python 3.12 venv for fal (matches the scope image).
+if [ ! -d "$VENV_DIR" ]; then
+    echo "Creating Python 3.12 venv at $VENV_DIR..."
+    uv venv --python 3.12 "$VENV_DIR"
+fi
+
+if ! "$VENV_DIR/bin/python" -c "import fal" &>/dev/null; then
+    echo "Installing fal..."
+    uv pip install --python "$VENV_DIR/bin/python" fal
+fi
+
+if ! "$VENV_DIR/bin/fal" auth whoami &>/dev/null; then
+    echo "Not logged in to fal. Running 'fal auth login' (interactive)..."
+    "$VENV_DIR/bin/fal" auth login
+fi
+
+echo "Deploying src/scope/cloud/livepeer_fal_app.py"
+echo "  → app:  $SCOPE_FAL_APP_NAME"
+echo "  → env:  $SCOPE_FAL_ENV"
+echo "  → auth: $SCOPE_FAL_AUTH"
+
+"$VENV_DIR/bin/fal" deploy \
+    "$HERE/src/scope/cloud/livepeer_fal_app.py" \
+    --app "$SCOPE_FAL_APP_NAME" \
+    --auth "$SCOPE_FAL_AUTH" \
+    --env "$SCOPE_FAL_ENV"