Skip to content

fix(harness/01-travel-agent): unblock the browser tool, redesign the code-interpreter step, handle stream errors - #1882

Open
rmncardoso wants to merge 1 commit into
awslabs:mainfrom
rmncardoso:fix/travel-agent-browser-iam-and-code-interpreter
Open

fix(harness/01-travel-agent): unblock the browser tool, redesign the code-interpreter step, handle stream errors#1882
rmncardoso wants to merge 1 commit into
awslabs:mainfrom
rmncardoso:fix/travel-agent-browser-iam-and-code-interpreter

Conversation

@rmncardoso

Copy link
Copy Markdown
Contributor

What this fixes

Two of the six parts in the Travel Guide use case look like they work but silently don't,
plus some error-handling gaps. Every change below was verified with a full 6-part live run on
a fresh execution role in a real AWS account, ending in clean teardown.

Part 5 (browser) — silently returns fabricated data

The agentcore_browser tool opens a CDP WebSocket to the browser automation stream, but the
execution role didn't allow it, so the connection was rejected:

BrowserType.connect_over_cdp: WebSocket error:
wss://.../browser-streams/aws.browser.v1/sessions/.../automation 403 Forbidden
User is not authorized to access automation stream

The agent then quietly fell back to guessing — a run in August produced December
weather. Nothing in the output said the browser had failed.

Fix: add bedrock-agentcore:ConnectBrowserAutomationStream and
ConnectBrowserLiveViewStream to utils/iam.py, scoped to the browser resource. The
built-in browser is owned by the aws account
(arn:aws:bedrock-agentcore:*:aws:browser/aws.browser.v1, confirmed via
list-browsers --type SYSTEM), not the caller's, so an ARN built from the caller's account
ID would never match. Action names were verified with accessanalyzer validate-policy; the
full policy returns zero Access Analyzer ERROR/SECURITY_WARNING findings.

Part 6 (code interpreter) — structurally impossible as written

The code interpreter runs in a separate sandbox whose filesystem is isolated from the
harness microVM, in both directions (verified: the sandbox gets [Errno 2] No such file or directory for a VM file, and a marker the sandbox writes is not visible from the VM). The
original code asked the sandbox to read /tmp/tourism_data.json from the VM and write a
.png back to it — neither crosses the boundary, so the chart/report step always failed,
papered over by || echo 'No report' and || echo 'NO_CHART'.

Rendering on the VM instead isn't possible as written either — the VM has no matplotlib and
no pip/uv — and bridging a PNG's base64 back through the model overflows the turn's token
limit (runtimeClientError: Model stopped generating due to maximum token limit).

Redesign: the sandbox does the analysis as text, with the data passed inline (text is the
only thing that crosses the boundary), and the VM renders a self-contained HTML/CSS chart
— exactly how Parts 2, 3 and 5 already emit HTML. Live result: a 13,137-char
amsterdam_tourism.html with real figures and zero external dependencies.

Error handling and robustness

  • stream_response (and the chat server's loop) only handled internalServerException.
    validationException and runtimeClientError streamed by silently, so a failed turn
    looked like a success and a later step acted on empty output. Now all three of the stream's
    error members are checked, and BotoCoreError/ClientError from the call itself is wrapped
    — a failed turn stops the demo loudly.
  • run_command / fetch_file discarded the shell exitCode, so a command that failed
    was indistinguishable from one that succeeded. They now surface it, and fetch_file returns
    "" on a missing file instead of relying on a shell fallback.
  • travel_chat/server.py: os.environ["HARNESS_ARN"] raised a bare KeyError at import
    when unset — now a clear SystemExit with setup guidance (empty string handled too). Added
    the omitted model= so the chat server and travel_agent.py use the same model rather than
    a server-side default that could differ. Fixed the two pre-existing ruff findings the file
    carried (EXE001 shebang-without-exec-bit, G201 exc_info=Truelogger.exception),
    which CI flags once the file is touched; both are behaviour-preserving.

README

Corrected the browser troubleshooting entry (it blamed VPC/internet reachability; the real
cause is the IAM permission), corrected the "same session, different tools" note (it implied
the sandbox shares the VM's filesystem), added a sandbox-isolation Key Concept, and updated
the architecture diagram to match the new Part 6.

User experience

Before: Part 5 prints a confident weather forecast that is invented — wrong season, no
indication anything failed. Part 6 prints No report / NO_CHART and produces no chart.
An invoke_harness turn that fails mid-stream carries on as a success. Starting the chat
server without HARNESS_ARN gives a bare KeyError traceback on import.

After: Part 5 reaches a real weather site and returns real temperatures. Part 6 produces
a real data-driven HTML/CSS chart plus a markdown report. A failed turn raises with the cause
named. The chat server exits with a message telling you what to set.

Testing

  • Full live run of all 6 parts on a fresh execution role; clean teardown (harness, memory and
    role all deleted, account back to baseline).
  • Part 5 reached the live site, 9 browser tool calls, 0×403, concrete temperatures.
  • Part 6 produced a self-contained HTML/CSS chart (0 external dependencies) plus the report.
  • ruff check and ruff format --check clean on all three Python files, run the same way CI
    runs them, with the repo's pyproject.toml.
  • Verified against the current main tip: the four files are untouched upstream, and the
    change applies and lints clean on a pristine checkout.

Note on the shared file

utils/iam.py is shared by the samples in 01-features/01-harness. The addition is
Allow-only and scoped to browser ARNs, and 01-travel-agent is the only sample in the folder
that uses agentcore_browser — so it is a no-op for the others.

Checklist

  • I have reviewed the contributing guidelines
  • Add your name to CONTRIBUTORS.md
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Are you uploading a dataset?
  • Have you documented Introduction, Architecture Diagram, Prerequisites, Usage, Sample Prompts, and Clean Up steps in your example README?
  • I agree to resolve any issues created for this example in the future.
  • I have performed a self-review of this change
  • Changes have been tested
  • Changes are documented

…code-interpreter step, handle stream errors

Two of the six parts looked like they worked but silently did not.

Part 5 (browser) had no IAM permission for the automation stream. The
agentcore_browser tool opens a CDP WebSocket, which was refused with
"403 Forbidden ... not authorized to access automation stream", and the agent then
quietly fell back to guessing — a run in August produced December weather. Adds
the two ConnectBrowser*Stream actions to utils/iam.py, scoped to the browser
resource; the built-in browser is owned by the `aws` account, not the caller's, so
an ARN built from the caller's account ID would never match.

Part 6 was structurally impossible. The code interpreter runs in a separate
sandbox whose filesystem is isolated from the harness microVM in both directions,
so asking it to read /tmp/tourism_data.json from the VM and write a .png back
never worked, and two `||` fallbacks hid it. Rendering on the VM instead is not
possible either (no matplotlib, no pip/uv), and bridging a PNG's base64 back
through the model overflows the turn's token limit. The sandbox now does the
analysis as text with the data passed inline, and the VM renders a self-contained
HTML/CSS chart the same way Parts 2 and 5 produce their HTML.

Also: the stream loops checked only internalServerException, so a
validationException or runtimeClientError streamed by silently and a failed turn
looked like a success; run_command and fetch_file discarded the shell exit code;
and travel_chat/server.py raised a bare KeyError at import when HARNESS_ARN was
unset, omitted the model parameter the CLI script passes, and carried two
pre-existing ruff findings that CI flags once the file is touched.

README: the browser troubleshooting entry blamed VPC/internet reachability rather
than the IAM permission, and the "same session, different tools" note implied the
sandbox shares the VM's filesystem.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Latest scan for commit: 247c058 | Updated: 2026-08-03 14:08:06 UTC

Security Scan Results

Scan Metadata

  • Project: ASH
  • Scan executed: 2026-08-03T14:07:46+00:00
  • ASH version: 3.0.0

Summary

Scanner Results

The table below shows findings by scanner, with status based on severity thresholds and dependencies:

Column Explanations:

Severity Levels (S/C/H/M/L/I):

  • Suppressed (S): Security findings that have been explicitly suppressed/ignored and don't affect the scanner's pass/fail status
  • Critical (C): The most severe security vulnerabilities requiring immediate remediation (e.g., SQL injection, remote code execution)
  • High (H): Serious security vulnerabilities that should be addressed promptly (e.g., authentication bypasses, privilege escalation)
  • Medium (M): Moderate security risks that should be addressed in normal development cycles (e.g., weak encryption, input validation issues)
  • Low (L): Minor security concerns with limited impact (e.g., information disclosure, weak recommendations)
  • Info (I): Informational findings for awareness with minimal security risk (e.g., code quality suggestions, best practice recommendations)

Other Columns:

  • Time: Duration taken by each scanner to complete its analysis
  • Action: Total number of actionable findings at or above the configured severity threshold that require attention

Scanner Results:

  • PASSED: Scanner found no security issues at or above the configured severity threshold - code is clean for this scanner
  • FAILED: Scanner found security vulnerabilities at or above the threshold that require attention and remediation
  • MISSING: Scanner could not run because required dependencies/tools are not installed or available
  • SKIPPED: Scanner was intentionally disabled or excluded from this scan
  • ERROR: Scanner encountered an execution error and could not complete successfully

Severity Thresholds (Thresh Column):

  • CRITICAL: Only Critical severity findings cause scanner to fail
  • HIGH: High and Critical severity findings cause scanner to fail
  • MEDIUM (MED): Medium, High, and Critical severity findings cause scanner to fail
  • LOW: Low, Medium, High, and Critical severity findings cause scanner to fail
  • ALL: Any finding of any severity level causes scanner to fail

Threshold Source: Values in parentheses indicate where the threshold is configured:

  • (g) = global: Set in the global_settings section of ASH configuration
  • (c) = config: Set in the individual scanner configuration section
  • (s) = scanner: Default threshold built into the scanner itself

Statistics calculation:

  • All statistics are calculated from the final aggregated SARIF report
  • Suppressed findings are counted separately and do not contribute to actionable findings
  • Scanner status is determined by comparing actionable findings to the threshold
Scanner S C H M L I Time Action Result Thresh
bandit 0 0 0 0 0 0 519ms 0 PASSED MED (g)
cdk-nag 0 0 0 0 0 0 6.3s 0 PASSED MED (g)
cfn-nag 0 0 0 0 0 0 36ms 0 PASSED MED (g)
checkov 0 0 0 0 0 0 5.1s 0 PASSED MED (g)
detect-secrets 0 0 0 0 0 0 1.2s 0 PASSED MED (g)
grype 0 0 0 0 0 0 55.6s 0 PASSED MED (g)
npm-audit 0 0 0 0 0 0 161ms 0 PASSED MED (g)
opengrep 0 0 0 0 0 0 <1ms 0 SKIPPED MED (g)
semgrep 0 0 0 0 0 0 <1ms 0 MISSING MED (g)
syft 0 0 0 0 0 0 2.2s 0 PASSED MED (g)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant