fix(harness/01-travel-agent): unblock the browser tool, redesign the code-interpreter step, handle stream errors - #1882
Open
rmncardoso wants to merge 1 commit into
Conversation
…code-interpreter step, handle stream errors Two of the six parts looked like they worked but silently did not. Part 5 (browser) had no IAM permission for the automation stream. The agentcore_browser tool opens a CDP WebSocket, which was refused with "403 Forbidden ... not authorized to access automation stream", and the agent then quietly fell back to guessing — a run in August produced December weather. Adds the two ConnectBrowser*Stream actions to utils/iam.py, scoped to the browser resource; the built-in browser is owned by the `aws` account, not the caller's, so an ARN built from the caller's account ID would never match. Part 6 was structurally impossible. The code interpreter runs in a separate sandbox whose filesystem is isolated from the harness microVM in both directions, so asking it to read /tmp/tourism_data.json from the VM and write a .png back never worked, and two `||` fallbacks hid it. Rendering on the VM instead is not possible either (no matplotlib, no pip/uv), and bridging a PNG's base64 back through the model overflows the turn's token limit. The sandbox now does the analysis as text with the data passed inline, and the VM renders a self-contained HTML/CSS chart the same way Parts 2 and 5 produce their HTML. Also: the stream loops checked only internalServerException, so a validationException or runtimeClientError streamed by silently and a failed turn looked like a success; run_command and fetch_file discarded the shell exit code; and travel_chat/server.py raised a bare KeyError at import when HARNESS_ARN was unset, omitted the model parameter the CLI script passes, and carried two pre-existing ruff findings that CI flags once the file is touched. README: the browser troubleshooting entry blamed VPC/internet reachability rather than the IAM permission, and the "same session, different tools" note implied the sandbox shares the VM's filesystem.
|
Latest scan for commit: Security Scan ResultsScan Metadata
SummaryScanner ResultsThe table below shows findings by scanner, with status based on severity thresholds and dependencies: Column Explanations: Severity Levels (S/C/H/M/L/I):
Other Columns:
Scanner Results:
Severity Thresholds (Thresh Column):
Threshold Source: Values in parentheses indicate where the threshold is configured:
Statistics calculation:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Two of the six parts in the Travel Guide use case look like they work but silently don't,
plus some error-handling gaps. Every change below was verified with a full 6-part live run on
a fresh execution role in a real AWS account, ending in clean teardown.
Part 5 (browser) — silently returns fabricated data
The
agentcore_browsertool opens a CDP WebSocket to the browser automation stream, but theexecution role didn't allow it, so the connection was rejected:
The agent then quietly fell back to guessing — a run in August produced December
weather. Nothing in the output said the browser had failed.
Fix: add
bedrock-agentcore:ConnectBrowserAutomationStreamandConnectBrowserLiveViewStreamtoutils/iam.py, scoped to the browser resource. Thebuilt-in browser is owned by the
awsaccount(
arn:aws:bedrock-agentcore:*:aws:browser/aws.browser.v1, confirmed vialist-browsers --type SYSTEM), not the caller's, so an ARN built from the caller's accountID would never match. Action names were verified with
accessanalyzer validate-policy; thefull policy returns zero Access Analyzer
ERROR/SECURITY_WARNINGfindings.Part 6 (code interpreter) — structurally impossible as written
The code interpreter runs in a separate sandbox whose filesystem is isolated from the
harness microVM, in both directions (verified: the sandbox gets
[Errno 2] No such file or directoryfor a VM file, and a marker the sandbox writes is not visible from the VM). Theoriginal code asked the sandbox to read
/tmp/tourism_data.jsonfrom the VM and write a.pngback to it — neither crosses the boundary, so the chart/report step always failed,papered over by
|| echo 'No report'and|| echo 'NO_CHART'.Rendering on the VM instead isn't possible as written either — the VM has no matplotlib and
no pip/uv — and bridging a PNG's base64 back through the model overflows the turn's token
limit (
runtimeClientError: Model stopped generating due to maximum token limit).Redesign: the sandbox does the analysis as text, with the data passed inline (text is the
only thing that crosses the boundary), and the VM renders a self-contained HTML/CSS chart
— exactly how Parts 2, 3 and 5 already emit HTML. Live result: a 13,137-char
amsterdam_tourism.htmlwith real figures and zero external dependencies.Error handling and robustness
stream_response(and the chat server's loop) only handledinternalServerException.validationExceptionandruntimeClientErrorstreamed by silently, so a failed turnlooked like a success and a later step acted on empty output. Now all three of the stream's
error members are checked, and
BotoCoreError/ClientErrorfrom the call itself is wrapped— a failed turn stops the demo loudly.
run_command/fetch_filediscarded the shellexitCode, so a command that failedwas indistinguishable from one that succeeded. They now surface it, and
fetch_filereturns""on a missing file instead of relying on a shell fallback.travel_chat/server.py:os.environ["HARNESS_ARN"]raised a bareKeyErrorat importwhen unset — now a clear
SystemExitwith setup guidance (empty string handled too). Addedthe omitted
model=so the chat server andtravel_agent.pyuse the same model rather thana server-side default that could differ. Fixed the two pre-existing ruff findings the file
carried (
EXE001shebang-without-exec-bit,G201exc_info=True→logger.exception),which CI flags once the file is touched; both are behaviour-preserving.
README
Corrected the browser troubleshooting entry (it blamed VPC/internet reachability; the real
cause is the IAM permission), corrected the "same session, different tools" note (it implied
the sandbox shares the VM's filesystem), added a sandbox-isolation Key Concept, and updated
the architecture diagram to match the new Part 6.
User experience
Before: Part 5 prints a confident weather forecast that is invented — wrong season, no
indication anything failed. Part 6 prints
No report/NO_CHARTand produces no chart.An
invoke_harnessturn that fails mid-stream carries on as a success. Starting the chatserver without
HARNESS_ARNgives a bareKeyErrortraceback on import.After: Part 5 reaches a real weather site and returns real temperatures. Part 6 produces
a real data-driven HTML/CSS chart plus a markdown report. A failed turn raises with the cause
named. The chat server exits with a message telling you what to set.
Testing
role all deleted, account back to baseline).
ruff checkandruff format --checkclean on all three Python files, run the same way CIruns them, with the repo's
pyproject.toml.maintip: the four files are untouched upstream, and thechange applies and lints clean on a pristine checkout.
Note on the shared file
utils/iam.pyis shared by the samples in01-features/01-harness. The addition isAllow-only and scoped to browser ARNs, and
01-travel-agentis the only sample in the folderthat uses
agentcore_browser— so it is a no-op for the others.Checklist