Skip to content

Standardize local-dev example and fix library bugs#203

Open
mykola-pereyma wants to merge 3 commits intoawslabs:mainfrom
mykola-pereyma:fix/examples-lexical-graph-local-dev
Open

Standardize local-dev example and fix library bugs#203
mykola-pereyma wants to merge 3 commits intoawslabs:mainfrom
mykola-pereyma:fix/examples-lexical-graph-local-dev

Conversation

@mykola-pereyma
Copy link
Copy Markdown
Contributor

Summary

Standardize the local-dev example to match hybrid-dev patterns, fix notebook hardcoded credentials, and fix two library bugs found during testing.

Changes

Docker & Docs

  • Standardize container names (neo4j-local, pgvector-local, jupyter-local)
  • Use same service names in standard and dev compose for DNS resolution
  • Separate dev ports (7477/7690/5434/8890)
  • Remove mysql from dev compose
  • Consolidate 7 scripts → start-containers.sh/ps1 with --dev, --mac, --reset
  • Fix wrong connection strings, container names, and mount paths in troubleshooting.md
  • Fix stale model name in README
  • Add AWS Prerequisites and Configure Environment to Quick Start

Notebooks & AWS

  • Replace hardcoded aws_profile, bucket names, and Bedrock ARNs with env vars
  • Rename notebook to remove spaces in filename
  • Replace hardcoded S3 bucket with os.environ.get('S3_BUCKET_NAME')
  • Add AWS setup scripts (S3, DynamoDB, IAM, Bedrock prompts)
  • Add SYSTEM_PROMPT_ARN/USER_PROMPT_ARN to .env.template

Library Bug Fixes

  • BedrockPromptProviderConfig: _resolve_prompt_arn() produced invalid ARN when aws_region was None — now falls back to session.region_name
  • WikipediaReaderProvider: _init_reader() had NameError due to scoped import — store class reference for lazy init
  • Added unit tests for both fixes

Testing

All 6 notebooks tested end-to-end in dev mode (50/51 cells pass, 1 GitHub token skip by design). All Docker startup modes verified. All AWS scripts executed successfully.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Docker:
- Standardize container names (neo4j-local, pgvector-local, jupyter-local)
- Use same service names in standard and dev compose for DNS resolution
- Separate dev ports (7477/7690/5434/8890) to avoid conflicts with hybrid
- Remove mysql from dev compose
- Move Dockerfile to jupyter/Dockerfile.dev
- Consolidate 7 redundant scripts into start-containers.sh/ps1
  with --dev (switches compose file) and --reset (cleans extracted/)

Docs:
- Add AWS Prerequisites and Configure Environment to README Quick Start
- Update docker_services.md with new container names and dev comparison
- Fix troubleshooting.md: wrong connection strings, container names,
  volume names, mount paths throughout
- Fix README: stale model name, mount path reference
- Add SYSTEM_PROMPT_ARN/USER_PROMPT_ARN to .env.template
Notebooks:
- Rename '03-Querying with prompting' to '03-Querying-with-Prompting'
- Replace hardcoded aws_profile='padmin', bucket='ccms-prompts',
  aws_region='ap-south-1' with env vars matching hybrid-dev pattern
- Use BedrockPromptProviderConfig() with env vars (no hardcoded ARNs)
- Replace hardcoded S3 bucket in 04-Advanced-Configuration-Examples
  with f-string using os.environ.get('S3_BUCKET_NAME')

AWS scripts (copied from hybrid-dev):
- setup-bedrock-batch.sh/ps1: S3 bucket, DynamoDB, IAM roles
- create_custom_prompt.sh/ps1: Bedrock managed prompts
- create_prompt_role.sh/ps1: IAM role for Bedrock
- Prompt JSON files and IAM policy
BedrockPromptProviderConfig._resolve_prompt_arn():
- aws_region defaults to None, producing invalid ARN 'arn:aws:bedrock:None:...'
- Fix: fall back to session.region_name when aws_region is not set
- Add unit test for region fallback behavior

WikipediaReaderProvider._init_reader():
- WikipediaReader imported inside __init__ but used in _init_reader
  where it was out of scope, causing NameError
- Fix: store class as self._reader_cls for lazy initialization
- Add 4 unit tests (import error, lazy init, single creation, empty input)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant