Skip to content

Commit 3bb6b77

Browse files
ashwin2002ashwin2002
committed
Adding initial version of AGENST.md and archietecture info files
Change-Id: I609613fecfd8a23bb7c012cbf8294ac3953ca9ad Reviewed-on: https://review.couchbase.org/c/CapellaRESTAPIs/+/241794 Reviewed-by: Ritesh Agarwal <ritesh.agarwal@couchbase.com> Tested-by: Ashwin <ashwin.govindarajulu@couchbase.com>
1 parent d45d92b commit 3bb6b77

File tree

6 files changed

+1070
-0
lines changed

6 files changed

+1070
-0
lines changed

‎AGENTS.md‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# CapellaRESTAPIs - Agent Documentation
2+
3+
## Project Purpose and Scope
4+
Python client library for Couchbase Capella REST APIs, used as a submodule by testrunner and TAF repositories for QA/test automation. Provides programmatic access to Capella Cloud APIs across multiple service types (dedicated, serverless, columnar, common) with dual API version support (v3 and v4).
5+
6+
**Primary Users**: Internal QE/test automation teams
7+
**Usage Pattern**: Library imported and called directly from parent repositories during test execution
8+
9+
## Core Commands
10+
11+
### Installation and Setup
12+
```bash
13+
# Development installation (required for submodule usage)
14+
pip install -e .
15+
16+
# Verify installation
17+
python -c "from capella.lib.APIRequests import APIRequests; print('OK')"
18+
```
19+
20+
### Library Verification
21+
```bash
22+
# Test core infrastructure
23+
python -c "from capella.lib.APIAuth import APIAuth; from capella.lib.APIRequests import APIRequests; print('Core OK')"
24+
25+
# Test service imports
26+
python -c "from capella.dedicated.CapellaAPI_v4 import ClusterOperationsAPIs; from capella.serverless.CapellaAPI import CapellaAPI; from capella.columnar.ColumnarAPI_v4 import ColumnarAPIs; from capella.common.CapellaAPI_v4 import OrganizationOperationsAPIs; print('Services OK')"
27+
```
28+
29+
### Operational Patterns
30+
```python
31+
# v4 API usage pattern (Bearer token)
32+
from capella.dedicated.CapellaAPI_v4 import ClusterOperationsAPIs
33+
api = ClusterOperationsAPIs(
34+
url="https://cloudapi.qe-17.sandbox.nonprod-project-avengers.com",
35+
secret=secret_key, # v3 HMAC auth
36+
access=access_key, # v3 HMAC auth
37+
bearer_token=token # v4 Bearer auth
38+
)
39+
response = api.list_clusters(organizationId, projectId)
40+
```
41+
42+
## Repo Layout
43+
44+
### Core Infrastructure
45+
- `capella/lib/` - Base components (APIAuth, APIRequests, APIExceptions)
46+
- `capella/common/` - Organization-level and shared operations
47+
- `capella/dedicated/` - Dedicated cluster operations
48+
- `capella/serverless/` - Serverless cluster operations
49+
- `capella/columnar/` - Columnar database operations
50+
- `docs/agent-context/` - Agent documentation context
51+
- `setup.py` - Package configuration
52+
53+
### File Conventions
54+
- `CapellaAPI.py` - v3 API implementations (HMAC auth)
55+
- `CapellaAPI_v4.py` - v4 API implementations (Bearer auth)
56+
- `ColumnarAPI_v4.py` - Columnar-specific v4 operations
57+
58+
## Development Patterns and Constraints
59+
60+
### Authentication Strategy
61+
- **v3 APIs**: HMAC signature using secret key, access key, timestamp
62+
- **v4 APIs**: Bearer token authentication
63+
- **Detection**: API determines auth method based on URL containing "v4"
64+
65+
### Request Patterns
66+
- Use service-specific API classes (ClusterOperationsAPIs, OrganizationOperationsAPIs)
67+
- All methods follow pattern: `operation_{name}(required_params, optional_params=None, headers=None, **kwargs)`
68+
- HTTP methods: `api_get()`, `api_post()`, `api_put()`, `api_patch()`, `api_del()`
69+
70+
### Code Style Constraints
71+
- No automated linting or formatting currently configured
72+
- Thread-safety: Lock-based JWT refresh in internal API calls
73+
- Session reuse: `network_session` maintains HTTP connections
74+
- **SSL verification disabled** across all requests (security concern)
75+
76+
### API Version Compatibility
77+
- Maintain both v3 (legacy) and v4 (current) versions when adding features
78+
- v4 preferred for new implementations
79+
- Check existing service files to determine version requirements
80+
81+
## Validation and Evidence Required Before Completion
82+
83+
### Mandatory Verification Steps
84+
1. **Import Validation**: All service classes import correctly in parent repository context
85+
2. **Authentication Test**: Successful API call using credentials provided by parent repository
86+
3. **Endpoint Verification**: Constructed API endpoints match Capella API documentation
87+
4. **Error Handling**: Custom exceptions properly raised and propagate correctly
88+
5. **Response Parsing**: JSON responses parse correctly without errors
89+
90+
### Testing Evidence Requirements
91+
- Manual API call verification for each service type (dedicated, serverless, columnar, common)
92+
- Both v3 and v4 endpoint testing when applicable
93+
- Error path verification (invalid credentials, missing resources)
94+
- Integration testing through parent repositories (testrunner, TAF)
95+
96+
### Validation Commands
97+
```bash
98+
# Manual verification pattern
99+
python -c "
100+
from capella.dedicated.CapellaAPI_v4 import ClusterOperationsAPIs
101+
api = ClusterOperationsAPIs(base_url, secret, access, bearer_token)
102+
# Perform test API call
103+
response = api.list_clusters(org_id, project_id)
104+
print('Success' if response.status_code == 200 else 'Failed')
105+
"
106+
```
107+
108+
## Security and Sensitive-Path Guidance
109+
110+
### Credential Management
111+
- **Never hardcode** API keys, secret keys, or bearer tokens in code
112+
- Credentials are **passed from parent repositories** at runtime
113+
- Store credentials in parent repository environment variables
114+
- No credential storage in CapellaRESTAPIs
115+
116+
### URL and Endpoint Security
117+
- Base URLs vary by environment (sandbox, staging, production)
118+
- Example: `https://cloudapi.qe-17.sandbox.nonprod-project-avengers.com`
119+
- Verify endpoint URLs match intended Capella environment
120+
121+
### SSL Verification Warning
122+
- **Critical Security Issue**: SSL verification (`verify=False`) is disabled throughout codebase
123+
- This creates vulnerability to man-in-middle attacks
124+
- **Immediate Action Required**: Implement proper SSL certificate handling
125+
- For development: May be acceptable temporarily
126+
- For production: SSL verification must be enabled
127+
128+
### Logging Security
129+
- DEBUG logging may include request/response content
130+
- Review logs before sharing to ensure no sensitive data exposure
131+
- Credentials should never appear in log output
132+
133+
### Git Security
134+
- Never commit environment files with actual credentials
135+
- `.gitignore` should prevent credential file commits
136+
- Review diffs carefully for accidental credential inclusion
137+
138+
### External References
139+
- Capella API documentation for endpoint verification
140+
- Security team guidance for SSL certificate management
141+
- Credential rotation policies from Capella platform
142+
143+
## Supporting Documentation
144+
145+
### Agent Context Documents
146+
- [Repo Inventory](docs/agent-context/repo-inventory.md) - Complete repository analysis
147+
- [Build and Test Matrix](docs/agent-context/build-test-matrix.md) - Command patterns and validation workflows
148+
- [Domain Glossary](docs/agent-context/domain-glossary.md) - Capella terminology and concepts
149+
150+
### Architecture Documentation
151+
- [Architecture for Agents](docs/architecture.agents.md) - System design and component relationships
152+
153+
### Operational Guidance
154+
- When adding new API operations: Follow existing method signature patterns
155+
- When creating new service types: Inherit from APIRequests and implement service-specific methods
156+
- When debugging API failures: Check exception hierarchy and API response content
157+
- When updating authentication: Verify both v3 and v4 paths as needed
158+
159+
### Integration Context
160+
- Primary integration: testrunner repository (QA test automation)
161+
- Secondary integration: TAF repository (Test Automation Framework)
162+
- Credentials flow: Parent repos → CapellaRESTAPIs → Capella Cloud APIs
163+
- No direct user interaction: Library called programmatically only

‎docs/agent-context/build-test-matrix.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
# Build and Test Matrix
2+
3+
## Current State
4+
This repository currently has no automated build, test, or validation infrastructure. The library is used as a submodule and called directly by parent repositories (testrunner, TAF).
5+
6+
## Build Commands
7+
8+
### Package Installation
9+
```bash
10+
# Development installation (editable mode)
11+
pip install -e .
12+
13+
# Standard installation
14+
pip install .
15+
```
16+
17+
### Distribution
18+
```bash
19+
# Build source distribution
20+
python setup.py sdist
21+
22+
# Build wheel distribution
23+
python setup.py bdist_wheel
24+
```
25+
26+
## Test Commands (Not Currently Implemented)
27+
28+
### Unit Tests (Recommended Addition)
29+
```bash
30+
# Install test dependencies (to be added to requirements-dev.txt)
31+
pip install pytest pytest-cov pytest-mock
32+
33+
# Run unit tests with coverage
34+
pytest tests/ --cov=capella --cov-report=html --cov-report=term
35+
36+
# Run specific test files
37+
pytest tests/test_api_auth.py -v
38+
```
39+
40+
### Integration Tests (Recommended Addition)
41+
```bash
42+
# Run integration tests (requires valid credentials)
43+
pytest tests/integration/ --skip-slow -v
44+
45+
# Run with specific environment
46+
pytest tests/integration/ --env=sandbox -v
47+
```
48+
49+
### Lint Commands (Recommended Addition)
50+
```bash
51+
# Install lint tools (to be added to requirements-dev.txt)
52+
pip install flake8 black isort mypy
53+
54+
# Code formatting
55+
black capella/
56+
57+
# Import sorting
58+
isort capella/
59+
60+
# Linting
61+
flake8 capella/
62+
63+
# Type checking
64+
mypy capella/
65+
```
66+
67+
## Validation Evidence Required Before Completion
68+
69+
### Manual Validation (Current Process)
70+
1. **Authentication verification**
71+
- Test v3 API calls with HMAC signature authentication
72+
- Test v4 API calls with Bearer token authentication
73+
- Verify credentials passed from parent repositories correctly
74+
75+
2. **Service-specific validation**
76+
- Test each service type (dedicated, serverless, columnar, common)
77+
- Verify API endpoints respond correctly
78+
- Check error handling and exception propagation
79+
80+
3. **Integration verification**
81+
- Import library in parent repositories (testrunner, TAF)
82+
- Execute API calls through parent test frameworks
83+
- Verify response parsing and error handling
84+
85+
### Recommended Automated Validation
86+
87+
#### Pre-commit Checks
88+
```bash
89+
#!/bin/bash
90+
# .git/hooks/pre-commit (recommended)
91+
92+
# Run linting
93+
flake8 capella/ || exit 1
94+
95+
# Check formatting
96+
black --check capella/ || exit 1
97+
98+
# Type checking
99+
mypy capella/ || exit 1
100+
101+
# Run unit tests
102+
pytest tests/unit/ || exit 1
103+
```
104+
105+
#### Smoke Tests (Recommended Addition)
106+
```bash
107+
# Quick sanity checks before major changes
108+
python -m pytest tests/smoke/ -v --timeout=30
109+
```
110+
111+
## Component-Specific Commands
112+
113+
### Core Infrastructure
114+
```bash
115+
# Test authentication layer
116+
python -c "from capella.lib.APIAuth import APIAuth; print('APIAuth OK')"
117+
118+
# Test request handling
119+
python -c "from capella.lib.APIRequests import APIRequests; print('APIRequests OK')"
120+
121+
# Test exceptions
122+
python -c "from capella.lib.APIExceptions import *; print('APIExceptions OK')"
123+
```
124+
125+
### Service Types
126+
```bash
127+
# Test dedicated service imports
128+
python -c "from capella.dedicated.CapellaAPI_v4 import ClusterOperationsAPIs; print('Dedicated OK')"
129+
130+
# Test serverless service imports
131+
python -c "from capella.serverless.CapellaAPI import CapellaAPI; print('Serverless OK')"
132+
133+
# Test columnar service imports
134+
python -c "from capella.columnar.ColumnarAPI_v4 import ColumnarAPIs; print('Columnar OK')"
135+
136+
# Test common service imports
137+
python -c "from capella.common.CapellaAPI_v4 import OrganizationOperationsAPIs; print('Common OK')"
138+
```
139+
140+
## Environment-Specific Configuration
141+
142+
### Development Environment
143+
```bash
144+
# Set up development environment
145+
python -m venv venv
146+
source venv/bin/activate # On Windows: venv\Scripts\activate
147+
pip install -e .[dev] # Development dependencies (to be added in setup.py)
148+
```
149+
150+
### Production Deployment
151+
```bash
152+
# Standard production installation
153+
pip install capellaApi==1.0
154+
155+
# or from specific commit/branch
156+
pip install git+https://github.com/couchbaselabs/CapellaRESTAPIs.git@master
157+
```
158+
159+
## Dependency Management (Recommended Addition)
160+
161+
### requirements.txt (Recommended)
162+
```
163+
requests>=2.25.0
164+
```
165+
166+
### requirements-dev.txt (Recommended)
167+
```
168+
requests>=2.25.0
169+
pytest>=6.0.0
170+
pytest-cov>=2.12.0
171+
pytest-mock>=3.6.0
172+
flake8>=3.9.0
173+
black>=21.5b0
174+
isort>=5.9.0
175+
mypy>=0.910
176+
```
177+
178+
## Common Issues and Solutions
179+
180+
### Import Errors
181+
```bash
182+
# Ensure editable install or correct PYTHONPATH
183+
pip install -e .
184+
# or
185+
export PYTHONPATH="${PYTHONPATH}:$(pwd)"
186+
```
187+
188+
### SSL Verification Warnings
189+
```bash
190+
# Currently SSL verification is disabled (verify=False)
191+
# This is a security concern and should be addressed
192+
# Recommended: Implement proper SSL certificate handling
193+
```
194+
195+
### Authentication Failures
196+
```bash
197+
# Verify credentials are being passed correctly
198+
# Check that environment variables are set in parent repositories
199+
# Ensure Bearer tokens are valid and not expired
200+
```
201+
202+
## Missing Infrastructure
203+
204+
### High Priority
205+
1. **Test framework setup** - pytest configuration and test structure
206+
2. **CI/CD pipeline** - GitHub Actions or equivalent for automated testing
207+
3. **Pre-commit hooks** - Automated code quality checks before commits
208+
4. **Requirements files** - Formal dependency management
209+
210+
### Medium Priority
211+
1. **Code formatting** - black and isort configuration
212+
2. **Type hints** - Add mypy for type checking
213+
3. **Documentation generation** - API documentation from docstrings
214+
4. **Security scanning** - Dependency vulnerability checks
215+
216+
### Low Priority
217+
1. **Performance benchmarks** - API response time monitoring
218+
2. **Integration test environments** - Staging sandbox configuration
219+
3. **Release automation** - Automated version tagging and publishing

0 commit comments

Comments
 (0)