Skip to content

GEMINI.md

Latest commit

 

History

History
220 lines (164 loc) · 9.75 KB

GEMINI.md

File metadata and controls

220 lines (164 loc) · 9.75 KB

Cloud Foundation Fabric (CFF)

Project Overview

Cloud Foundation Fabric is a comprehensive suite of Terraform modules and end-to-end blueprints designed for Google Cloud Platform (GCP). It serves two primary purposes:

  1. Modules: A library of composable, production-ready Terraform modules (e.g., project, net-vpc, gke-cluster).
  2. FAST (Fabric FAST): An opinionated, stage-based landing zone toolkit for bootstrapping enterprise-grade GCP organizations.

Key Components

1. Modules (/modules)

  • Philosophy: Lean, composable, and close to the underlying provider resources.
  • Structure:
    • Standardized interfaces: IAM, logging, organization policies, etc.
    • Self-contained: Dependency injection via context variables is preferred over complex remote state lookups within modules.
    • Flat: avoid using sub-modules to reduce complexity and minimize layer traversals.
    • Naming: Avoid random suffixes; use explicit prefix variables.
  • Factories: Many modules implement a data-driven "factory" pattern (often via a factories_config variable) to manage resources at scale using YAML data files. See FACTORIES.md for a comprehensive list.
    • Validation: Factory YAML files must conform to JSON schemas (typically stored in a schemas/ folder). Use a modeline (e.g., # yaml-language-server: $schema=../schemas/project.schema.json) to enable IDE validation.
  • Usage: Modules are designed to be forked/owned or referenced via Git tags (e.g., source = "github.com/...//modules/project?ref=v30.0.0").

2. FAST (/fast)

  • Purpose: Rapidly set up a secure, scalable GCP organization.
  • Architecture: Divided into sequential "stages" (0-org-setup, 1-vpcsc, 2-security, 2-networking, etc.).
  • Factories: Extensively uses YAML-based datasets and module factory patterns to drive configuration at scale, acting as a "translation machine" that expresses different architectural designs without changing the underlying stage code.

3. Tools (/tools)

  • Python-based utility scripts for documentation, linting, and CI/CD tasks.
  • Key Scripts:
    • tfdoc.py: Auto-generates input/output tables in README.md files.
    • check_boilerplate.py: Enforces license headers.
    • check_documentation.py: Verifies README consistency.
    • changelog.py: Generates CHANGELOG.md sections based on version diffs.

Development Workflow

Prerequisites

  • Terraform (or OpenTofu)
  • Python 3.10+
  • Dependencies: 
    pip install -r tests/requirements.txt
pip install -r tools/requirements.txt

Common Tasks

1. Formatting & Linting

Always format code and update documentation before committing.

# Format Terraform code (check then fix)
terraform fmt -check -recursive modules/<module-name>
terraform fmt -recursive modules/<module-name>

# Check README consistency (variables table must match variables.tf)
python3 tools/check_documentation.py modules/<module-name>

# Regenerate README variables/outputs tables when check fails
# Note: tfdoc uses special HTML comments (<!-- BEGIN TFDOC -->) in READMEs. Do not manually edit these sections.
python3 tools/tfdoc.py --replace modules/<module-name>

# YAML linting
yamllint -c .yamllint --no-warnings <yaml-files>

# License/boilerplate check
python3 tools/check_boilerplate.py --scan-files <files>

Common gotcha — unsorted variables ([SV] error): check_documentation.py requires variables in variables.tf to be in strict alphabetical order. When adding a new variable, insert it at the correct alphabetical position, not at the top of the file.

2. Testing

Our testing philosophy is simple: test to ensure the code works and does not break due to dependency changes. Example-based testing via README.md is the preferred approach.

Tests are triggered from HCL Markdown fenced code blocks using a special # tftest directive at the end of the block.

module "my-module" {
  source = "./modules/my-module"
  # ...
}
# tftest modules=1 resources=2 inventory=my-inventory.yaml
  • Inventory files (YAML): Used to assert specific values, resource counts, or outputs from the terraform plan against an expected dataset.
  • Legacy Tests: Python-based tests using pytest and tftest are supported but example-based tests should be used whenever possible.
# Run all tests
pytest tests

# Run specific module examples
pytest -k 'modules and <module-name>:' tests/examples

# Automatically generate an inventory file from a successful plan
pytest -s 'tests/examples/test_plan.py::test_example[terraform:modules/<module-name>:Heading Name:Index]'

Note: TF_PLUGIN_CACHE_DIR is recommended to speed up tests.

3. Contributing

  • Branching: Use username/feature-name.
  • Commits: Atomic commits with clear messages.
  • Docs: Do not manually edit the variables/outputs tables in READMEs; use tfdoc.py.

Adding Context Support to a Module

Several modules support symbolic variable interpolation via a context variable. This allows callers to pass symbolic references like "$project_ids:myprj" instead of raw values, which get resolved at plan time.

Pattern

1. Add a context variable in variables.tf at its alphabetical position. Use keys relevant to the module — standard keys are locations, networks, project_ids, subnets; module-specific keys may be added (e.g., kms_keys, artifact_registries, secrets):

variable "context" {
  description = "Context-specific interpolations."
  type = object({
    kms_keys    = optional(map(string), {})
    locations   = optional(map(string), {})
    networks    = optional(map(string), {})
    project_ids = optional(map(string), {})
  })
  default  = {}
  nullable = false
}

2. Build ctx and ctx_p locals in main.tf. If the module has IAM condition support, exclude condition_vars from the flattening (it is passed directly to templatestring()):

locals {
  ctx = {
    for k, v in var.context : k => {
      for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv
    } # add: if k != "condition_vars"  — only when condition_vars is a key
  }
  ctx_p      = "$"
  project_id = lookup(local.ctx.project_ids, var.project_id, var.project_id)
  region     = lookup(local.ctx.locations, var.region, var.region)
}

3. Apply lookups in resources. Three patterns:

# Simple field
project = local.project_id

# Nullable field (null must stay null, not looked up)
encryption_key_name = (
  var.encryption_key_name == null
  ? null
  : lookup(local.ctx.kms_keys, var.encryption_key_name, var.encryption_key_name)
)

# Deeply optional nested field
private_network = (
  try(var.network_config.psa_config.private_network, null) == null
  ? null
  : lookup(local.ctx.networks, var.network_config.psa_config.private_network,
      var.network_config.psa_config.private_network)
)

# Per-element list
nat_subnets = [for s in var.nat_subnets : lookup(local.ctx.subnets, s, s)]

4. Long ternaries are wrapped in parentheses with condition and branches on separate lines:

ip_address = (
  var.address == null
  ? null
  : lookup(local.ctx.addresses, var.address, var.address)
)

Tests

Add a context test alongside existing module tests:

  • tests/modules/<module_name>/tftest.yaml — declare the module path and list context: under tests:
  • tests/modules/<module_name>/context.tfvars — provide all required module variables using symbolic references; include a context block with maps that resolve them
  • tests/modules/<module_name>/context.yaml — assert resolved (concrete) values in the plan output

README example

Modify one existing README example (do not add a new one) to demonstrate context usage. The resolved values should match the existing inventory YAML so no inventory changes are needed.

Architecture & Conventions

  • Variables & Interfaces:
    • Prefer object variables (e.g., iam = { ... }) over many individual scalar variables.
    • Design compact variable spaces by leveraging Terraform's optional() function with defaults extensively.
    • Use maps instead of lists for multiple items to ensure stable keys in state and avoid for_each dynamic value issues.
  • Naming: Never use random strings for resource naming. Rely on an optional prefix variable implemented consistently across modules.
  • IAM: Implemented within resources (authoritative _binding or additive _member) via standard interfaces.
  • Outputs: Explicitly depend on internal resources to ensure proper ordering (depends_on).
  • File Structure:
    • Move away from main.tf, variables.tf, outputs.tf.
    • Use descriptive filenames: iam.tf, gcs.tf, mounts.tf.
  • Style & Formatting:
    • Line Length: Enforce a 79-character line length limit for legibility (relaxed for long resource attributes and descriptions).
    • Ternary Operators & Functions: Wrap complex ternary operators in parentheses and break lines to align ? and :. Split function calls with many arguments across multiple lines.
    • Locals Separation: Use module-level locals for values referenced directly by resources/outputs. Use block-level "private" locals prefixed with an underscore (_) for intermediate transformations.
    • Complex Transformations: Move complex data transformations in for or for_each loops to locals to keep resource blocks clean.

Preferred Workflow

  • Always break down complex requests into small, iterative tasks.
  • For each task, propose the necessary edits and explicitly wait for user confirmation or discussion before proceeding.
  • Always use the replace tool to both perform and cleanly display the proposed text modifications. Do not silently overwrite files or use shell commands for text edits.