#263: Refactor Tugboat integration

[rabbitlair]

Closes #263

Warning: Contains a breaking change, so it requires, at least, a minor version bump.

This PR refactors the Tugboat integration to use DDEV configuration as the single source of truth for all service definitions. The new implementation generates a complete config.yml using Twig templates instead of separate bash scripts.

Architecture

• New dedicated plugin: TugboatConfigPlugin handles all Tugboat logic (previously in ScaffoldInstallerPlugin)
• Template-based generation: Uses Twig template to generate config.yml file with sub-templates for each build phase in main PHP service
• DDEV-driven configuration: All service versions and types extracted from .ddev/config.yaml and docker-compose add-on files

Build Phase Templates

• config.yml.twig - Main configuration template that includes phase templates
• php-init.yml.twig - System packages, PHP extensions, Node.js installation, composer install
• php-update.yml.twig - Composer install, sync task, file permissions changes, update tasks
• php-build.yml.twig - Build tasks

Taskfile Integration

• All services: Supports tugboat:{service}:init tasks (e.g., tugboat:redis:init)
• PHP service: Supports sync:tugboat, update:tugboat, build:tugboat, online:tugboat tasks
• Custom init script: custom-init-command.sh wrapper for non-PHP service init tasks
• Task installation: Automatically installs Taskfile in init phase using version from .taskfile file

Template Customization

• Default templates: Located in vendor/lullabot/drainpipe/scaffold/tugboat/templates/
• Project overrides: Place custom templates in .tugboat/drainpipe-templates/
• Template precedence: Project templates take priority over defaults
• Custom Twig filters: Added indent filter for proper YAML formatting

Files Structure

scaffold/tugboat/
├── templates/
│   ├── config.yml.twig           # Main Tugboat config
│   ├── php-init.yml.twig         # Init phase commands
│   ├── php-update.yml.twig       # Update phase commands
│   ├── php-build.yml.twig        # Build phase commands
│   └── settings.tugboat.php.twig # Drupal settings
└── scripts/
    ├── custom-init-command.sh    # Init wrapper for non-PHP services
    └── install-mysql-client.sh   # MySQL client installer

.tugboat/ (generated)
├── config.yml                    # Generated config (overwritten on each run)
├── config.drainpipe-override.yml # User overrides (preserved)
├── drainpipe-templates/          # User template overrides (preserved)
└── scripts/                      # Copied scripts

Please, note this is a first working version. I would like to know your thoughts about the new approach - any feedback will be more than welcome. As far as I can tell, this refactor does not break backwards compatibility while adding the custom templates feature, which I think will make easier for users to customize their Tugboat integration.

Testing Instructions

First, create a new Drupal project and configure Drainpipe with Tugboat integration.

# Create a new Drupal project
mkdir testing
cd testing
composer create-project drupal/recommended-project . --ignore-platform-reqs
# Copy this Drainpipe branch to `testing/drainpipe` 

# Configure DDEV
ddev config --auto
ddev config --php-version "8.4"
ddev config --nodejs-version "24"
ddev get ddev/ddev-redis --version v1.2.0
ddev get ddev/ddev-elasticsearch --version v0.3.2
ddev start

# Configure Composer to work with Drainpipe
ddev exec --raw composer config extra.drupal-scaffold.gitignore true
ddev exec --raw composer config --json extra.drupal-scaffold.allowed-packages \[\"lullabot/drainpipe\"]
ddev exec --raw composer config --no-plugins allow-plugins.composer/installers true
ddev exec --raw composer config --no-plugins allow-plugins.drupal/core-composer-scaffold true
ddev exec --raw composer config --no-plugins allow-plugins.lullabot/drainpipe true
ddev exec --raw composer config repositories.drainpipe --json '{"type": "path", "url": "drainpipe", "options": {"symlink": false}}'
ddev exec --raw composer config minimum-stability dev

# Configure Tugboat integration
ddev exec --raw composer config extra.drainpipe --json '{"tugboat": {}}'
mkdir -p .tugboat
echo "php:" >  .tugboat/config.drainpipe-override.yml
echo "  aliases:" >> .tugboat/config.drainpipe-override.yml
echo "    - foo" >> .tugboat/config.drainpipe-override.yml
echo "  urls:" >> .tugboat/config.drainpipe-override.yml
echo "    - /" >> .tugboat/config.drainpipe-override.yml
echo "    - /?v=1" >> .tugboat/config.drainpipe-override.yml
echo "  screenshot:" >> .tugboat/config.drainpipe-override.yml
echo "    timeout: 45" >> .tugboat/config.drainpipe-override.yml
echo "  visualdiff:" >> .tugboat/config.drainpipe-override.yml
echo "    fullPage: false" >> .tugboat/config.drainpipe-override.yml

# Install Drainpipe
ddev composer require lullabot/drainpipe --with-all-dependencies

Verify the generated .tugboat/config.yml file is correct, and the services declared match the DDEV settings.

Test scenarios

1. Override a template for a PHP service's build phase

mkdir -p .tugboat/drainpipe-templates
cp vendor/lullabot/drainpipe/scaffold/tugboat/templates/php-update.yml.twig .tugboat/drainpipe-templates
# Add some new command to the file
ddev composer install

Then verify the new command is included in .tugboat/config.yml.

2. Add a custom init command for a secondary service:

echo "" >> Taskfile.yml
echo "  tugboat:redis:init:" >> Taskfile.yml
echo "    cmds:" >> Taskfile.yml
echo "      - whoami" >> Taskfile.yml
ddev composer install

Then verify the new command is included in .tugboat/config.yml.

3. Pantheon integration:

ddev exec --raw composer config extra.drainpipe.tugboat --json '{"pantheon": true}'
ddev composer install

Then verify Pantheon integration is included in .tugboat/config.yml.

More testing

Tested in a project currently using Drainpipe, so we can be confident about nothing breaking. Tested on lullabotdotcom.

[gemini-code-assist]

Summary of Changes

Hello @rabbitlair, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the Tugboat integration within the project. The primary goal is to centralize configuration management by using DDEV settings as the authoritative source for all service definitions. This is achieved through a new dedicated Composer plugin that dynamically generates the config.yml using a flexible Twig templating system, replacing previous bash scripts. The change aims to provide greater customizability and maintainability for Tugboat environments.

Highlights

• Dedicated Tugboat Plugin: Introduced TugboatConfigPlugin to centralize all Tugboat-related logic, moving it out of ScaffoldInstallerPlugin.
• Template-Driven Configuration: Switched to generating config.yml using Twig templates, with DDEV configuration as the single source of truth for service definitions.
• Modular Build Phases: Tugboat build phases (init, update, build) are now defined in separate, modular Twig sub-templates for better organization and customization.
• Enhanced Taskfile Integration: Improved support for custom Taskfile commands, allowing users to define tugboat:{service}:init, sync:tugboat, update:tugboat, build:tugboat, and online:tugboat tasks.
• Custom Template Overrides: Users can now override default Tugboat templates by placing custom Twig files in .tugboat/drainpipe-templates/.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This is a great refactoring of the Tugboat integration. Moving the logic to a dedicated plugin and adopting a template-based approach significantly improves maintainability and extensibility. The new TugboatConfigPlugin is well-structured. However, I've identified several critical regressions where conditional logic from the old shell scripts was lost in the new Twig templates, potentially breaking functionality for some project configurations (e.g., non-MariaDB databases, Memcached, Apache, Pantheon). I've also included some suggestions to improve code clarity and fix a bug in the new plugin. After addressing these points, this will be a very solid contribution.

[deviantintegral]

A nice first pass!

[deviantintegral]

👏

[rabbitlair]

After testing this branch with another real world project that has a complex Tugboat integration, I added the following changes:

• New template php-online.yml.twig to easily customize Online build phase
• Fixed a bug that prevented database service to properly run its custom init commands
• Fixed a bug when installing Taskfile to run custom init commands for all services
• Updated the MySQL repository GPG key, as the current one is expired
• Implemented support for using Percona as the Docker image for database service

[rabbitlair]

I'll be merging this one to main, in order to prepare for the major release to v4.0.0 in the next days. Please, feel free to add any comment or suggestion in this pull request, even if it's already merged.

[rabbitlair]

@YesCT, when you have a chance, please, can you approve this one if you are fine with all changes? I can not merge it until you confirm your requested changes are implemented. Thank you!

[YesCT]

I resolved my remaining comments and approved. (I did not look at the changes pushed since my last review.) I'm ok with that. If there are bugs, we can fix them up. Since Zequi said he was gonna merge, I went ahead and merged while there were no conflicts or failing tests, etc.