Tugboat

Tugboat is a system for orchestrating virtual machines in a Kubernetes‑like manner.

Introduction

Tugboat is a VM orchestration tool that provides Kubernetes‑style functionality while avoiding the heaviness of KubeVirt and the complexity of OpenStack.

It aims to manage VMs declaratively—similar to Kubernetes—using a minimal set of components: etcd, tugboat‑apiserver, tugboat‑agent, tugboat‑runtime, tugboat‑scheduler, and tugboat‑controller‑manager.

Why Tugboat?

Existing VM orchestration systems come with significant challenges:

KubeVirt

Heavy due to running VMs on top of Kubernetes (double layering)
Complex CRDs and libvirt integration

OpenStack

Too many components for individuals or small teams
Very high learning cost
Operationally difficult Tugboat solves these problems by:
Inheriting Kubernetes design principles
Not depending on Kubernetes itself
Providing a lightweight architecture optimized for VMs
Remaining simple enough for individuals to run
Scaling to large deployments through a clean, minimal design

Key Features

Kubernetes‑compatible API manifests
- Uses familiar structures such as TypeMeta and ObjectMeta
- Can use kubectl
Declarative cluster powered by etcd
- The apiserver is stateless; etcd is the single source of truth
Lightweight control plane
- Only the apiserver and scheduler are required
Direct QEMU execution
- No libvirt; QEMU is invoked directly
Clear separation of agent and runtime responsibilities
- Similar to Kubernetes’ kubelet/runtime model
VM images as OCI artifacts
- Imagefile -> build -> push to registry -> referenced by Ship
- Supports qcow2 and raw boot disk formats
CNI support
- NetworkClass / ClusterNetworkClass based network configuration
- agent publishes plugin readiness to Node.status.cniPlugins
- scheduler filters nodes with NetworkFit
RBAC / ServiceAccount
- Role / ClusterRole, RoleBinding / ClusterRoleBinding, ServiceAccount API (authorization/v1)
- Fine-grained verb- and resource-level access control enforced in the apiserver
- Built-in roles: cluster-admin, admin, edit, view
- Opaque bearer tokens generated per ServiceAccount, stored as Secrets
CustomResourceDefinition (CRD) support
- User-defined resources with Kubernetes-like CustomResourceDefinition
- OpenAPI v3 schema validation, Namespaced/Cluster scope, and optional /status subresource
High availability design
- Apiserver can scale horizontally
- Scheduler uses Lease‑based leader election

Architecture Overview

The current crate and extension boundaries are documented in docs/architecture.md. Contributor-facing resource registration steps are documented in docs/resource-registration.md. VM image build, format, runtime support, and cache layout details are documented in docs/vm-images.md.

Mapping to Kubernetes Concepts

Kubernetes	Tugboat
Pod	Ship
ReplicaSet	ReplicaSet
Deployment	Deployment
Node	Node
Container image	VM image (OCI)
Dockerfile	Imagefile
kubelet	agent
RuntimeClass	RuntimeClass

Note: Fleet is a Tugboat-specific resource for grouping multiple Ship types that share a private network — it has no direct Kubernetes equivalent.

Manifest Examples

ShipClass

Defines a VM machine type. This is a cluster‑scoped resource.

apiVersion: v1
kind: ShipClass
metadata:
  name: lightweight
spec:
  cpu:
    architecture: x64
    cores: 2
  memory:
    size: 4Gi

Ship

Defines a VM instance. This is a namespaced resource.

apiVersion: v1
kind: Ship
metadata:
  namespace: default
  name: ship
spec:
  image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
  shipClass: lightweight
  volumes:
    - name: data-disk
      persistentVolumeClaim:
        claimName: data-disk
    - name: app-config
      configMap:
        name: app-config
    - name: app-secret
      secret:
        secretName: app-secret

Fleet

Groups multiple Ship types that share a private network. This is a namespaced resource (apps/v1).

apiVersion: apps/v1
kind: Fleet
metadata:
  namespace: default
  name: my-fleet
spec:
  networkClassName: my-network-class
  components:
    - name: frontend
      replicas: 2
      shipTemplate:
        metadata:
          labels:
            role: frontend
        spec:
          image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
          shipClass: lightweight
    - name: backend
      replicas: 3
      shipTemplate:
        metadata:
          labels:
            role: backend
        spec:
          image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
          shipClass: lightweight

CustomResourceDefinition

Tugboat supports user-defined resources via CustomResourceDefinitions, similar to Kubernetes. A CRD declares the resource group, version, kind, plural name, scope, OpenAPI v3 schema, and optional status subresource.

apiVersion: apiextensions/v1
kind: CustomResourceDefinition
metadata:
  name: databases.example.com
spec:
  group: example.com
  names:
    plural: databases
    singular: database
    kind: Database
    listKind: DatabaseList
  scope: Namespaced
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openApiV3Schema: |
          {"type":"object","required":["spec"],"properties":{"spec":{"type":"object"}}}
      subresources:
        status: {}

See docs/custom-resources.md for the full guide.

Deployment

Manages a set of identical Ships with rolling-update support. This is a namespaced resource (apps/v1).

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: default
  name: my-deployment
spec:
  replicas: 3
  selector:
    app: my-app
  shipTemplate:
    metadata:
      labels:
        app: my-app
    spec:
      image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
      shipClass: lightweight

ReplicaSet

Maintains a stable set of replica Ships. This is a namespaced resource (apps/v1).

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  namespace: default
  name: my-replicaset
spec:
  replicas: 2
  selector:
    app: my-app
  shipTemplate:
    metadata:
      labels:
        app: my-app
    spec:
      image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
      shipClass: lightweight

RuntimeClass

Declares the capabilities of the VM runtime on a node (live migration support, hotplug support). This is a cluster‑scoped resource.

apiVersion: v1
kind: RuntimeClass
metadata:
  name: standard
spec:
  liveMigration: true
  hotplug:
    cpu:
      add: true
      remove: false
    memory:
      add: true
      remove: false
    nic:
      add: true
      remove: true
    storage:
      add: true
      remove: true

A Ship can reference a RuntimeClass by name via spec.runtimeClass. The scheduler uses the RuntimeClassFit plugin to ensure Ships are only placed on nodes whose associated RuntimeClass satisfies the Ship's requirements (e.g., live migration capability).

CSI support

CSI-backed volumes can be referenced through spec.volumes[].persistentVolumeClaim. The legacy volumeClaimRef field is still accepted for backward compatibility.

Current node-side support matrix:

Block volumeMode
Filesystem volumeMode
drivers that require NodeStageVolume / NodeUnstageVolume
nodePublishSecretRef / nodeStageSecretRef
agent restart recovery from persisted publish state
explicit fs_type and volume_attributes
NodeExpandVolume / volume expansion
drivers that require controller publish context
NodeGetVolumeStats / CSI volume health + usage surfacing on PV/PVC conditions

Filesystem volumes are exposed to the guest as a 9p share. The mount tag is the Ship volume name, so both CSI Filesystem claims and projected ConfigMap / Secret volumes are passed to the guest through the same mechanism.

Control-plane storage support includes PersistentVolume, PersistentVolumeClaim, and StorageClass APIs plus dynamic CSI provisioning, managed PV cleanup, capacity-aware provisioning/expansion, filesystem claims, and CSI secret / fsType propagation in tugboat-controller-manager. Node-side support also includes controller-publish-context handling, live NodeExpandVolume (without Ship recreate) when the driver advertises it, and NodeGetVolumeStats-backed PV/PVC condition updates for CSI health and usage. The main remaining gap is richer recovery beyond persisted publish state.

Topology-aware scheduling

Tugboat supports topology-aware scheduling to ensure Ships are placed on nodes that have access to the required resources, such as storage. This is achieved through well-known labels like topology.tugboat.cloud/region and topology.tugboat.cloud/zone.

Example manifest for a Ship with node affinity:

apiVersion: v1
kind: Ship
metadata:
  name: topology-aware-ship
spec:
  image: ghcr.io/sileader/tugboat-vm-images/ubuntu:24.04
  shipClass: lightweight
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.tugboat.cloud/zone
                operator: In
                values:
                  - zone-a

See topology-aware scheduling for more details.

Volume snapshots and clones

Volume snapshots allow you to create a point-in-time copy of a volume. These snapshots can then be used to provision new volumes (restore) or to create clones of existing volumes.

Example VolumeSnapshot manifest:

apiVersion: snapshot/v1
kind: VolumeSnapshot
metadata:
  name: my-snapshot
  namespace: default
spec:
  volumeSnapshotClassName: csi-hostpath-snapclass
  source:
    persistentVolumeClaimName: data-disk

See volume snapshots for more details.

Ship snapshots

Ship snapshots capture the entire state of a VM, including its memory and the state of its volumes. This allows for full backup and restoration of a VM instance.

Example ShipSnapshot manifest:

apiVersion: snapshot/v1
kind: ShipSnapshot
metadata:
  name: my-ship-snapshot
  namespace: default
spec:
  shipName: my-ship
  includeVolumes: true

See ship snapshots for more details.

CNI status and Flannel validation

Tugboat now exposes observed CNI readiness through Node.status.cniPlugins and publishes NetworkClass.status.readyNodes / ClusterNetworkClass.status.readyNodes from tugboat-controller-manager. The scheduler's NetworkFit filter uses that status to reject nodes that do not advertise the plugins required by a Ship's requested NetworkClass / ClusterNetworkClass.

The current rollout assumes Flannel itself is installed and managed externally. For a manual multi-node validation flow:

Install the required CNI binaries (bridge, loopback, flannel, and portmap when port mappings are enabled) on each node under the configured CNI bin directory.
Bring up Flannel externally so each node has the expected runtime state (by default /run/flannel/subnet.env and /var/lib/cni/flannel).
Start tugboat-agent on each node and confirm kubectl get node -o yaml shows status.cniPlugins with the expected readiness.
Apply a ClusterNetworkClass or NetworkClass using cniPlugin: flannel and confirm its status.readyNodes contains the nodes that passed the probe.
Create Ships that reference that network class and verify they schedule only to ready nodes before performing cross-node connectivity checks.

RBAC

Tugboat provides a RBAC system. Access control is enforced in the apiserver for every request.

Detailed documentation:

Resources

Resource	API Group	Scope	Description
`ServiceAccount`	`core/v1`	Namespaced	Identity for automated processes and controllers
`Role`	`authorization/v1`	Namespaced	Permission rules scoped to a single namespace
`ClusterRole`	`authorization/v1`	Cluster	Permission rules that apply cluster-wide
`RoleBinding`	`authorization/v1`	Namespaced	Bind a `Role` or `ClusterRole` to subjects within a namespace
`ClusterRoleBinding`	`authorization/v1`	Cluster	Bind a `ClusterRole` to subjects cluster-wide

Verbs and subject types

Supported verbs: get, list, watch, create, update, patch, delete, deletecollection

Subject types: User, Group, ServiceAccount

Permission can be further narrowed to specific resource instances via resourceNames.

Built-in ClusterRoles

ClusterRole	Description
`cluster-admin`	Full access to all resources
`admin`	Full access within a namespace; cannot modify RBAC or namespace itself
`edit`	Read/write access to most namespaced resources; cannot read Secrets or modify RBAC
`view`	Read-only access to most namespaced resources

Manifest examples

ServiceAccount

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: my-service-account

Role

Grants read access to Ships within the default namespace.

apiVersion: authorization/v1
kind: Role
metadata:
  namespace: default
  name: ship-reader
rules:
  - apiGroups: [ "" ]
    resources: [ "ships" ]
    verbs: [ "get", "list", "watch" ]

ClusterRole

Grants read access to Nodes cluster-wide.

apiVersion: authorization/v1
kind: ClusterRole
metadata:
  name: node-reader
rules:
  - apiGroups: [ "" ]
    resources: [ "nodes" ]
    verbs: [ "get", "list", "watch" ]

RoleBinding

Binds ship-reader to a user and a ServiceAccount within the default namespace.

apiVersion: authorization/v1
kind: RoleBinding
metadata:
  namespace: default
  name: ship-reader-binding
roleRef:
  apiGroup: authorization
  kind: Role
  name: ship-reader
subjects:
  - kind: User
    name: alice
  - kind: ServiceAccount
    namespace: default
    name: my-service-account

ClusterRoleBinding

Grants the node-reader ClusterRole to an entire group cluster-wide.

apiVersion: authorization/v1
kind: ClusterRoleBinding
metadata:
  name: node-reader-binding
roleRef:
  apiGroup: authorization
  kind: ClusterRole
  name: node-reader
subjects:
  - kind: Group
    name: ops-team

Aggregated ClusterRole

Extends the view role with custom resource permissions.

apiVersion: authorization/v1
kind: ClusterRole
metadata:
  name: my-extension-view
  labels:
    rbac.tugboat.cloud/aggregate-to-view: "true"
rules:
  - apiGroups: ["my.example.com"]
    resources: ["myresources"]
    verbs: ["get", "list", "watch"]

OIDC RoleBinding

apiVersion: authorization/v1
kind: RoleBinding
metadata:
  name: oidc-developers-binding
  namespace: default
subjects:
  - kind: Group
    name: oidc:developers
    apiGroup: authorization/v1
    roleRef:
      kind: ClusterRole
      name: view
      apiGroup: authorization/v1

Ship with projected token

apiVersion: core/v1
kind: Ship
metadata:
  name: my-ship
spec:
  serviceAccountName: my-sa
  automountServiceAccountToken: true

Roadmap

Installation

The installer/systemd/ directory contains shell scripts that install and configure Tugboat on a Debian/Ubuntu host using systemd. Root access and a working apt-get are required.

Control plane

sudo installer/systemd/install-control-plane.sh \
    --build \
    --apiserver-host control-plane.example.com \
    --apiserver-ip 192.168.0.1

This installs etcd, tugboat-apiserver, tugboat-scheduler, and tugboat-controller-manager as systemd services. By default the API server listens on 0.0.0.0:8443 with TLS (auto-generated CA + server certificate). Pass --insecure to use plain HTTP on port 8080 instead.

Use --use-prebuilt --bin-dir <path> in place of --build to supply pre-compiled binaries.

Worker

sudo installer/systemd/install-worker.sh \
    --use-prebuilt --bin-dir /path/to/bins \
    --apiserver-url https://192.168.0.1:8443 \
    --ca-cert /etc/tugboat/pki/ca.crt \
    --service-account-token /path/to/agent-token

This installs tugboat-agent and the selected VM runtime (--runtime qemu or --runtime cloud-hypervisor). CNI plugins are installed automatically. Flannel mode is controlled by --flannel-mode static|vxlan|host-gw (default: static, which writes /run/flannel/subnet.env without running flanneld).

Uninstall

sudo installer/systemd/uninstall.sh --control-plane   # control-plane node
sudo installer/systemd/uninstall.sh --worker          # worker node
sudo installer/systemd/uninstall.sh --control-plane --worker --purge  # full cleanup

Testing the installer

The installer/systemd/test/ directory provides a Docker-based isolation environment (Ubuntu 24.04 with systemd as PID 1). Run all scenarios with:

installer/systemd/test/run-tests.sh

Contributing

Tugboat is still in an early stage, and contributions of any kind are welcome.

License

Apache License 2.0 See LICENSE for details.

Name		Name	Last commit message	Last commit date
Latest commit History 540 Commits
.github		.github
dockerfiles		dockerfiles
docs		docs
installer/systemd		installer/systemd
manifests		manifests
sample-configs		sample-configs
tests/integration		tests/integration
tugboat-agent		tugboat-agent
tugboat-apiserver		tugboat-apiserver
tugboat-cli		tugboat-cli
tugboat-client		tugboat-client
tugboat-cloud-hypervisor-runtime		tugboat-cloud-hypervisor-runtime
tugboat-cni-operator		tugboat-cni-operator
tugboat-controller-manager		tugboat-controller-manager
tugboat-csi-operator		tugboat-csi-operator
tugboat-qemu-runtime		tugboat-qemu-runtime
tugboat-resource-store		tugboat-resource-store
tugboat-resources		tugboat-resources
tugboat-runtime-common		tugboat-runtime-common
tugboat-scheduler		tugboat-scheduler
tugboat-vm-image		tugboat-vm-image
tugboat-vm-runtime-interface		tugboat-vm-runtime-interface
.dockerignore		.dockerignore
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
Imagefile		Imagefile
LICENSE		LICENSE
README.md		README.md
README_ja.md		README_ja.md
deny.toml		deny.toml
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation

Tugboat

Introduction

Why Tugboat?

KubeVirt

OpenStack

Key Features

Architecture Overview

Mapping to Kubernetes Concepts

Manifest Examples

ShipClass

Ship

Fleet

CustomResourceDefinition

Deployment

ReplicaSet

RuntimeClass

CSI support

Topology-aware scheduling

Volume snapshots and clones

Ship snapshots

CNI status and Flannel validation

RBAC

Resources

Verbs and subject types

Built-in ClusterRoles

Manifest examples

ServiceAccount

Role

ClusterRole

RoleBinding

ClusterRoleBinding

Aggregated ClusterRole

OIDC RoleBinding

Ship with projected token

Roadmap

Installation

Control plane

Worker

Uninstall

Testing the installer

Contributing

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages