hypersdk

Multi-cloud VM export provider layer for the hyper2kvm migration toolkit

🎯 Overview

hypersdk is a high-performance, daemon-based VM export system that provides a provider layer abstraction for multiple clouds. It handles VM discovery, export, download, and metadata operations across different cloud platforms.

Supported Providers

✅ vSphere (VMware vCenter/ESXi) - Production Ready
✅ AWS (Amazon EC2) - Production Ready
✅ Azure (Microsoft Azure VMs) - Production Ready
✅ GCP (Google Compute Engine) - Production Ready
✅ Hyper-V (Microsoft Hyper-V) - Production Ready
✅ OCI (Oracle Cloud Infrastructure) - Production Ready
✅ OpenStack (Nova/Swift) - Production Ready
✅ Alibaba Cloud (Aliyun ECS/OSS) - Production Ready
✅ Proxmox VE (Proxmox Virtual Environment) - Production Ready

Architecture

graph TB
    A[hyper2kvm Python<br/>Main Migration Orchestrator]
    B[hypersdk Go<br/>Provider Layer Abstraction]
    C[vSphere Provider<br/>Production Ready]
    D[AWS Provider<br/>Production Ready]
    E[Azure Provider<br/>Production Ready]
    F[GCP Provider<br/>Production Ready]
    G[Hyper-V Provider<br/>Production Ready]
    H[OCI Provider<br/>Production Ready]
    I[OpenStack Provider<br/>Production Ready]
    J[Alibaba Cloud<br/>Production Ready]
    K[Proxmox VE<br/>Production Ready]

    A -->|REST API| B
    B --> C
    B --> D
    B --> E
    B --> F
    B --> G
    B --> H
    B --> I
    B --> J
    B --> K

    style A fill:#4CAF50,stroke:#2E7D32,color:#fff
    style B fill:#2196F3,stroke:#1565C0,color:#fff
    style C fill:#4CAF50,stroke:#2E7D32,color:#fff
    style D fill:#4CAF50,stroke:#2E7D32,color:#fff
    style E fill:#4CAF50,stroke:#2E7D32,color:#fff
    style F fill:#4CAF50,stroke:#2E7D32,color:#fff
    style G fill:#4CAF50,stroke:#2E7D32,color:#fff
    style H fill:#4CAF50,stroke:#2E7D32,color:#fff
    style I fill:#4CAF50,stroke:#2E7D32,color:#fff
    style J fill:#4CAF50,stroke:#2E7D32,color:#fff
    style K fill:#4CAF50,stroke:#2E7D32,color:#fff

✨ Features

Core Capabilities

Multi-Cloud Support - 9 cloud providers: vSphere, AWS, Azure, GCP, Hyper-V, OCI, OpenStack, Alibaba Cloud, Proxmox VE
Direct SDK Integration - Native Go SDK clients (no external binaries)
Beautiful Terminal UI - Modern pterm-based interface with animations
REST JSON API - 51+ endpoints for complete automation
React Dashboard - Modern real-time monitoring with WebSocket updates
Libvirt Integration - Full KVM/libvirt management capabilities
Connection Pooling - Efficient connection reuse (30-40% faster)
Webhook Integration - Real-time job notifications
Schedule Persistence - SQLite-based job scheduling
OVA Format Support - Export to OVA with compression
Daemon Architecture - Background service with systemd integration
Configuration Management - YAML config files + environment variables
Concurrent Processing - Goroutine-based parallel VM exports
Resumable Downloads - Automatic retry with exponential backoff
Progress Tracking - Real-time progress bars and ETAs
Production Ready - Comprehensive error handling, logging, timeouts

Three Tools + Dashboard

Component	Purpose	Use Case
`hyperexport`	Standalone Export Tool	Interactive & scriptable VM exports with CLI flags
`hypervisord`	Background daemon	Automation, REST API, batch processing
`hyperctl`	Migration Commander	Interactive TUI migration, daemon control, job management
Web Dashboard	Browser UI	VM monitoring, console access, job management

HyperExport Features

Standalone VM Export Tool with both interactive and non-interactive modes:

Interactive Mode:

Beautiful pterm-based UI with animations
VM discovery and selection
Real-time progress tracking
Power management prompts

Command-Line Flags (Non-Interactive):

Batch Export: Export multiple VMs from a file
OVA Format: Package exports as OVA (TAR archive)
Compression: Gzip compression for OVA files (30-50% size reduction)
Verification: SHA256 checksum validation
Dry-Run: Preview exports without executing
Folder Filtering: Filter VMs by folder path
Auto Power-Off: Automatic VM shutdown before export
Quiet Mode: Minimal output for scripting
Multi-Provider: Ready for AWS, Azure, GCP, Hyper-V

Web Dashboard Features

Modern React/TypeScript Dashboard:

Real-time Monitoring - WebSocket-based live metrics updates
Interactive Charts - Historical data visualization with Recharts
Job Management - View, filter, sort, and cancel jobs
Alert System - Real-time alerts and notifications
Provider Analytics - Multi-cloud provider comparison
System Health - CPU, memory, goroutines monitoring
WebSocket Connections - Real-time client tracking
Responsive Design - Works on desktop and mobile devices
📦 Manifest Converter Tab - One-shot export and conversion to KVM format

🆕 New in v0.2.0

HyperCTL Enhancements:

Schedule Management - Create, manage cron-based scheduled exports

hyperctl schedules create daily-backup "0 2 * * *" -vm /dc/vm/prod/web01

Webhook Integration - Configure notifications for job events

hyperctl webhooks add https://hooks.slack.com/xxx job.completed job.failed

Real-Time Watching - Monitor job progress live
```
hyperctl watch <job-id>
```
Log Streaming - View and follow job logs
```
hyperctl logs -f <job-id>
```

HyperExport Daemon Mode:

Submit to Daemon - Submit jobs to daemon for centralized management
```
hyperexport --use-daemon -vm /dc/vm/web01 --watch
```

Job Monitoring - Watch and list daemon jobs

hyperexport --daemon-list running
hyperexport --daemon-watch job-12345

Schedule Creation - Create scheduled exports from CLI

hyperexport --daemon-schedule "backup:0 3 * * *" -vm /dc/vm/db01

Daemon Status - Check daemon health and metrics
```
hyperexport --daemon-status
```

Web Dashboard Manifest Converter:

Tabbed Navigation - Dashboard, Jobs, and Manifest Converter tabs
One-Shot Conversion - Export and convert to KVM in single operation
Interactive Form - Easy configuration with target format selection (QCOW2/RAW/VDI)
Real-Time Progress - Live progress bars and streaming logs
Automatic Verification - Checksum validation for data integrity
Hyper2KVM Integration - Seamless integration with conversion tool

See the New Features Guide for detailed usage and examples.

🚀 Quick Start

Installation from Source

# Clone repository
git clone https://github.com/ssahani/hypersdk
cd hypersdk

# Build binaries
go build -o hyper2kvm ./cmd/hyper2kvm
go build -o hypervisord ./cmd/hypervisord
go build -o hyperctl ./cmd/hyperctl

# Install (requires root)
sudo ./install.sh

Installation from RPM (Fedora/RHEL/CentOS)

# Install package
sudo dnf install hypersdk

# Configure
sudo vi /etc/hypervisord/config.yaml

# Start daemon
sudo systemctl start hypervisord
sudo systemctl enable hypervisord

Configuration

Option 1: Environment Variables

export GOVC_URL='https://vcenter.example.com/sdk'
export GOVC_USERNAME='[email protected]'
export GOVC_PASSWORD='your-password'
export GOVC_INSECURE=1
export DAEMON_ADDR='localhost:8080'
export LOG_LEVEL='info'

Option 2: Configuration File

Create /etc/hypervisord/config.yaml:

VCenterURL: "https://vcenter.example.com/sdk"
Username: "[email protected]"
Password: "your-password"
Insecure: true
DaemonAddr: "localhost:8080"
LogLevel: "info"
DownloadWorkers: 4

📖 Usage Examples

HyperExport - Standalone Export Tool

Interactive Mode

# Launch interactive mode with beautiful UI
./hyperexport

# Guided workflow:
# 1. Connect to vSphere
# 2. Discover and list VMs
# 3. Interactive selection with search
# 4. VM info display
# 5. Optional graceful shutdown
# 6. Real-time progress tracking
# 7. Export summary

Non-Interactive Mode (CLI Flags)

# View all available options
./hyperexport -h

# Simple export with VM name
./hyperexport -vm "/datacenter/vm/web-server-01"

# Export as compressed OVA
./hyperexport -vm myvm -format ova -compress

# Batch export from file
cat > vms.txt <<EOF
/datacenter/vm/web-01
/datacenter/vm/web-02
/datacenter/vm/db-01
EOF
./hyperexport -batch vms.txt -format ova -compress

# Export with verification (SHA256 checksums)
./hyperexport -vm myvm -verify

# Dry-run preview (no actual export)
./hyperexport -vm myvm -dry-run

# Scripted export with quiet mode
./hyperexport -vm myvm \
  -output /backup/myvm \
  -format ova \
  -compress \
  -power-off \
  -verify \
  -quiet

# Filter VMs by folder
./hyperexport -folder /Production/WebServers

# Custom parallel downloads
./hyperexport -vm myvm -parallel 8

# Show version
./hyperexport -version

Advanced Examples

# Production batch export with all features
./hyperexport \
  -batch production-vms.txt \
  -output /backup/$(date +%Y%m%d) \
  -format ova \
  -compress \
  -verify \
  -power-off \
  -parallel 6 \
  -quiet

# Quick test with dry-run
./hyperexport -folder /Test -dry-run

# Emergency backup (auto power-off)
./hyperexport -vm critical-vm \
  -format ova \
  -compress \
  -power-off \
  -verify

# Multi-provider support (coming soon)
./hyperexport -provider aws -vm i-1234567890abcdef
./hyperexport -provider azure -vm my-azure-vm
./hyperexport -provider gcp -vm my-gcp-instance

Advanced Interactive TUI Mode

HyperExport now includes an advanced Terminal User Interface (TUI) with powerful selection and filtering features:

# Launch advanced TUI mode
./hyperexport -tui
# or
./hyperexport -interactive

TUI Features:

Bulk Regex Selection (Press A): Select multiple VMs using regex patterns
- ^web-.* - All VMs starting with "web-"
- .*-prod$ - All production VMs
- db-[0-9]+ - Database VMs with numbers
Quick Filters (Number keys 1-8):
- 1 - Powered ON VMs
- 2 - Powered OFF VMs
- 3 - Linux VMs
- 4 - Windows VMs
- 5 - High CPU (8+ cores)
- 6 - High Memory (16GB+)
- 7 - Large Storage (500GB+)
- 8 - Clear all filters
Export Templates (Press t):
- Quick Export - Fast OVF without compression
- Production Backup - OVA with compression and verification
- Archive Mode - Maximum compression for long-term storage
- Development - Uncompressed for quick testing
Real-Time Export Progress:
- Live progress bars with current file download status
- Speed calculation (MB/s) and ETA estimation
- Visual status indicators (✅ ⏳ ⏸)
Keyboard Shortcuts:
- ↑/↓ or j/k - Navigate VM list
- Space - Toggle VM selection
- Enter - Confirm selection
- / - Search VMs
- s - Sort (name, size, state)
- f - Filter by power state
- o - Filter by OS
- A - Bulk regex selection
- t - Select export template
- c - Toggle compression
- v - Toggle verification
- q or Esc - Quit/Cancel
- ? - Show help

Export Validation

HyperExport includes comprehensive pre-export and post-export validation:

# Run validation checks only (no export)
./hyperexport -vm myvm -validate-only

# Validation checks include:
# - Disk space availability (with 10% overhead)
# - Output directory write permissions
# - VM power state (warns if powered on)
# - Existing export conflicts
# - Post-export file integrity
# - OVF descriptor validity
# - VMDK file presence
# - File size verification
# - Checksum validation (if -verify enabled)

Pre-Export Validation:

Disk space check (requires 10% overhead)
Permission validation
VM state verification
Existing export detection

Post-Export Validation:

OVF file integrity
Referenced VMDK files existence
File size validation
Checksum verification

Export History and Reporting

Track and analyze all exports with built-in history and reporting:

# View recent export history
./hyperexport -history
./hyperexport -history -history-limit 20

# Generate statistics report
./hyperexport -report

# Save report to file
./hyperexport -report -report-file export-report.txt

# Clear export history
./hyperexport -clear-history

History includes:

Timestamp and duration
VM name and path
Export format (OVF/OVA)
Total size and file count
Success/failure status
Error messages (if failed)
Compression and verification flags

Report statistics:

Total exports performed
Success rate
Total data exported
Average export duration
Exports by format (OVF vs OVA)
Exports by provider

History is stored in ~/.hyperexport/history.json.

Resume Capability

HyperExport supports resuming interrupted exports:

# Resume an interrupted export
./hyperexport -vm myvm -resume

# Features:
# - Checkpoint system tracks download progress
# - HTTP Range headers for resuming file downloads
# - Automatic retry with exponential backoff
# - Progress restoration from last known state

Complete Flag Reference

Flag	Type	Default	Description
`-vm`	string		VM name/path to export
`-provider`	string	vsphere	Provider type (vsphere, aws, azure, gcp)
`-output`	string	./export-	Output directory
`-format`	string	ovf	Export format: ovf or ova
`-compress`	bool	false	Enable compression for OVA
`-verify`	bool	false	Verify export with SHA256 checksums
`-dry-run`	bool	false	Preview export without executing
`-batch`	string		File with VM list (one per line)
`-filter`	string		Filter VMs by tag (key=value)
`-folder`	string		Filter VMs by folder path
`-power-off`	bool	false	Auto power off VM before export
`-parallel`	int	4	Number of parallel downloads
`-quiet`	bool	false	Minimal output for scripting
`-version`	bool	false	Show version and exit
`-interactive`	bool	false	Launch advanced TUI mode
`-tui`	bool	false	Launch advanced TUI mode (alias)
`-validate-only`	bool	false	Only run validation checks
`-resume`	bool	false	Resume interrupted export
`-history`	bool	false	Show export history
`-history-limit`	int	10	Number of recent exports to show
`-report`	bool	false	Generate export statistics report
`-report-file`	string		Save report to file (default: stdout)
`-clear-history`	bool	false	Clear export history
`-upload`	string		Upload to cloud storage (s3://, azure://, gs://, sftp://)
`-stream-upload`	bool	false	Stream export directly to cloud (no local storage)
`-keep-local`	bool	true	Keep local copy after cloud upload
`-encrypt`	bool	false	Encrypt export files
`-encrypt-method`	string	aes256	Encryption method: aes256 or gpg
`-passphrase`	string		Encryption passphrase
`-keyfile`	string		Encryption key file
`-gpg-recipient`	string		GPG recipient email for encryption
`-profile`	string		Use saved export profile
`-save-profile`	string		Save current settings as a profile
`-list-profiles`	bool	false	List available profiles
`-delete-profile`	string		Delete a saved profile
`-create-default-profiles`	bool	false	Create default profiles

Cloud Storage Integration

Upload exports directly to cloud storage:

# Upload to AWS S3
export AWS_ACCESS_KEY_ID="your-key"
export AWS_SECRET_ACCESS_KEY="your-secret"
export AWS_REGION="us-east-1"
./hyperexport -vm myvm -upload s3://my-bucket/backups/

# Upload to Azure Blob Storage
export AZURE_STORAGE_ACCOUNT="myaccount"
export AZURE_STORAGE_KEY="mykey"
./hyperexport -vm myvm -upload azure://mycontainer/backups/

# Upload to Google Cloud Storage
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
./hyperexport -vm myvm -upload gs://my-bucket/backups/

# Upload via SFTP
./hyperexport -vm myvm -upload sftp://user:pass@server/backups/

# Upload and remove local copy
./hyperexport -vm myvm -upload s3://bucket/path --keep-local=false

Encryption

Encrypt exports for security:

# AES-256 encryption with passphrase
./hyperexport -vm myvm -encrypt -passphrase "my-secret-password"

# AES-256 with key file
./hyperexport -vm myvm -encrypt -keyfile /path/to/key.txt

# GPG encryption
./hyperexport -vm myvm -encrypt -encrypt-method gpg -gpg-recipient [email protected]

# Encrypted + compressed OVA + cloud upload
./hyperexport -vm myvm \
  -format ova \
  -compress \
  -encrypt \
  -passphrase "secret" \
  -upload s3://bucket/encrypted-backups/

Export Profiles

Save and reuse export configurations:

# Create default profiles
./hyperexport -create-default-profiles

# List available profiles
./hyperexport -list-profiles

# Use a profile
./hyperexport -vm myvm -profile production-backup

# Save current settings as a profile
./hyperexport -vm myvm \
  -format ova \
  -compress \
  -verify \
  -save-profile my-custom-backup

# Delete a profile
./hyperexport -delete-profile old-profile

# Built-in profiles:
# - quick-export: Fast export without compression
# - production-backup: OVA with compression and verification
# - encrypted-backup: Encrypted backup for sensitive data
# - cloud-backup: Backup and upload to cloud storage
# - development: Quick export for development/testing

Advanced Workflows

Combine features for powerful workflows:

# Complete production backup workflow
./hyperexport -vm production-db \
  -profile production-backup \
  -encrypt -passphrase "${BACKUP_PASSWORD}" \
  -upload s3://prod-backups/$(date +%Y-%m-%d)/ \
  --keep-local=false

# Encrypted cloud backup with notifications (coming soon)
./hyperexport -vm critical-app \
  -format ova \
  -compress \
  -verify \
  -encrypt -keyfile /secure/backup.key \
  -upload azure://backups/critical/ \
  -notify-email [email protected] \
  -notify-slack https://hooks.slack.com/...

# Multi-step workflow with validation
./hyperexport -vm myvm -validate-only  # Pre-check
./hyperexport -vm myvm -profile cloud-backup  # Export and upload
./hyperexport -history | grep myvm  # Verify in history

Output Examples

Interactive Mode:

╔═══════════════════════════════════════╗
║          HYPEREXPORT v0.2.0          ║
╚═══════════════════════════════════════╝

✓ Connected to vSphere successfully!
✓ Found 15 virtual machines

Select a VM to export:
> web-server-01
  web-server-02
  db-server-01
  ...

┌─────────────────┬──────────────────┐
│ Property        │ Value            │
├─────────────────┼──────────────────┤
│ Name            │ web-server-01    │
│ Power State     │ ● poweredOn      │
│ Guest OS        │ Ubuntu Linux 64  │
│ Memory          │ 8192 MB          │
│ CPUs            │ 4                │
│ Storage         │ 100.0 GiB        │
└─────────────────┴──────────────────┘

✓ Export Completed Successfully!

Quiet Mode:

$ ./hyperexport -vm myvm -quiet
success: myvm exported to ./export-myvm (25.3 GiB)

$ ./hyperexport -batch vms.txt -quiet
batch-summary: total=5 success=5 failed=0

Daemon Mode

# Start daemon
./hypervisord --config /etc/hypervisord/config.yaml

# Or with flags
./hypervisord --addr localhost:8080 --log-level debug

HyperCTL - Interactive Migration Commander

Interactive TUI Mode - Modern terminal user interface with advanced features:

# Launch interactive TUI
./hyperctl

# Or run against daemon
./hyperctl --daemon-url http://localhost:8080

🎯 TUI Features

Advanced Selection & Filtering:

Bulk Regex Selection (Press A): Select multiple VMs using regex patterns
- ^web-.* - All VMs starting with "web-"
- .*-prod$ - All production VMs
- (db|database) - VMs containing "db" or "database"
- Live preview showing matching VMs
Quick Filters (Number keys 1-8):
- 1 - Powered ON VMs
- 2 - Powered OFF VMs
- 3 - Linux VMs
- 4 - Windows VMs
- 5 - High CPU (8+ cores)
- 6 - High Memory (16GB+)
- 7 - Large Storage (500GB+)
- 0 - Clear all filters
Export Templates (Press t):
- Quick Export - Fast OVF without compression
- Production Backup - OVA with compression and verification
- Development - Speed-optimized OVF
- Archive - Compressed OVA for storage

Real-Time Export Progress:

Live progress bars with current file download status
Speed calculation (MB/s) and ETA estimation
Elapsed time tracking
Current file name display
Overall VM export progress (X/Y completed)
Visual status indicators (✅ ⏳ ⏸)

Search & Sort:

/ - Search by name, path, or OS
s - Cycle sort modes (name, CPU, memory, storage, power)
f - Toggle power state filter
c - Clear all filters

VM Operations:

Space - Toggle VM selection
a - Select all visible VMs
n - Deselect all
d/i - View detailed VM information
r - Toggle dry-run mode (preview without executing)
h/? - Toggle comprehensive help panel

Additional Features:

Enhanced status bar showing connection mode and active filters
Real-time statistics dashboard (total CPUs, memory, storage)
Disk space validation before export
Export preview with size calculations
Compact multi-column VM display
Keyboard shortcut help with categorized commands

CLI Mode (Non-Interactive)

# Submit single VM export
hyperctl submit -vm "/datacenter/vm/my-vm" -output "/tmp/export"

# Submit from YAML file
hyperctl submit -file job.yaml

# Query all jobs
hyperctl query -all

# Get daemon status
hyperctl status

# Cancel a job
hyperctl cancel -id <job-id>

Job Definition (YAML)

Single VM (job.yaml):

name: "vm-export-1"
vm_path: "/datacenter/vm/my-vm"
output_path: "/tmp/export"
options:
  parallel_downloads: 4
  remove_cdrom: true

Batch (batch.yaml):

jobs:
  - name: "vm-1"
    vm_path: "/datacenter/vm/vm-1"
    output_path: "/tmp/export-1"

  - name: "vm-2"
    vm_path: "/datacenter/vm/vm-2"
    output_path: "/tmp/export-2"

Web Dashboard

Access the React dashboard in your browser:

# React dashboard
http://localhost:8080/web/dashboard/

# or simply
http://localhost:8080/

Dashboard Features:

Real-time metrics with WebSocket updates
Interactive charts - Jobs, resources, and provider analytics
Job management - Table with sorting, filtering, and cancel actions
Alert system - Real-time notifications for issues
Provider comparison - Multi-cloud analytics
System monitoring - Memory, CPU, goroutines, uptime
WebSocket clients - Track active connections
Responsive design - Mobile-friendly interface

REST API

# Health check
curl http://localhost:8080/health

# Daemon status
curl http://localhost:8080/status

# List libvirt VMs
curl http://localhost:8080/libvirt/domains

# Submit job
curl -X POST http://localhost:8080/jobs/submit \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-export",
    "vm_path": "/datacenter/vm/my-vm",
    "output_path": "/tmp/export"
  }'

# Query jobs (GET or POST)
curl http://localhost:8080/jobs/query?all=true

# Get job progress
curl http://localhost:8080/jobs/progress/<job-id>

# VM operations
curl -X POST http://localhost:8080/libvirt/domain/start \
  -d '{"name": "my-vm"}'

# Create snapshot
curl -X POST http://localhost:8080/libvirt/snapshot/create \
  -d '{"name": "my-vm", "snapshot_name": "snap1"}'

# Get console info
curl http://localhost:8080/console/info?name=my-vm

51+ API Endpoints Available:

Core services (health, status, capabilities)
Job management (submit, query, cancel, progress, logs, ETA)
VMware VM operations (list, info, shutdown, poweroff)
Libvirt domains (list, start, stop, reboot, pause, resume)
Console access (VNC, serial, screenshots)
Snapshots (create, revert, delete, list)
Networks (list, create, delete, start, stop)
Volumes (create, clone, resize, delete, upload)
ISO management (list, upload, attach, detach)
Backups (create, restore, verify, list, delete)
Monitoring (stats, CPU, memory, disk, network)
Batch operations (bulk start/stop/reboot/snapshot/delete)
Cloning & templates (clone, deploy, export)
Workflows (conversion, status)
Schedules (list, create, update, delete, enable/disable)
Webhooks (list, add, delete, test)

See API documentation for complete reference.

Python Integration

import requests
import time

BASE_URL = "http://localhost:8080"

# Submit job
response = requests.post(f"{BASE_URL}/jobs/submit", json={
    "name": "python-export",
    "vm_path": "/datacenter/vm/my-vm",
    "output_path": "/tmp/export",
    "options": {
        "parallel_downloads": 4,
        "remove_cdrom": True
    }
})

job_id = response.json()["job_ids"][0]
print(f"Job submitted: {job_id}")

# Poll for completion
while True:
    response = requests.post(f"{BASE_URL}/jobs/query",
        json={"job_ids": [job_id]})

    job = response.json()["jobs"][0]
    status = job["status"]

    if status == "completed":
        print(f"✅ Export completed: {job['result']['ovf_path']}")
        break
    elif status == "failed":
        print(f"❌ Export failed: {job.get('error')}")
        break

    if job.get("progress"):
        progress = job["progress"]
        print(f"⏳ {progress['phase']}: {progress['percent_complete']:.1f}%")

    time.sleep(5)

🏗️ Architecture

Directory Structure

hypersdk/
├── cmd/
│   ├── hyper2kvm/          # Interactive CLI
│   ├── hypervisord/        # Daemon service
│   └── hyperctl/           # Control CLI
│
├── providers/
│   ├── vsphere/            # vSphere provider (ready)
│   │   ├── client.go       # Connection management
│   │   ├── export.go       # Export logic
│   │   ├── vm_operations.go# VM management
│   │   └── types.go        # Data structures
│   ├── aws/                # AWS provider (planned)
│   ├── azure/              # Azure provider (planned)
│   └── common/             # Shared provider code
│
├── daemon/
│   ├── api/                # REST API server (51+ endpoints)
│   │   ├── server.go       # Base API server
│   │   ├── enhanced_server.go  # Enhanced server with all routes
│   │   ├── *_handlers.go   # Endpoint handlers (24 files)
│   │   ├── libvirt_handlers.go # Libvirt VM management
│   │   ├── console_handlers.go # Console access (VNC/Serial)
│   │   ├── snapshot_handlers.go # Snapshot operations
│   │   ├── iso_handlers.go  # ISO management
│   │   ├── backup_handlers.go # Backup/restore
│   │   ├── progress_handlers.go # Job progress tracking
│   │   └── ... (17 more handler files)
│   ├── jobs/               # Job management
│   ├── models/             # Data models
│   └── dashboard/          # Web dashboard
│       └── static/         # Static HTML/CSS/JS files
│
├── web/
│   └── dashboard/          # Dashboard web UI
│       ├── index.html      # Main dashboard
│       └── vm-console.html # Console viewer
│
├── config/                 # Configuration management
├── logger/                 # Structured logging
├── progress/               # Progress tracking
│
├── docs/                   # Complete documentation
│   ├── API_ENDPOINTS.md    # Full API reference
│   ├── API_REFERENCE_NEW_FEATURES.md # Phase 2 features
│   ├── getting-started.md  # Quick start guide
│   └── project-summary.md  # Architecture overview
│
├── scripts/                # Utility scripts
│   └── test-api.sh         # API testing script
│
├── config.yaml.example     # Example configuration
├── hypervisord.service     # Systemd unit file
├── hypersdk.spec           # RPM spec file
└── install.sh              # Installation script

Technology Stack

Language: Go 1.24+
vSphere SDK: govmomi v0.52.0
AWS SDK: AWS SDK for Go v2
Azure SDK: Azure SDK for Go
GCP SDK: Google Cloud Client Libraries
Hyper-V: PowerShell via WinRM
Libvirt: virsh command-line integration
Terminal UI: pterm v0.12.82
Progress Bars: progressbar v3.19.0
Config: YAML (gopkg.in/yaml.v3)
HTTP Server: Go standard library
React Dashboard: React 18 + TypeScript + Vite + Recharts
Legacy Dashboard: Pure HTML/CSS/JavaScript

Concurrency Model

Goroutines for job execution (not threads/forks)
Channels for communication
Mutexes for shared state protection
Worker pools for parallel downloads
Context for cancellation and timeouts

📊 Performance

Connection Time: ~1-2 seconds
VM Discovery: ~1 second (200+ VMs)
Download Speed: Network-limited
Memory Usage: Low (streaming downloads)
Concurrent Jobs: Unlimited (goroutine-based)
API Response: < 50ms

🔒 Security

Disable Web Dashboard (API-Only Mode)

For security-conscious deployments, embedded systems, or pure automation use cases, you can disable the web dashboard:

# Option 1: CLI flag
./hypervisord --disable-web

# Option 2: Config file
# config.yaml
web:
  disabled: true

Benefits:

Reduced attack surface (no web UI)
Lower resource usage
Perfect for headless/embedded systems
API-only automation environments

When disabled, the daemon runs in API-only mode with all 51+ REST endpoints still available via CLI or scripts.

Systemd Hardening

The systemd service includes security hardening:

NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ProtectKernelTunables=true

Credentials

Store credentials in config file: /etc/hypervisord/config.yaml
Protect config file: chmod 600 /etc/hypervisord/config.yaml
Or use environment variables for temporary use
Never commit credentials to git

🛠️ Development

Build

# Build all binaries
go build -o hyper2kvm ./cmd/hyper2kvm
go build -o hypervisord ./cmd/hypervisord
go build -o hyperctl ./cmd/hyperctl

Test

Comprehensive Test Coverage: 584+ test functions across all packages

# Run all tests
go test ./...

# Run tests with coverage report
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out

# Run specific package tests
go test ./daemon/api          # API handler tests (38 test files)
go test ./daemon/audit         # Audit logging tests
go test ./daemon/auth          # Authentication tests
go test ./providers/vsphere    # vSphere provider tests

# Run tests with verbose output
go test -v ./daemon/api

# Run specific test functions
go test -run TestHandleListUsers ./daemon/api
go test -run TestHandleSecurity ./daemon/api

# Generate coverage summary
go test -cover ./daemon/api

Test Coverage Highlights:

✅ API Handlers: 584 test functions, 27 handlers at 100% coverage
✅ Security & Compliance: Encryption, audit logging, compliance frameworks
✅ User Management: RBAC, API keys, session management
✅ Cloud Providers: AWS, Azure, GCP, vSphere, Hyper-V, OCI, OpenStack, Alibaba, Proxmox
✅ Infrastructure: Libvirt, monitoring, networking, storage
✅ Utilities: Logger (96.7%), retry logic, manifest parsing

Test Patterns Implemented:

HTTP method validation
Invalid JSON handling
Success path testing
Error path coverage
Table-driven tests
Concurrent operations testing

Format

# Format code
go fmt ./...

# Vet code
go vet ./...

Dependencies

# Download dependencies
go mod download

# Tidy dependencies
go mod tidy

📚 Documentation

Getting Started

Getting Started Guide - Quick start tutorial with dashboard access
Documentation Index - Complete documentation catalog
Project Summary - Architecture and design overview

API Reference

API Endpoints - Complete API reference (51+ endpoints)
API Reference - New Features - Phase 2 features documentation
General API Documentation - API usage guide

Testing & Implementation

Dashboard Testing Report - Comprehensive endpoint testing results
Dashboard Implementation - Implementation details and features
Test Results - Test coverage and results
Test API Script - Automated API testing

Features & Guides

Interactive Mode Guide - Interactive TUI usage
VM Export Guide - Step-by-step export procedures
Integration Guide - Integrating HyperSDK into workflows

🤝 Contributing

Contributions welcome! Please:

Fork the repository
Create a feature branch
Make your changes
Add tests
Submit a pull request

📄 License

LGPL-3.0-or-later - see LICENSE file for details

👤 Author

Susant Sahani [email protected]

🔗 Related Projects

hyper2kvm - Python migration orchestrator
govmomi - VMware vSphere Go SDK
pterm - Modern terminal UI library

🎉 Acknowledgments

Built with ❤️ using:

govmomi - VMware SDK
pterm - Terminal UI
progressbar - Progress bars
Go standard library

Made by Susant Sahani Part of the hyper2kvm project family

Name		Name	Last commit message	Last commit date
Latest commit History 220 Commits
.github		.github
cmd		cmd
config		config
daemon		daemon
demos		demos
dist		dist
docs		docs
examples		examples
logger		logger
man/man1		man/man1
manifest		manifest
monitoring		monitoring
network		network
packaging		packaging
progress		progress
providers		providers
retry		retry
scripts		scripts
systemd		systemd
web		web
.dockerignore		.dockerignore
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
SECURITY.md		SECURITY.md
build.sh		build.sh
config.example.yaml		config.example.yaml
go.mod		go.mod
go.sum		go.sum
hypersdk.spec		hypersdk.spec
install.sh		install.sh
migration_orchestrator_example		migration_orchestrator_example
network-monitor-demo		network-monitor-demo
test-dashboard-endpoints.sh		test-dashboard-endpoints.sh
test_rpmbuild.sh		test_rpmbuild.sh

License

ssahani/hypersdk

Folders and files

Latest commit

History

Repository files navigation