Refactor: simplify options with reflection and rename project to rclone-vfs

- Implemented a tag-driven reflection system for vfsproxy.Options to automate flag registration, Caddyfile parsing, and rclone configuration.
- Renamed project from 'vfscache-proxy' to 'rclone-vfs' across module, imports, Dockerfile, and documentation.
- Added comprehensive tests for reflection-based configuration in both pkg/vfsproxy and caddy modules.
- Updated README.md to reflect the new options system and remove obsolete metadata cache references.

Author: divyam234

.goreleaser.yml

@@ -1,10 +1,11 @@
 version: 2
-project_name: vfscache_proxy
+project_name: rclone-vfs
 
 builds:
   - env:
       - CGO_ENABLED=0
     main: main.go
+    binary: rclone-vfs
     flags: -trimpath
     ldflags:
       - -s -w

Dockerfile

@@ -4,6 +4,6 @@ RUN apk add --no-cache ca-certificates tzdata
 
 ARG TARGETPLATFORM
 
-COPY $TARGETPLATFORM/vfscache_proxy /vfscache_proxy
+COPY $TARGETPLATFORM/rclone-vfs /rclone-vfs
 
-CMD ["/vfscache_proxy"]
+CMD ["/rclone-vfs"]

README.md

@@ -1,154 +1,92 @@
-# VFS Cache Proxy
+# rclone-vfs
 
 A high-performance streaming proxy server built on top of [Rclone's VFS](https://rclone.org/commands/rclone_mount/#vfs-file-caching) layer. This service allows you to stream files from any HTTP/HTTPS URL with the benefits of Rclone's smart caching, buffering, and read-ahead capabilities.
 
-It also features an internal metadata cache using `freecache` to minimize upstream requests for file size and modification times.
-
 ## Features
 
 - **Rclone VFS Integration**: Leverages Rclone's robust VFS for disk caching, sparse file support, and efficient streaming.
-- **Smart Metadata Caching**: In-memory caching of file size and modification times to reduce latency and upstream API calls.
 - **Flexible Input**: Supports passing target URLs via query parameters or Base64-encoded paths.
-- **Deduplication**: Optional query parameter stripping to cache identical files with different access tokens together.
-- **Docker Ready**: minimal Alpine-based Docker image.
+- **Deduplication**: Optional query parameter and domain stripping to maximize cache hits for mirrored content.
+- **Caddy Ready**: Includes a native Caddy module for easy integration into your web server.
+- **Docker Ready**: Minimal Alpine-based Docker image.
 
 ## Getting Started
 
 ### Prerequisites
 
-- Docker
+- Docker or Go 1.25+
 
 ### Installation & Run
 
-You can run the VFS Proxy directly using Docker. This will pull the image (if hosted) or you can build it locally.
-
-To run the server with a persistent cache directory:
+You can run `rclone-vfs` directly using Docker:
 
 ```bash
 docker run -d \
   -p 8080:8080 \
-  -v /path/to/host/cache:/app/cache \
-  vfs-proxy --cache-dir /app/cache
+  -v /path/to/host/cache:/tmp/rclone_vfs_cache \
+  ghcr.io/tgdrive/rclone-vfs --cache-dir /tmp/rclone_vfs_cache
 ```
 
-You can pass any CLI flags (see below) to the end of the docker run command.
-
 ## CLI Flags
 
+All Rclone VFS settings are supported. Below are the most common ones:
+
 | Flag | Default | Description |
 |------|---------|-------------|
 | `--port` | `8080` | Port to listen on. |
 | `--cache-dir` | System Temp | Directory to store the VFS disk cache. |
+| `--cache-mode` | `off` | VFS cache mode (`off`, `minimal`, `writes`, `full`). |
 | `--chunk-size` | `64M` | The chunk size for read requests. |
 | `--chunk-streams` | `2` | Number of parallel streams to read at once. |
 | `--max-age` | `1h` | Max age of files in the VFS cache. |
 | `--max-size` | `off` | Max total size of objects in the cache. |
-| `--strip-query` | `false` | If true, strips query parameters from the URL when generating the cache key. Useful for signed URLs. |
-| `--strip-domain` | `false` | If true, strips domain and protocol from the URL when generating the cache key. Useful for content mirrored across multiple domains. |
-| `--metadata-cache-size` | `5M` | Size of the in-memory metadata cache. |
+| `--strip-query` | `false` | If true, strips query parameters from the URL when generating the cache key. |
+| `--strip-domain` | `false` | If true, strips domain and protocol from the URL. |
+| `--shard-level` | `1` | Number of directory levels for sharding the cache. |
+
+*Run `rclone-vfs --help` to see all available flags, including advanced VFS permissions and timing settings.*
 
 ## API Endpoints
 
 ### 1. Stream via Query Parameter
-
 ```http
 GET /stream?url=https://example.com/video.mp4
 ```
 
 ### 2. Stream via Base64 Path
-
-Useful for tools or players that struggle with query parameters in URLs.
-
-1. Base64 encode your target URL (URL-safe or standard).
-   - `https://example.com/video.mp4` -> `aHR0cHM6Ly9leGFtcGxlLmNvbS92aWRlby5tcDQ`
-2. Request the path:
-
-```http
-GET /stream/aHR0cHM6Ly9leGFtcGxlLmNvbS92aWRlby5tcDQ
-```
+Useful for players that struggle with query parameters.
+1. Base64 encode your URL: `https://example.com/video.mp4` -> `aHR0cHM6Ly9leGFtcGxlLmNvbS92aWRlby5tcDQ`
+2. Request: `GET /stream/aHR0cHM6Ly9leGFtcGxlLmNvbS92aWRlby5tcDQ`
 
 ## Caddy Plugin
 
-VFS Cache Proxy can be used as a Caddy module. This allows Caddy to act as a high-performance caching reverse proxy for specific upstreams.
-
-### 1. Build with xcaddy
-
-Since this is a Go plugin, you must compile it into Caddy:
-
+Build Caddy with the module:
 ```bash
-xcaddy build \
-    --with github.com/tgdrive/vfscache-proxy/caddy
+xcaddy build --with github.com/tgdrive/rclone-vfs/caddy
 ```
 
-### 2. Caddyfile Usage
-
-#### Pattern A: Simple Reverse Proxy
-Acts as a transparent caching layer for an upstream server.
-
+### Caddyfile Usage
 ```caddyfile
 :8080 {
     vfs https://upstream.com {
         cache_dir /var/cache/vfs
+        cache_mode full
         max_age 24h
-    }
-}
-```
-*Request:* `GET /movie.mp4` -> *Fetches & Caches:* `https://upstream.com/movie.mp4`
-
-#### Pattern B: Specific Route
-Only proxy specific paths through the VFS layer.
-
-```caddyfile
-example.com {
-    route /media/* {
-        vfs https://s3.amazonaws.com/my-bucket {
-            chunk_size 128M
-            chunk_streams 4
-        }
-    }
-    
-    # Other standard Caddy logic
-    file_server browse
-}
-```
-
-#### Pattern C: Stripping Prefix (`handle_path`)
-Useful if the upstream does not expect the local path prefix.
-
-```caddyfile
-:8080 {
-    handle_path /stream/* {
-        vfs https://cdn.example.net {
-            strip_domain
-            strip_query
-        }
+        strip_query
     }
 }
 ```
 
 ### Caddyfile Directives
-
-| Directive | Description |
-|-----------|-------------|
-| `upstream` (arg) | **Required**. The base URL of the source server. |
-| `cache_dir` | Directory for the VFS disk cache. |
-| `max_age` | Max age of files in cache (e.g., `1h`, `24h`). |
-| `max_size` | Max total size of the disk cache. |
-| `chunk_size` | Chunk size for read requests (default `64M`). |
-| `chunk_streams` | Parallel download streams per request. |
-| `strip_query` | Strip query parameters for the metadata cache key. |
-| `strip_domain` | Strip domain/protocol for the metadata cache key. |
-| `metadata_cache_size` | Size of the in-memory metadata cache (default `5M`). |
-| `fs_name` | Custom name for the VFS file system. |
-| `cache_mode` | VFS cache mode (`off`, `minimal`, `writes`, `full`). |
-
----
+- `upstream` (argument): The base URL of the source server.
+- `cache_dir`: Path to disk cache.
+- `cache_mode`: `off`, `minimal`, `writes`, or `full`.
+- `max_age`, `max_size`, `chunk_size`, `chunk_streams`.
+- `strip_query`, `strip_domain`, `shard-level`.
+- `read_only`, `no_seek`, `no_checksum`, etc.
 
 ## How it Works
 
-1. **Request**: The server receives a request for a URL.
-2. **Metadata**: It checks an in-memory `freecache` for the file's size and modification time.
-   - If missing, it performs an HTTP `HEAD` (or `GET` range) request to the upstream URL and caches the result for 1 hour.
-3. **VFS Mount**: The file is virtually "mounted" using the `link` backend.
-4. **Streaming**: Rclone's VFS layer handles the reading, downloading chunks in parallel (`--chunk-streams`), and caching them to disk (`--cache-dir`).
-5. **Response**: The content is streamed to the client with support for Range requests (seeking).
+1. **VFS Mapping**: The requested URL is mapped to a unique deterministic path in a virtual rclone file system.
+2. **Streaming**: Rclone's VFS layer handles the heavy lifting—on-demand downloading, parallel chunk streaming, and local disk persistence.
+3. **Efficiency**: Range requests are fully supported, allowing clients to seek through large files without downloading the entire file.

caddy/vfs.go

@@ -15,7 +15,7 @@ import (
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
 	"go.uber.org/zap"
 
-	"github.com/tgdrive/vfscache-proxy/pkg/vfsproxy"
+	"github.com/tgdrive/rclone-vfs/pkg/vfsproxy"
 )
 
 func init() {

caddy/vfs_test.go

@@ -0,0 +1,67 @@
+package vfs
+
+import (
+	"testing"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	"github.com/tgdrive/rclone-vfs/pkg/vfsproxy"
+)
+
+func TestUnmarshalCaddyfile(t *testing.T) {
+	d := caddyfile.NewTestDispenser(`
+		vfs https://example.com {
+			cache_dir /tmp/cache
+			cache_mode full
+			max_age 24h
+			chunk_streams 4
+			strip_query
+			shard-level 3
+			read_only
+		}
+	`)
+
+	v := &VFS{
+		Options: vfsproxy.DefaultOptions(),
+	}
+
+	err := v.UnmarshalCaddyfile(d)
+	if err != nil {
+		t.Fatalf("failed to unmarshal caddyfile: %v", err)
+	}
+
+	if v.Upstream != "https://example.com" {
+		t.Errorf("expected Upstream 'https://example.com', got '%s'", v.Upstream)
+	}
+
+	// Test reflection-mapped string options
+	if v.CacheDir != "/tmp/cache" {
+		t.Errorf("expected CacheDir '/tmp/cache', got '%s'", v.CacheDir)
+	}
+	if v.CacheMode != "full" {
+		t.Errorf("expected CacheMode 'full', got '%s'", v.CacheMode)
+	}
+	if v.CacheMaxAge != "24h" {
+		t.Errorf("expected CacheMaxAge '24h', got '%s'", v.CacheMaxAge)
+	}
+
+	// Test reflection-mapped integer options
+	if v.CacheChunkStreams != 4 {
+		t.Errorf("expected CacheChunkStreams 4, got %d", v.CacheChunkStreams)
+	}
+	if v.ShardLevel != 3 {
+		t.Errorf("expected ShardLevel 3, got %d", v.ShardLevel)
+	}
+
+	// Test reflection-mapped boolean flags
+	if !v.StripQuery {
+		t.Error("expected StripQuery to be true")
+	}
+	if !v.ReadOnly {
+		t.Error("expected ReadOnly to be true")
+	}
+
+	// Test defaults for things not in the Caddyfile
+	if v.FsName != "rclone-vfs" {
+		t.Errorf("expected default FsName 'rclone-vfs', got '%s'", v.FsName)
+	}
+}

go.mod

@@ -1,4 +1,4 @@
-module github.com/tgdrive/vfscache-proxy
+module github.com/tgdrive/rclone-vfs
 
 go 1.25.6
 

main.go

@@ -11,7 +11,7 @@ import (
 	"syscall"
 	"time"
 
-	"github.com/tgdrive/vfscache-proxy/pkg/vfsproxy"
+	"github.com/tgdrive/rclone-vfs/pkg/vfsproxy"
 
 	"github.com/rclone/rclone/fs/config"
 	"github.com/spf13/pflag"

pkg/vfsproxy/options_test.go

@@ -0,0 +1,83 @@
+package vfsproxy
+
+import (
+	"testing"
+
+	"github.com/spf13/pflag"
+)
+
+func TestDefaultOptions(t *testing.T) {
+	opt := DefaultOptions()
+
+	// Test default values from tags
+	if opt.FsName != "rclone-vfs" {
+		t.Errorf("expected FsName 'rclone-vfs', got '%s'", opt.FsName)
+	}
+	if opt.ShardLevel != 1 {
+		t.Errorf("expected ShardLevel 1, got %d", opt.ShardLevel)
+	}
+
+	// Test a value that should come from rclone defaults (vfscommon.Opt)
+	// rclone default for vfs_read_chunk_size is usually "128Mi" or similar
+	if opt.CacheChunkSize == "" {
+		t.Error("expected CacheChunkSize to be populated from rclone defaults, but it's empty")
+	}
+}
+
+func TestToConfigMap(t *testing.T) {
+	opt := Options{
+		CacheMode:    "full",
+		CacheMaxAge:  "24h",
+		ReadOnly:     true,
+		ShardLevel:   5, // Has no vfs tag, should be ignored
+		StripQuery:   true, // Has no vfs tag, should be ignored
+	}
+
+	m := opt.ToConfigMap()
+
+	if m["vfs_cache_mode"] != "full" {
+		t.Errorf("expected vfs_cache_mode 'full', got '%s'", m["vfs_cache_mode"])
+	}
+	if m["vfs_cache_max_age"] != "24h" {
+		t.Errorf("expected vfs_cache_max_age '24h', got '%s'", m["vfs_cache_max_age"])
+	}
+	if m["read_only"] != "true" {
+		t.Errorf("expected read_only 'true', got '%s'", m["read_only"])
+	}
+
+	// Check that fields with vfs:"-" are NOT in the map
+	if _, exists := m["shard_level"]; exists {
+		t.Error("shard_level should not be in the config map")
+	}
+}
+
+func TestAddFlags(t *testing.T) {
+	fs := pflag.NewFlagSet("test", pflag.ContinueOnError)
+	opt := Options{
+		FsName: "test-fs",
+	}
+
+	opt.AddFlags(fs)
+
+	// Verify flags were registered
+	f := fs.Lookup("fs-name")
+	if f == nil {
+		t.Fatal("flag 'fs-name' not found")
+	}
+	if f.DefValue != "test-fs" {
+		t.Errorf("expected default value 'test-fs', got '%s'", f.DefValue)
+	}
+
+	// Test parsing a flag
+	err := fs.Parse([]string{"--fs-name", "overridden", "--shard-level", "3"})
+	if err != nil {
+		t.Fatalf("failed to parse flags: %v", err)
+	}
+
+	if opt.FsName != "overridden" {
+		t.Errorf("expected FsName 'overridden', got '%s'", opt.FsName)
+	}
+	if opt.ShardLevel != 3 {
+		t.Errorf("expected ShardLevel 3, got %d", opt.ShardLevel)
+	}
+}