OpenClaw Phone Assistant

Real-time voice assistant for OpenClaw — talk to Snaps over WebRTC (browser) or phone (Twilio).

Built with pipecat and OpenAI Realtime API (or Google Gemini Live).

How it works

Browser (mic/speaker) <──WebRTC──> pipecat pipeline <──WebSocket──> OpenClaw Gateway
                                        │
Phone (Twilio) <──────────────> pipecat pipeline
                                        │
                                   OpenAI Realtime
                                   (speech-to-speech)

• You talk via browser or phone, pipecat streams audio to OpenAI Realtime for speech-to-speech processing
• The voice agent has one tool: ask_openclaw, which delegates any real task to OpenClaw's agent (Snaps) via the gateway WebSocket
• Hold music plays while OpenClaw processes the request
• If you interrupt during hold music, the agent responds to you while keeping the request running in the background — ask again and it picks up where it left off
• Conversation transcripts are synced to OpenClaw's session JSONL
• Phone calls are filtered by caller ID (configurable via ALLOWED_CALLER_NUMBERS)

Quick start

Requires Python 3.12+, uv, and a running OpenClaw gateway.

# Install dependencies
make install

# Configure environment
cp .env.example .env
# Edit .env with your API keys

# Run in browser mode
make webrtc
# Open http://localhost:7860/client

Phone call setup (Twilio)

1. Create a Twilio account

1. Sign up at twilio.com
2. From the Twilio Console dashboard, copy your Account SID and Auth Token
3. Buy a phone number: Console > Phone Numbers > Buy a Number (pick one with Voice capability)
4. Add these to your .env:

   TWILIO_ACCOUNT_SID=ACxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
   TWILIO_AUTH_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
   

2. Set up cloudflared tunnel

Twilio needs a public HTTPS URL to reach your bot. A cloudflared named tunnel gives you a stable hostname that never changes.

# One-time setup (installs cloudflared if needed, creates tunnel)
make tunnel-setup

# Route a subdomain to the tunnel
make tunnel-dns SUBDOMAIN=phone.yourdomain.com

# Add the hostname to .env
echo "TUNNEL_HOST=phone.yourdomain.com" >> .env

For quick dev/testing without a domain, use a temporary tunnel:

make tunnel-quick
# Copy the generated *.trycloudflare.com URL and use it as -x arg

3. Configure Twilio webhook

1. Go to Twilio Console > Phone Numbers > Active Numbers > click your number
2. Under Voice Configuration:
   • Set "A call comes in" to Webhook
   • URL: https://phone.yourdomain.com/ (your TUNNEL_HOST)
   • Method: HTTP POST
3. Save

4. Set caller filtering

Only allow specific numbers to call (E.164 format, comma-separated):

ALLOWED_CALLER_NUMBERS=+1234567890,+0987654321

Leave empty to allow all callers.

5. Run

# Terminal 1: Start the tunnel
make tunnel

# Terminal 2: Start the bot
make twilio

Call your Twilio number — you should hear the bot greet you.

Running as a daemon

To keep the bot running 24/7 as a systemd service:

# Install both services (bot + tunnel)
make daemon-install

# Start
make daemon-start

# Check status
make daemon-status

# View logs
make daemon-logs

# Stop
make daemon-stop

# Restart (after code changes)
make daemon-restart

Configuration

All config is via environment variables (or .env file):

Variable	Default	Description
VOICE_PROVIDER	openai	openai (Realtime API) or gemini (Gemini Live)
OPENAI_API_KEY	—	OpenAI API key (required for openai provider)
OPENAI_REALTIME_MODEL	gpt-4o-realtime-preview	OpenAI Realtime model
GOOGLE_API_KEY	—	Google API key (required for gemini provider)
OPENCLAW_GATEWAY_URL	ws://127.0.0.1:18789	OpenClaw gateway WebSocket URL
OPENCLAW_GATEWAY_TOKEN	—	Gateway auth token (from ~/.openclaw/openclaw.json)
OPENCLAW_SESSION_KEY	agent:main:main	OpenClaw session to use
OPENCLAW_TOOL_TIMEOUT	120	Seconds to wait for OpenClaw responses
TWILIO_ACCOUNT_SID	—	Twilio Account SID
TWILIO_AUTH_TOKEN	—	Twilio Auth Token
TWILIO_API_BASE_URL	api.twilio.com	Twilio API base URL for regional accounts (e.g. api.dublin.ie1.twilio.com for Ireland)
ALLOWED_CALLER_NUMBERS	—	Comma-separated E.164 numbers (empty = allow all)
TUNNEL_HOST	—	Cloudflared tunnel hostname for Twilio webhook

Makefile targets

make help            # Show all targets
make install         # Install dependencies
make webrtc          # Run in WebRTC mode (browser)
make twilio          # Run in Twilio mode (phone)
make test            # Run tests
make tunnel-setup    # One-time cloudflared tunnel creation
make tunnel-dns      # Route subdomain to tunnel
make tunnel          # Start the named tunnel
make tunnel-quick    # Start a temporary tunnel (dev)
make daemon-install  # Install systemd services
make daemon-start    # Start bot + tunnel
make daemon-stop     # Stop bot + tunnel
make daemon-restart  # Restart bot + tunnel
make daemon-status   # Show service status
make daemon-logs     # Tail service logs

Architecture

bot.py                  — Main entry point, dual transport (WebRTC/Twilio), tool handler
openclaw_client.py      — WebSocket client for OpenClaw gateway protocol v3
config.py               — Environment config
hold_music.py           — Async hold music player (injects audio frames into pipeline)
audio_debug.py          — Audio level monitoring
transcript_sync.py      — Syncs voice transcripts to OpenClaw session JSONL
providers/
  openai_rt.py          — OpenAI Realtime provider setup
  gemini_live.py        — Google Gemini Live provider setup
tests/
  test_openclaw_client.py — Integration tests for the gateway client

Tests

make test

Requires a running OpenClaw gateway.