Skip to content

feat: add local AI image generation via ComfyUI (replacing Pexels)#840

Open
elulue wants to merge 1 commit intoharry0703:mainfrom
elulue:feature/local-ai-generation
Open

feat: add local AI image generation via ComfyUI (replacing Pexels)#840
elulue wants to merge 1 commit intoharry0703:mainfrom
elulue:feature/local-ai-generation

Conversation

@elulue
Copy link

@elulue elulue commented Mar 20, 2026

Summary

This PR adds a new video_source="ai" option that generates video materials locally using ComfyUI, instead of searching Pexels/Pixabay stock footage.

  • New file: app/services/comfyui.py — ComfyUI API client with FLUX.1-schnell text-to-image workflow + Ken Burns effect (zoom/pan) for automatic image-to-video conversion
  • Modified: app/services/task.py — added elif video_source == "ai" branch in get_video_materials()
  • Modified: app/models/schema.py — documented "ai" as valid video_source value
  • Modified: config.example.toml — added ComfyUI configuration section

How it works

search_terms → enhance_prompt() → ComfyUI FLUX.1-schnell generate image
→ FFmpeg Ken Burns effect (random zoom-in/zoom-out/pan) → 5s video clip
→ repeat until audio duration is covered → return video paths (same interface as Pexels)

Benefits

  • Fully local generation, zero API costs
  • Zero copyright concerns (all AI-generated)
  • Content precisely matches the script (no keyword-to-stock-footage mismatch)
  • Works with any ComfyUI-supported image model (FLUX, SD3, etc.)
  • Configurable prompt prefix/suffix for style control
  • Non-breaking: existing Pexels/Pixabay workflows remain unchanged

Configuration

Add to config.toml:

comfyui_api_url = "http://127.0.0.1:8188"
comfyui_flux_model = "flux1-schnell.safetensors"
comfyui_flux_vae = "ae.safetensors"
comfyui_t5xxl = "t5xxl_fp16.safetensors"
comfyui_clip_l = "clip_l.safetensors"
comfyui_timeout = 120
comfyui_prompt_prefix = "Professional cinematic photo, high quality, detailed, 4K, "

Then set video_source = "ai" in WebUI or API request.

Requirements

  • A running ComfyUI instance with FLUX.1-schnell (or compatible) model loaded
  • FFmpeg installed (for Ken Burns effect conversion)

Closes #586

Add video_source="ai" option that generates video materials locally
using ComfyUI API + FLUX.1-schnell, instead of searching Pexels/Pixabay.

Changes:
- New app/services/comfyui.py: ComfyUI API client with FLUX.1-schnell
  text-to-image workflow, Ken Burns effect (zoom/pan) for image-to-video
  conversion, and configurable prompt enhancement
- Modified app/services/task.py: added "ai" branch in get_video_materials()
- Modified app/models/schema.py: documented "ai" as valid video_source
- Updated config.example.toml: added ComfyUI configuration section

Closes harry0703#586
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

老哥可以整合下支持本地sd生成图片?

1 participant