README.md.jinja2

<!-- Note: This file is a jinja2 template of a Markdown file.                -->
<!--
{% set project_name='comfy-catapult' %}
{% set project_name_proper='Comfy Catapult' %}
{% set pypi_project_name='comfy_catapult' %}
{% set last_release=shell('tomlq -r -e \'.["tool"]["comfy_catapult-project-metadata"]["last_release"]\' pyproject.toml',
                                   include_args=False) | trim %}
{% set last_stable_release=shell('tomlq -r -e \'.["tool"]["comfy_catapult-project-metadata"]["last_stable_release"]\' pyproject.toml',
                                 include_args=False) | trim %}
{% set tested_python_versions=shell('python -m yq -r -c \'[.jobs["build-and-test"]["strategy"]["matrix"]["python-version"][].gh] | join(", ")\' .github/workflows/build-and-test.yml',
                                 include_args=False) | trim %}
{% set badge_color='0A1E1E' %}
-->

# <div align="center">![{{ project_name_proper }}][1]</div>

<div align="center">
<!-- Icons from https://lucide.dev/icons/users -->
<!-- Icons from https://lucide.dev/icons/laptop-minimal -->

![**Audience:** Developers][2] ![**Platform:** Linux][3]

</div>

<p align="center">
  <strong>
    <a href="https://github.com/realazthat/{{project_name}}">🏠Home</a>
    &nbsp;&bull;&nbsp;
    <a href="#-features">🎇Features</a>
    &nbsp;&bull;&nbsp;
    <a href="#-installation">🔨Installation</a>
    &nbsp;&bull;&nbsp;
    <a href="#-usage">🚜Usage</a>
    &nbsp;&bull;&nbsp;
    <a href="#-documentation">📘Documentation</a>
    &nbsp;&bull;&nbsp;
    <a href="#-api">🤖API</a>
  </strong>
</p>

<p align="center">
  <strong>
    <a href="#-requirements">✅Requirements</a>
    &nbsp;&bull;&nbsp;
    <a href="#-command-line-options">💻CLI</a>
    &nbsp;&bull;&nbsp;
    <a href="#-docker-image">🐳Docker</a>
    &nbsp;&bull;&nbsp;
    <a href="#-limitations">🚸Limitations</a>
  </strong>
</p>

<div align="center">

![Top language][4] ![GitHub License][5] [![PyPI - Version][6]][7]
[![Python Version][8]][7]

**Python library to programmatically schedule ComfyUI workflows via the ComfyUI
API**

</div>

---

<div align="center">

| Branch        | Build Status                | Commits Since             | Last Commit        |
| ------------- | --------------------------- | ------------------------- | ------------------ |
| [Master][9]   | [![Build and Test][10]][11] | [![since tagged][12]][13] | ![last commit][14] |
| [Develop][15] | [![Build and Test][16]][17] | [![since tagged][18]][19] | ![last commit][20] |

</div>

```
ComfyUI API Endpoint <| <=  Comfy Catapult <=> HTTP Server <| <=  Public users
                     <|                                    <|
                     <|         Your python program        <| Your Webui/JS frontend
                     <|                                    <|
                     <|           Your workflows           <|
                     <|          Your HTTP server          <|
```

## What is it?

Comfy Catapult is a library for scheduling and running ComfyUI workflows from a
Python program, via the existing API endpoint. ComfyUI typically works by
hosting this API endpoint for its user interface.

This makes it easier for you to make workflows via the UI, and then use it from
a program.

## 🎇 Features

- ComfyUI API client
  ({{path('./comfy_catapult/api_client_base.py', text='interface', link='md')}},
  {{path('./comfy_catapult/api_client.py', text='implementation', link='md')}}).
- ComfyUI Workflow scheduler
  ({{path('./comfy_catapult/catapult_base.py', text='interface', link='md')}},
  {{path('./comfy_catapult/catapult.py', text='implementation', link='md')}}).
- ComfyUI API Pydantic Schema
  ({{path('./comfy_catapult/comfy_schema.py', link='md')}}).
- Helpers to handle uploading and downloading files to/from ComfyUI.
- Simple CLI to execute workflows.

## 🔨 Installation

```bash
# Inside your environment:

# From pypi:
pip install comfy_catapult

# From git:
pip install git+https://github.com/realazthat/{{project_name}}.git@v{{last_stable_release}}
```

## 🚜 Usage

## Related Projects

| Project                           | ComfyUI API Wrapper | Outsource Backend | Distribute Execution | Wrap Workflow | Studio |
| --------------------------------- | ------------------- | ----------------- | -------------------- | ------------- | ------ |
| [CushyStudio][21]                 | ?                   | ?                 | ?                    | ?             | Yes    |
| [ComfyUI-Serving-Toolkit][22]     | X                   | ?                 | ?                    | Yes           | ?      |
| [ComfyUI_NetDist][23]             | X                   | ?                 | Yes                  | ?             | ?      |
| [ComfyUI script_examples][24]     | Yes                 | No                | No                   | No            | No     |
| [comfyui-python-api][25]          | ?                   | ?                 | ?                    | Yes           | ?      |
| [comfyui-deploy][26]              | ?                   | ?                 | ?                    | Yes           | ?      |
| [ComfyUI-to-Python-Extension][27] | ?                   | ?                 | ?                    | Yes           | ?      |
| [ComfyScript][28]                 | ?                   | ?                 | ?                    | Yes           | ?      |
| [hordelib][29]                    | ?                   | Yes               | ?                    | ?             | ?      |
| [comfyui-cloud][30]               | ?                   | Yes               | ?                    | ?             | ?      |
| [comfy_runner][31]                | ?                   | ?                 | ?                    | ?             | ?      |
| [ComfyUI-ComfyRun][32]            | ?                   | ?                 | ?                    | ?             | ?      |

## 📘 Documentation

### Scheduling a Job

From
[`{{ path('comfy_catapult/catapult_base.py') }}`](comfy_catapult/catapult_base.py):

`{{ pysignature('comfy_catapult/catapult_base.py', symbol='ComfyCatapultBase.Catapult', backtickify='py') }}`

### Example usage:

From
[`{{ path('examples/sdxlturbo_example_catapulter.py') }}`](examples/sdxlturbo_example_catapulter.py):

`{{ pysnippet('examples/sdxlturbo_example_catapulter.py', symbol='ExampleWorkflowInfo', backtickify='py') }}`

`{{ pysnippet('examples/sdxlturbo_example_catapulter.py', symbol='RunExampleWorkflow', backtickify='py') }}`

### Exporting workflows in the API json format

In ComfyUI web interface:

1. Open settings (gear box in the corner).
2. Enable the ability to export in the API format, `Enable Dev mode Options`.
3. Click new menu item `Save (API format)`.

![ComfyUI API format export instructions](.github/comfy-export-instructions.png)

### Example workflow: Prepare ComfyUI

**If you don't want to try the example workflow, you can skip this section.**

You need to get `sd_xl_turbo_1.0_fp16.safetensors` into the ComfyUI model
directory.

Hugging Face page:
[huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors](https://huggingface.co/stabilityai/sdxl-turbo/blob/main/sd_xl_turbo_1.0_fp16.safetensors).

Direct download link:
[huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors](huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors).

### Download the example workflow, and export it in the API format

**This is optional, you can use the example workflow in `test_data/` instead and
skip this step.**

```bash
# Download the workflow:
wget https://github.com/comfyanonymous/ComfyUI_examples/raw/master/sdturbo/sdxlturbo_example.png

# 1. Open the Workflow in ComfyUI and export it. AFAIK there isn't a nice way
# to automated this right now.
#
# 2, Save to `./sdxlturbo_example_api.json`.
#
# Or just use `test_data/sdxlturbo_example_api.json`.
```

### Run the examples

```bash


# If you set this environment variable, you don't have to specify it as an
# argument.
export COMFY_API_URL=http://127.0.0.1:8188
# Note, in WSL2 you may have to use the IP of the host to connect to ComfyUI.


python -m comfy_catapult.examples.sdxlturbo_example_catapulter \
  --api_workflow_json_path "$PWD/sdxlturbo_example_api.json" \
  --tmp_path "$PWD/.deleteme/tmp/" \
  --output_path "$PWD/.deleteme/output.png" \
  --positive_prompt "amazing cloudscape, towering clouds, thunderstorm, awe" \
  --negative_prompt "dull, blurry, nsfw"

# Optional if you don't want to set the environment variable:
#   --comfy_api_url "..."

# Done! Now $PWD/.deleteme/output.png should contain the output image.

# Some other examples:
python -m comfy_catapult.examples.add_a_node
python -m comfy_catapult.examples.using_pydantic

```

### 🤖 API

- Examine {{path('./examples/sdxlturbo_example_catapulter.py', link='md')}} to
  see how to use the main `ComfyCatapult` library.
- Examine {{path('./test_data/sdxlturbo_example_api.json', link='md')}} to see
  the API format. This will be necessary in order to programmatically set the
  proper inputs for the workflow.
  - (Optional) See {{path('./examples/using_pydantic.py', link='md')}} for how
    to parse the API format into the Pydantic models schema for easier
    navigation.
  - (Optional) See {{path('./examples/add_a_node.py', link='md')}} for how to
    add a new node to a workflow. This is useful when you need to add nodes at
    runtime (such as adding a bunch of LoadImage nodes).
- See {{path('./comfy_catapult/catapult_base.py', link='md')}} for the main
  library interface.
- (Optional) See {{path('./comfy_catapult/catapult.py', link='md')}} for the
  main library implementation.
- (Optional) See {{path('./comfy_catapult/api_client_base.py', link='md')}} for
  the direct ComfyUI API endpoint client library interface; you don't need to
  use this usually.
- (Optional) For those who want to do use the raw API themselves and learn how
  it works: Examine {{path('./comfy_catapult/api_client.py', link='md')}} to see
  the API client implementation if you want to directly interface with ComfyUI
  endpoints yourself.
  - (Optional) Also see
    [ComfyUI/server.py](https://github.com/comfyanonymous/ComfyUI/blob/977eda19a6471fbff253dc92c3c2f1a4a67b1793/server.py#L99)
    (pinned to a specific commit) for the server `@routes` endpoint
    implementations.

### Parsing the API format into the Pydantic models schema for easier navigation

From {{ path('./examples/using_pydantic.py', link='md') }}:

`{{ snippet('examples/using_pydantic.py', start='# SNIPPETSTART', end='# SNIPPETEND', backtickify='py') }}`

### Adding a new node to a workflow

From {{ path('examples/add_a_node.py', link='md') }}:

`{{ snippet('examples/add_a_node.py', start='# SNIPPETSTART', end='# SNIPPETEND', backtickify='py') }}`

### 💻 Command Line Options

Options:

<!--{{ shell('python -m comfy_catapult.cli --help',
             rich='README.help.generated.svg',
             rich_alt='Output of `python -m comfy_catapult.cli --help`',
             rich_bg_color='black',
             rich_term='xterm-256color',
             rich_cols=160,
             decomentify='nl') }}-->

`execute` options:

<!--{{ shell('python -m comfy_catapult.cli execute --help',
             rich='README.execute-help.generated.svg',
             rich_alt='Output of `python -m comfy_catapult.cli execute --help`',
             rich_bg_color='black',
             rich_term='xterm-256color',
             rich_cols=160,
             decomentify='nl') }}-->

Example usage:

`{{ snippet('examples/cli-example.sh', start='# SNIPPETSTART', end='# SNIPPETEND', backtickify='bash') }}`

## ✅ Requirements

- Python 3.10+
- ComfyUI server with API endpoint enabled.

### Known to work on

- WSL2/Windows11, Ubuntu 22.04.2 LTS: **Python
  {{rawsnippet('./.python-version')|trim}}**.
- Ubuntu 20.04, Python `{{tested_python_versions}}`, tested in GitHub Actions
  workflow ({{path('./.github/workflows/build-and-test.yml',  link='md')}}).

## 🐳 Docker Image

Docker images are published to [ghcr.io/realazthat/{{project_name}}][21] at each
tag.

```bash
# Use the published images at https://ghcr.io/realazthat/{{project_name}}.
docker run --rm --tty ghcr.io/realazthat/{{project_name}}:v{{last_stable_release}} --help

# /data in the docker image is the working directory, so paths are simpler.
docker run --rm --tty \
  -v "${PWD}:/data" \
  -e "COMFY_API_URL=${COMFY_API_URL}" \
  ghcr.io/realazthat/{{project_name}}:v{{last_stable_release}} \
  execute --workflow-path ./test_data/sdxlturbo_example_api.json
```

If you want to build the image yourself, you can use the Dockerfile in the
repository.

<!--{{snippet('./examples/local-docker-example.sh',
              start='# DOCKER_START',
              end='# DOCKER_END',
              backtickify='bash',
              decomentify='nl')|trim}}-->

## 🚸 Limitations

- Interrupting a job will interrupt any job, not the specific job interrupted.
  See [#5](https://github.com/realazthat/comfy-catapult/issues/5).

## TODO

- [ ] Helpers should support remote/cloud storage for ComfyUI input/output/model
      directories (Currently only supports local paths).
- [ ] ETA Estimator.
- [ ] Make sure the schema can parse the formats even if the format adds new
      fields.

## Contributions

### Development environment: Linux-like

- For running `pre.sh` (Linux-like environment).

  - From {{path('./.github/dependencies.yml', link='md')}}, which is used for
    the GH Action to do a fresh install of everything:

    {{shell('python -m yq --yaml-output  \'.dev\' .github/dependencies.yml',
    include_args=False,
    backtickify='yaml',
    indented=4
    )}}

  - Requires `pyenv`, or an exact matching version of python as in
    {{path('./.python-version', link='md')}}.
  - `jq`, ([installation](https://jqlang.github.io/jq/)) required for
    [yq](https://github.com/kislyuk/yq), which is itself required for our
    `{{path('./README.md')}}` generation, which uses `tomlq` (from the
    [yq](https://github.com/kislyuk/yq) package) to include version strings from
    {{path('./pyproject.toml', link='md')}}.
  - act (to run the GH Action locally):
    - Requires nodejs.
    - Requires Go.
    - docker.
  - Generate animation:
    - docker
  - docker (for building the docker image).

### Commit Process

1. (Optionally) Fork the `develop` branch.
2. Stage your files: `git add path/to/file.py`.
3. `bash scripts/pre.sh`, this will format, lint, and test the code.
4. `git status` check if anything changed (generated `{{path('./README.md')}}`
   for example), if so, `git add` the changes, and go back to the previous step.
5. `git commit -m "..."`.
6. Make a PR to `develop` (or push to develop if you have the rights).

## Release Process

These instructions are for maintainers of the project.

1. `develop` branch: Run `bash {{path('./scripts/pre.sh')}}` to ensure
   everything is in order.
2. `develop` branch: Bump the version in
   {{path('./pyproject.toml', link='md')}}, following semantic versioning
   principles. Also modify the `last_release` and `last_stable_release` in the
   `[tool.comfy_catapult-project-metadata]` table as appropriate.
3. `develop` branch: Commit these changes with a message like "Prepare release
   X.Y.Z". (See the contributions section [above](#commit-process)).
4. `master` branch: Merge the `develop` branch into the `master` branch:
   `git checkout master && git merge develop --no-ff`.
5. `master` branch: Tag the release: Create a git tag for the release with
   `git tag -a vX.Y.Z -m "Version X.Y.Z"`.
6. Publish to PyPI: Publish the release to PyPI with
   `bash {{path('./scripts/utilities/deploy-to-pypi.sh')}}`.
7. Push to GitHub: Push the commit and tags to GitHub with `git push` and
   `git push --tags`.
8. `git checkout develop && git merge master` The `--no-ff` option adds a commit
   to the master branch for the merge, so refork the develop branch from the
   master branch.
9. `git push origin develop` Push the develop branch to GitHub.

[1]: .github/logo-exported.svg
[2]:
  https://img.shields.io/badge/Audience-Developers-{{badge_color}}?style=plastic&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJsdWNpZGUgbHVjaWRlLXVzZXJzIj48cGF0aCBkPSJNMTYgMjF2LTJhNCA0IDAgMCAwLTQtNEg2YTQgNCAwIDAgMC00IDR2MiIvPjxjaXJjbGUgY3g9IjkiIGN5PSI3IiByPSI0Ii8+PHBhdGggZD0iTTIyIDIxdi0yYTQgNCAwIDAgMC0zLTMuODciLz48cGF0aCBkPSJNMTYgMy4xM2E0IDQgMCAwIDEgMCA3Ljc1Ii8+PC9zdmc+
[3]:
  https://img.shields.io/badge/Platform-Linux-{{badge_color}}?style=plastic&logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJsdWNpZGUgbHVjaWRlLWxhcHRvcC1taW5pbWFsIj48cmVjdCB3aWR0aD0iMTgiIGhlaWdodD0iMTIiIHg9IjMiIHk9IjQiIHJ4PSIyIiByeT0iMiIvPjxsaW5lIHgxPSIyIiB4Mj0iMjIiIHkxPSIyMCIgeTI9IjIwIi8+PC9zdmc+
[4]:
  https://img.shields.io/github/languages/top/realazthat/{{project_name}}.svg?style=plastic&color={{badge_color}}&cacheSeconds=28800
[5]:
  https://img.shields.io/github/license/realazthat/{{project_name}}?style=plastic&color={{badge_color}}
[6]:
  https://img.shields.io/pypi/v/{{pypi_project_name}}?style=plastic&color={{badge_color}}
[7]: https://pypi.org/project/{{pypi_project_name}}/
[8]:
  https://img.shields.io/pypi/pyversions/{{pypi_project_name}}?style=plastic&color={{badge_color}}
[9]: https://github.com/realazthat/{{project_name}}/tree/master
[10]:
  https://img.shields.io/github/actions/workflow/status/realazthat/{{project_name}}/build-and-test.yml?branch=master&style=plastic
[11]:
  https://github.com/realazthat/{{project_name}}/actions/workflows/build-and-test.yml
[12]:
  https://img.shields.io/github/commits-since/realazthat/{{project_name}}/v{{last_stable_release}}/master?style=plastic&color={{badge_color}}
[13]:
  https://github.com/realazthat/{{project_name}}/compare/v{{last_stable_release}}...master
[14]:
  https://img.shields.io/github/last-commit/realazthat/{{project_name}}/master?style=plastic&color={{badge_color}}
[15]: https://github.com/realazthat/{{project_name}}/tree/develop
[16]:
  https://img.shields.io/github/actions/workflow/status/realazthat/{{project_name}}/build-and-test.yml?branch=develop&style=plastic
[17]:
  https://github.com/realazthat/{{project_name}}/actions/workflows/build-and-test.yml
[18]:
  https://img.shields.io/github/commits-since/realazthat/{{project_name}}/v{{last_stable_release}}/develop?style=plastic&color={{badge_color}}
[19]:
  https://github.com/realazthat/{{project_name}}/compare/v{{last_stable_release}}...develop
[20]:
  https://img.shields.io/github/last-commit/realazthat/{{project_name}}/develop?style=plastic&color={{badge_color}}
[21]: https://github.com/rvion/CushyStudio
[22]: https://github.com/matan1905/ComfyUI-Serving-Toolkit
[23]: https://github.com/city96/ComfyUI_NetDist
[24]:
  https://github.com/comfyanonymous/ComfyUI/tree/89d0e9abeb31e44cccef46537cd10d8812130ef3/script_examples
  "Permalink"
[25]: https://github.com/andreyryabtsev/comfyui-python-api
[26]: https://github.com/BennyKok/comfyui-deploy
[27]: https://github.com/pydn/ComfyUI-to-Python-Extension
[28]: https://github.com/Chaoses-Ib/ComfyScript
[29]: https://pypi.org/project/hordelib/
[30]: https://github.com/nathannlu/comfyui-cloud
[31]: https://github.com/piyushK52/comfy_runner
[32]: https://github.com/thecooltechguy/ComfyUI-ComfyRun