COMFYCLAW in ComfyUI: We Tested the Agent That Builds and Repairs Workflows on Its Own

Leer en español →

On July 2, 2026, a paper landed on arXiv with a name that’s hard to ignore: COMFYCLAW (2607.01709, Zongxia Li, Dawei Liu, Fuxiao Liu and team, Lichao Sun’s lab). The idea: an agent that treats ComfyUI workflow construction as typed graph editing, uses a vision-language model (VLM) verifier to translate visual failures into concrete repairs, and progressively builds a library of reusable “skills” from its own successes and failures.

We didn’t stop at the paper’s abstract. We cloned the official code, installed it against our real ComfyUI on an RTX 3090, hit a genuine bug along the way, diagnosed it down to the exact line of code, and let the agent build and generate a real image — with Claude Code as the brain, no paid API involved.

What COMFYCLAW Is (and Isn’t)

COMFYCLAW is not an image generation model. It’s an agentic harness that drives an unmodified ComfyUI install from a panel inside the UI itself (or from the CLI, which is how we tested it here). You type a prompt, and the agent:

1. Builds the workflow graph as typed edits (adds nodes, wires them, validates that the graph is executable).
2. Sends the generation to your real ComfyUI.
3. A VLM verifier looks at the resulting image and translates visual failures into region-level critiques with concrete repair strategies.
4. If the result doesn’t pass the threshold, it repeats with the fixes. If it passes, it distills the trajectory into a reusable “skill” for next time.

Important heads-up before you search for it yourself: the name “ComfyClaw” is heavily overloaded on GitHub. We found at least 3-4 unrelated projects using the same name — a CLI for inspecting workflows, a set of nodes for prototyping agent structures, an OpenClaw skill. The real, verified repository for this paper is Moms-Organic-Agent-Lab/comfyclaw — we confirmed this by checking that the README cites the exact same nine authors as the paper, with links to their personal academic homepages.

The Real Install: What We Actually Did

Requirements: Python 3.10+, uv, a ComfyUI install with at least one checkpoint already loaded, and an agent backend. We used claude-code (our already-authenticated Claude Code session), without touching ANTHROPIC_API_KEY.

git clone https://github.com/Moms-Organic-Agent-Lab/comfyclaw.git
cd comfyclaw
uv sync --extra sync

cp .env.example .env
# In .env:
#   COMFYUI_DIR=/path/to/your/ComfyUI
#   COMFYCLAW_AGENT_BACKEND=claude-code

uv run comfyclaw install-node --comfyui-dir /path/to/your/ComfyUI
# restart ComfyUI

uv run comfyclaw doctor   # pre-flight check

doctor confirmed everything green: Python, .env, ComfyUI reachable, the ComfyClaw-Sync plugin correctly symlinked into custom_nodes/, and all 23 bundled skills loaded without error. The only warning (no API key set) is irrelevant with the claude-code backend.

The Real Bug: A False “0.0 GB Free” With a Genuinely Free GPU

On our first real generation attempt, with the RTX 3090 showing 22.7GB free per nvidia-smi, COMFYCLAW refused to generate:

Compute risk: Image generation likely needs about 4 GB free VRAM, but cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync reports 0.0 GB free.

This wasn’t a timing fluke — it reproduced identically on a second attempt minutes later. We traced the exact cause to harness.py (line ~878):

free = self._vram_gb(
    dev.get("torch_vram_free")   # <- wrong priority
    or dev.get("vram_free")
    or dev.get("vram_total")
    or dev.get("torch_vram_total")
)

The check reads torch_vram_free first, the field ComfyUI’s /system_stats exposes for memory free inside PyTorch’s own cached pool. The problem: when ComfyUI runs with the cudaMallocAsync backend (increasingly common because it reduces VRAM fragmentation — and exactly what our install uses), that pool never goes through PyTorch’s normal caching allocator, so torch_vram_free stays permanently near zero — in our test, 25MB out of 32MB total, rounding to “0.0 GB” — no matter how much VRAM is genuinely free at the device level (the correct field, vram_free, did show the real 22.7GB). Python’s or operator never gets to the correct field because the first one, however useless, isn’t exactly zero.

There’s no --force flag to skip this warning in CLI mode. To complete the test, we swapped the priority of the two fields in our local copy (vram_free first) — a one-line change. With that fix, generation went through fine.

If you use the panel inside ComfyUI instead of the CLI, this warning shows up as an interactive question (“are you sure you want to generate?”) that you can accept manually — the CLI, on the other hand, just silently skips generation, which matters a lot if you’re automating it unattended.

The Real Test: The Agent Builds, Generates, and Its Own Verifier Finds a Flaw

With the bug worked around, we ran:

uv run comfyclaw run --prompt "a red fox sitting in a snowy forest at dawn, photorealistic" --iterations 2

The agent (Claude Sonnet 4.5 via claude-code) built a complete SDXL pipeline from scratch — CheckpointLoaderSimple → CLIPTextEncode (positive/negative) → EmptyLatentImage → KSampler → VAEDecode → SaveImage — with a detailed positive prompt and a solid negative one, sampler tuned to 30 steps, cfg 6.5, dpmpp_2m/karras. It validated the graph (7 nodes, no errors) and sent it to our real ComfyUI.

Real measured timings: the first generation took 83.98s (most of it, ~54s, loading the Juggernaut XL checkpoint from disk for the first time); with the model already in memory, pure sampling for 30 steps at 1024×1024 took 8.32 seconds (~4.1 it/s on the RTX 3090).

The VLM verifier (Claude 3.5 Sonnet, with vision) scored the result 0.90 out of 1.0 and passed all 7 binary checks (is there a fox? is it red/orange? is it sitting? is it a forest? is there snow? does the light suggest dawn? is the style photorealistic?). Since 0.90 cleared the 0.85 threshold, it stopped after the first iteration without spending the second.

What’s interesting isn’t the high score — it’s what the verifier found despite passing. It flagged, as [HIGH] anatomy, an ambiguous dark shape next to the fox’s hindquarters (“doesn’t read as a coherent bushy tail, breaks anatomical continuity”), with concrete repair strategies (add_inpaint_pass, inject_lora_anatomy, add_controlnet_pose). Looking closely at the image, the critique is right: there’s a diffuse brownish mass behind the hind legs that doesn’t integrate well with the rest of the body. It also flagged, at lower severity, an overly uniform background of tree trunks and slightly merged front legs.

This is exactly what the paper promises: a verifier that doesn’t just give a global pass/fail, but points to where the problem is and what technique to try to fix it — although in our test, with only 1 of 2 allowed iterations spent, the system decided 0.90 was already good enough and never actually applied any of those fixes.

The Skills: 23 Built In, Including One for Qwen-Image

COMFYCLAW ships with 23 built-in skills following the Agent Skills specification (SKILL.md format with YAML frontmatter) — among them workflow-builder (the one our agent used), photorealistic, inpainting, controlnet-control, hires-fix, and one specifically for qwen-image-2512, the exact Chinese image model we’ve been covering in ComfyLab’s recent trend radar. With “progressive disclosure,” the agent only sees each skill’s name and a short description until it decides to load the full instructions — keeping context manageable even with dozens of skills available.

Conclusion: Is It Worth Trying Right Now?

🏆 Our Recommendation

If you already use Claude Code and ComfyUI together (like in our article on automating ComfyUI with Claude Code — Spanish only for now), COMFYCLAW is a logical next step: instead of asking Claude to hand-edit a workflow JSON, you give it a tool built specifically for that, with automatic visual verification included. It genuinely works — we confirmed it on our own GPU, not on the paper’s numbers — but it ships with an install that still has rough edges (the VRAM bug we documented here will block anyone running cudaMallocAsync, which isn’t a rare setup).

If you’re looking for something production-polished, this isn’t it — it’s days-old research code. But as a piece to experiment with workflow automation, backed by a real paper and with no API cost if you already pay for Claude Code, it’s one of the most interesting things we’ve tested in weeks.

Sources

• COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows (arXiv 2607.01709)
• Official repository: Moms-Organic-Agent-Lab/comfyclaw