Stable Diffusion ControlNet Canny Model Plugin for dashAI

This plugin integrates Stability AI's Stable Diffusion 3.5 with a ControlNet Canny conditioning into the dashAI platform using the Hugging Face diffusers backend. It enables structure-guided image generation, where a Canny edge map extracted from an input image steers the generated output, and supports private access using a Hugging Face API token.

Included Models

1. Stable Diffusion 3.5 Large

Base text-to-image diffusion model, based on stabilityai/stable-diffusion-3.5-large.

2. ControlNet Canny (SD 3.5 Large)

Canny edge conditioning model, based on stabilityai/stable-diffusion-3.5-large-controlnet-canny.

The two models work together: the input image is converted to a Canny edge map, which conditions the SD 3.5 Large pipeline to follow the original structure while regenerating content from the text prompt. Compatible with CPU or GPU inference through diffusers.

About ControlNet

ControlNet is a neural network architecture that adds spatial conditioning to text-to-image diffusion models. By extracting a control signal (here, Canny edges) from a reference image, it constrains generation to follow that structure.

Key features of this ControlNet Canny plugin:

• Structure-guided generation: output follows the edges of the input image
• Canny preprocessing: automatic edge detection applied to the input
• Text-conditioned: content is guided by natural language prompts
• Negative prompting: steer the model away from unwanted elements
• Adjustable conditioning strength via controlnet_conditioning_scale
• Gated weights with access control via Hugging Face

ControlNet is designed for deployment on laptops, desktops, and cloud infrastructure, making controllable image generation more accessible.

Features

• Structure-guided image generation from an input image plus text prompt
• Automatic Canny edge extraction from the input image
• Reproducible generation via fixed random seeds
• Automatic login to Hugging Face to access gated models
• Configurable generation parameters:
  • negative_prompt: elements to avoid in the image
  • num_inference_steps: number of denoising steps
  • guidance_scale: how strongly the model follows the prompt
  • controlnet_conditioning_scale: how closely output follows the control edges
  • num_images_per_prompt: number of images per prompt
  • device: detected GPU (e.g. "GPU 0: NVIDIA RTX 3090 - Compute Capability 8.6") or "CPU"

Model Parameters

Parameter	Description	Default
huggingface_key	Hugging Face API token to access restricted models	Required
negative_prompt	Text prompt for elements to avoid in the image	"" (optional)
num_inference_steps	Number of denoising steps (higher = better quality, slower)	15
guidance_scale	How strongly the model follows the prompt	3.5
controlnet_conditioning_scale	How closely the output follows the ControlNet edges	1.0
seed	Random seed for reproducibility (negative = random)	-1
num_images_per_prompt	Number of images to generate per prompt	1
device	Inference device — a detected GPU by name (e.g. "GPU 0: NVIDIA RTX 3090 - Compute Capability 8.6") or "CPU"	First detected GPU if available, else "CPU"

Requirements

• diffusers
• torch
• torchvision
• opencv-python
• Valid Hugging Face Access Token
• Model files from Hugging Face:
  • stabilityai/stable-diffusion-3.5-large
  • stabilityai/stable-diffusion-3.5-large-controlnet-canny

Access Notice: You must accept the model terms on Hugging Face and use a valid Hugging Face token. This repository is publicly accessible, but gated. You need to agree to share your contact information to access the model files.

Notes

This plugin extracts Canny edges from the input image using OpenCV, then conditions the SD 3.5 Large pipeline on that edge map.

The models are pretrained for inference and are not designed for fine-tuning through this plugin. Image dimensions are automatically adjusted to multiples of 8.