Skip to content

Commit b64ce64

Browse files
docs: Update ControlNet use case docs (#4519)
* Update ControlNext use case docs Signed-off-by: Sherlock113 <[email protected]> * ci: auto fixes from pre-commit.ci For more information, see https://pre-commit.ci --------- Signed-off-by: Sherlock113 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 504ff63 commit b64ce64

File tree

1 file changed

+20
-47
lines changed

1 file changed

+20
-47
lines changed

docs/source/use-cases/diffusion-models/controlnet.rst

Lines changed: 20 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,10 @@ Create BentoML :doc:`/guides/services` in a ``service.py`` file to specify the s
4040
4141
import typing as t
4242
43-
import cv2
4443
import numpy as np
4544
import PIL
4645
from PIL.Image import Image as PIL_Image
4746
48-
import torch
49-
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel, AutoencoderKL
5047
from pydantic import BaseModel
5148
5249
import bentoml
@@ -55,21 +52,21 @@ Create BentoML :doc:`/guides/services` in a ``service.py`` file to specify the s
5552
VAE_MODEL_ID = "madebyollin/sdxl-vae-fp16-fix"
5653
BASE_MODEL_ID = "stabilityai/stable-diffusion-xl-base-1.0"
5754
58-
5955
@bentoml.service(
6056
traffic={"timeout": 600},
6157
workers=1,
6258
resources={
63-
"gpu": "1",
59+
"gpu": 1,
6460
"gpu_type": "nvidia-l4",
65-
# we can also specify GPU memory requirement:
66-
# "memory": "16Gi",
6761
}
6862
)
69-
class SDXLControlNetService:
63+
class ControlNet:
7064
7165
def __init__(self) -> None:
7266
67+
import torch
68+
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel, AutoencoderKL
69+
7370
if torch.cuda.is_available():
7471
self.device = "cuda"
7572
self.dtype = torch.float16
@@ -94,58 +91,34 @@ Create BentoML :doc:`/guides/services` in a ``service.py`` file to specify the s
9491
torch_dtype=self.dtype
9592
).to(self.device)
9693
97-
98-
@bentoml.api
99-
async def generate(
100-
self,
101-
prompt: str,
102-
arr: np.ndarray[t.Any, np.uint8],
103-
**kwargs,
104-
):
105-
image = PIL.Image.fromarray(arr)
106-
return self.pipe(prompt, image=image, **kwargs).to_tuple()
107-
108-
109-
class Params(BaseModel):
110-
prompt: str
111-
negative_prompt: t.Optional[str]
112-
controlnet_conditioning_scale: float = 0.5
113-
num_inference_steps: int = 25
114-
115-
116-
@bentoml.service(
117-
name="sdxl-controlnet-service",
118-
traffic={"timeout": 600},
119-
workers=8,
120-
resources={"cpu": "1"}
121-
)
122-
class ControlNet:
123-
controlnet_service: SDXLControlNetService = bentoml.depends(SDXLControlNetService)
124-
12594
@bentoml.api
12695
async def generate(self, image: PIL_Image, params: Params) -> PIL_Image:
96+
import cv2
97+
12798
arr = np.array(image)
12899
arr = cv2.Canny(arr, 100, 200)
129100
arr = arr[:, :, None]
130101
arr = np.concatenate([arr, arr, arr], axis=2)
131102
params_d = params.dict()
132103
prompt = params_d.pop("prompt")
133-
res = await self.controlnet_service.generate(
104+
image = PIL.Image.fromarray(arr)
105+
return self.pipe(
134106
prompt,
135-
arr=arr,
107+
image=image,
136108
**params_d
137-
)
138-
return res[0][0]
139-
140-
This file defines the following classes:
109+
).to_tuple()[0][0]
141110
142-
* ``SDXLControlNetService``: A BentoML Service with custom configurations in timeout, worker count, and resources.
111+
class Params(BaseModel):
112+
prompt: str
113+
negative_prompt: t.Optional[str]
114+
controlnet_conditioning_scale: float = 0.5
115+
num_inference_steps: int = 25
143116
144-
- It loads the three pre-trained models and configures them to use GPU if available. The main pipeline (``StableDiffusionXLControlNetPipeline``) integrates these models.
145-
- It defines an API endpoint ``generate`` to process a text prompt and an image array. The processed image is converted to a tuple and returned.
117+
This file defines a BentoML Service ``ControlNet`` with custom :doc:`configurations </guides/configurations>` in timeout, worker count, and resources.
146118

147-
* ``Params``: This is a ``pydantic`` model defining the structure for input parameters.
148-
* ``ControlNet``: A BentoML Service with custom configurations in timeout, worker count, and resources. ``ControlNet`` doesn't create images itself. Instead, it preprocesses the image and forwards it along with the text prompt to the ``SDXLControlNetService`` Service. The ``generate`` method in ``ControlNet`` then returns the final generated image.
119+
- It loads the three pre-trained models and configures them to use GPU if available. The main pipeline (``StableDiffusionXLControlNetPipeline``) integrates these models.
120+
- It defines an asynchronous API endpoint ``generate``, which takes an image and a set of parameters as input. The parameters for the generation process are extracted from a ``Params`` instance, a Pydantic model that provides automatic data validation.
121+
- The ``generate`` method returns the generated image by calling the pipeline with the processed image and text prompts.
149122

150123
Run ``bentoml serve`` in your project directory to start the BentoML server.
151124

0 commit comments

Comments
 (0)