⚡ Bolt: Offload blocking I/O calls in FastAPI endpoints to worker threads

.jules/bolt.md

@@ -1,15 +1,3 @@
-## 2025-02-18 - Lazy Loading Config
-**Learning:** Instantiating `Config` was consuming ~30-70ms due to `import yaml` and file I/O, even for commands that didn't need it (like `init` or `help`).
-**Action:** Implemented lazy loading in `Config` class. `_config` is initialized to `None` and loaded only on first access to properties. This reduces startup time for simple commands and defers the cost for others.
-
-## 2025-02-18 - Caching Jinja2 Environment
-**Learning:** Instantiating Jinja2 Environment is surprisingly expensive (35ms vs 0.7ms) even with FileSystemLoader, likely due to filter registration and internal setup.
-**Action:** Cache Environment instances at class level when template directory is constant or keys are manageable.
-
-## 2024-05-22 - Lazy Imports for CLI Performance
-**Learning:** Top-level imports of heavy libraries (like Jinja2) in a CLI entry point slow down all commands, even those that don't use the library (like `--help`).
-**Action:** Move heavy imports inside the specific command functions where they are used.
-
-## 2025-02-18 - Regex Pre-compilation in Hot Paths
-**Learning:** Re-compiling regexes inside a frequently called function (like `latex_escape` which runs for every string) creates significant overhead. Pre-compiling them at module level yielded a ~3.2x speedup.
-**Action:** Always look for regex compilations inside loops or frequently called functions and move them to module level constants.
+## 2024-05-18 - Async IO offloading for FastAPI routes
+**Learning:** In FastAPI applications, synchronous blocking operations (such as file I/O or CPU-bound code) placed directly inside `async def` routes can block the entire event loop, starving other requests.
+**Action:** When working with FastAPI and heavy synchronous calls, remember to use `anyio.to_thread.run_sync()` in combination with `functools.partial()` (as `run_sync` only accepts positional arguments for the callable) to properly offload these blocking steps to a worker thread and keep the event loop non-blocking.

api/main.py

@@ -1,8 +1,10 @@
+import functools
 import logging
 import os
 import tempfile
 from pathlib import Path
 
+import anyio
 import yaml
 from fastapi import FastAPI, HTTPException, Response, Security
 from fastapi.middleware.cors import CORSMiddleware
@@ -171,13 +173,20 @@ async def render_pdf(request: ResumeRequest):
             # We output to a temp file
             output_pdf = temp_path / "output.pdf"
 
-            generator.generate(variant=request.variant, output_format="pdf", output_path=output_pdf)
+            await anyio.to_thread.run_sync(
+                functools.partial(
+                    generator.generate,
+                    variant=request.variant,
+                    output_format="pdf",
+                    output_path=output_pdf,
+                )
+            )
 
-            if not output_pdf.exists():
+            if not await anyio.to_thread.run_sync(output_pdf.exists):
                 raise HTTPException(status_code=500, detail="PDF generation failed")
 
             # Read bytes
-            content = output_pdf.read_bytes()
+            content = await anyio.to_thread.run_sync(output_pdf.read_bytes)
 
             return Response(
                 content=content,
@@ -215,8 +224,12 @@ async def tailor_resume(request: TailorRequest):
         # We pass None for yaml_path as we use direct data
         generator = AIGenerator(yaml_path=None, config=config)
 
-        tailored_data = generator.tailor_data(
-            resume_data=request.resume_data, job_description=request.job_description
+        tailored_data = await anyio.to_thread.run_sync(
+            functools.partial(
+                generator.tailor_data,
+                resume_data=request.resume_data,
+                job_description=request.job_description,
+            )
         )
 
         return tailored_data
@@ -248,7 +261,9 @@ async def ats_check(request: ATSRequest):
 
         # Generate ATS report directly from resume data
         generator = ATSGenerator(config=config, resume_data=request.resume_data)
-        report = generator.generate_report(request.job_description, request.variant)
+        report = await anyio.to_thread.run_sync(
+            functools.partial(generator.generate_report, request.job_description, request.variant)
+        )
 
         # Convert to JSON-serializable format
         result = {
@@ -300,11 +315,14 @@ async def generate_cover_letter(request: CoverLetterRequest):
 
         # Generate cover letter (always use non-interactive for API)
         # The provided answers will be used as context by the AI
-        outputs, job_details = generator.generate_non_interactive(
-            job_description=request.job_description,
-            company_name=request.company_name,
-            variant=request.variant,
-            output_formats=[request.format],
+        outputs, job_details = await anyio.to_thread.run_sync(
+            functools.partial(
+                generator.generate_non_interactive,
+                job_description=request.job_description,
+                company_name=request.company_name,
+                variant=request.variant,
+                output_formats=[request.format],
+            )
         )
 
         # Return the generated content
@@ -323,11 +341,12 @@ async def generate_cover_letter(request: CoverLetterRequest):
 
                 # Compile LaTeX to PDF
                 pdf_path = temp_path / "cover-letter.pdf"
-                if generator._compile_pdf(pdf_path, outputs["pdf"]):
+                if await anyio.to_thread.run_sync(
+                    functools.partial(generator._compile_pdf, pdf_path, outputs["pdf"])
+                ):
                     import base64
 
-                    with open(pdf_path, "rb") as f:
-                        pdf_bytes = f.read()
+                    pdf_bytes = await anyio.to_thread.run_sync(pdf_path.read_bytes)
                     return {
                         "content": base64.b64encode(pdf_bytes).decode("utf-8"),
                         "format": "pdf",
@@ -553,19 +572,28 @@ async def render_resume_pdf(resume_id: str, variant: str = "base"):
         converter = JSONResumeConverter()
         yaml_data = converter.json_resume_to_yaml(_resume_storage[resume_id]["json_resume"])
 
-        with open(resume_yaml_path, "w", encoding="utf-8") as f:
-            yaml.dump(yaml_data, f, default_flow_style=False)
+        yaml_str = yaml.dump(yaml_data, default_flow_style=False)
+        await anyio.to_thread.run_sync(
+            functools.partial(resume_yaml_path.write_text, yaml_str, encoding="utf-8")
+        )
 
         try:
             generator = TemplateGenerator(yaml_path=resume_yaml_path)
             output_pdf = temp_path / "output.pdf"
 
-            generator.generate(variant=variant, output_format="pdf", output_path=output_pdf)
+            await anyio.to_thread.run_sync(
+                functools.partial(
+                    generator.generate,
+                    variant=variant,
+                    output_format="pdf",
+                    output_path=output_pdf,
+                )
+            )
 
-            if not output_pdf.exists():
+            if not await anyio.to_thread.run_sync(output_pdf.exists):
                 raise HTTPException(status_code=500, detail="PDF generation failed")
 
-            content = output_pdf.read_bytes()
+            content = await anyio.to_thread.run_sync(output_pdf.read_bytes)
 
             return Response(
                 content=content,