Skip to content

⚡ Bolt: Optimize keyword density extraction and lookups #269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
anchapin wants to merge 1 commit into
base: main
Choose a base branch
from
Open

⚡ Bolt: Optimize keyword density extraction and lookups #269

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@
## 2025-02-18 - Regex Pre-compilation in Hot Paths
**Learning:** Re-compiling regexes inside a frequently called function (like `latex_escape` which runs for every string) creates significant overhead. Pre-compiling them at module level yielded a ~3.2x speedup.
**Action:** Always look for regex compilations inside loops or frequently called functions and move them to module level constants.

## 2025-02-18 - Set vs List for keyword lookups
**Learning:** Using a list for an `in` membership check inside a frequently called function (like `_suggest_sections_for_keyword`) results in O(n) performance and repeated allocations.
**Action:** Always convert static lists used for membership testing to module-level sets for O(1) performance and to avoid reallocation overhead.
113 changes: 58 additions & 55 deletions cli/utils/keyword_density.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,59 @@

console = Console()

# Pre-compiled regex patterns to avoid redundant compilation inside loops/frequent function calls
_TITLE_PATTERNS = [
re.compile(r"(?:job title|position|title):\s*([^\n]+)", re.IGNORECASE | re.MULTILINE),
re.compile(r"^([^\n]+)\s*[-|]\s*[^|]+$", re.IGNORECASE | re.MULTILINE),
re.compile(r"#\s*([^\n]+)", re.IGNORECASE | re.MULTILINE),
]

_COMPANY_PATTERNS = [
re.compile(r"(?:company|organization):\s*([^\n]+)", re.IGNORECASE),
re.compile(r"(?:at|from)\s+([A-Z][^\n]+?)(?:\s+[-\u2014]|\s+$)", re.IGNORECASE),
]

# Using a set for O(1) lookup performance instead of O(N) list search,
# and defining at module level to prevent reallocation on every function call.
_TECH_KEYWORDS = {
"python",
"javascript",
"typescript",
"react",
"vue",
"angular",
"node.js",
"django",
"flask",
"fastapi",
"kubernetes",
"docker",
"aws",
"gcp",
"azure",
"sql",
"mongodb",
"postgresql",
"redis",
"ci/cd",
"devops",
"machine learning",
"ai",
"llm",
"pytorch",
"tensorflow",
"graphql",
"rest api",
"microservices",
"java",
"go",
"rust",
"c++",
"c#",
".net",
"spring",
}


@dataclass
class KeywordInfo:
Expand Down Expand Up @@ -208,26 +261,15 @@ def _extract_job_details(self, job_description: str) -> Tuple[str, str]:
company = ""

# Try to extract job title (common patterns)
title_patterns = [
r"(?:job title|position|title):\s*([^\n]+)",
r"^([^\n]+)\s*[-|]\s*[^|]+$",
r"#\s*([^\n]+)", # Markdown headers often have job title
]

for pattern in title_patterns:
match = re.search(pattern, job_description, re.IGNORECASE | re.MULTILINE)
for pattern in _TITLE_PATTERNS:
match = pattern.search(job_description)
if match:
job_title = match.group(1).strip()
break

# Try to extract company name
company_patterns = [
r"(?:company|organization):\s*([^\n]+)",
r"(?:at|from)\s+([A-Z][^\n]+?)(?:\s+[-\u2014]|\s+$)",
]

for pattern in company_patterns:
match = re.search(pattern, job_description, re.IGNORECASE)
for pattern in _COMPANY_PATTERNS:
match = pattern.search(job_description)
if match:
company = match.group(1).strip()
break
Expand Down Expand Up @@ -398,46 +440,7 @@ def _suggest_sections_for_keyword(
suggestions = []

# Check if keyword is tech-related
tech_keywords = [
"python",
"javascript",
"typescript",
"react",
"vue",
"angular",
"node.js",
"django",
"flask",
"fastapi",
"kubernetes",
"docker",
"aws",
"gcp",
"azure",
"sql",
"mongodb",
"postgresql",
"redis",
"ci/cd",
"devops",
"machine learning",
"ai",
"llm",
"pytorch",
"tensorflow",
"graphql",
"rest api",
"microservices",
"java",
"go",
"rust",
"c++",
"c#",
".net",
"spring",
]

if keyword.lower() in tech_keywords:
if keyword.lower() in _TECH_KEYWORDS:
suggestions.append("Skills section")
Comment on lines +443 to 444
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Normalize the keyword more defensively (e.g., strip whitespace) before membership checks.

If keyword can include leading/trailing spaces or similar artifacts, known terms may fail the membership check (e.g., 'python ' vs 'python'). Normalizing locally with something like keyword = keyword.strip().lower() before the lookup would make this check more robust at minimal cost, even if upstream normalization usually occurs.

Suggested change
if keyword.lower() in _TECH_KEYWORDS:
suggestions.append("Skills section")
normalized_keyword = keyword.strip().lower()
if normalized_keyword in _TECH_KEYWORDS:
suggestions.append("Skills section")


# Check experience bullets
Expand Down
Loading