Improvement: Add proper error handling for nil embedding in ai-service processor

[pragnyanramtha]

Description

The _generate_embedding() method in KnowledgeProcessor returns None on failure, but this value is used without nil checks in the calling methods.

Location

File: backend/ai-service/app/llm/processor.py
Lines: 216-226 (method), called from lines 30, 54

Current Code

async def _generate_embedding(self, text: str) -> list:
    """Generate embedding vector for the text"""
    try:
        response = openai.Embedding.create(
            input=text,
            model=self.embedding_model
        )
        return response["data"][0]["embedding"]
    except Exception as e:
        logger.error(f"Error generating embedding: {e}")
        return None  # Returns None on failure!

Problem

When embedding fails, None is stored in the database, making the knowledge item unsearchable.

Suggested Fix

1. Add retry logic with exponential backoff for embedding generation
2. Re-raise the exception or handle None embedding gracefully (skip storage, mark as pending)
3. Add a separate worker to retry failed embeddings
4. Update store_knowledge() to handle None embeddings

Labels

bug, ai-service, embeddings, error-handling