[ENHANCEMENT] Add fallback handling for unreliable LLM outputs

[Lochit-Vinay]

Closes #430

Summary

This PR introduces a fallback mechanism to handle inconsistent or noisy outputs from the LLM during field extraction.

Problem

In certain cases, the LLM returns responses that are:

• prefixed with additional text (e.g., "Location: Delhi")
• wrapped in quotes or formatted inconsistently
• overly long or partially irrelevant
• missing or unclear (-1, null, etc.)

These outputs can lead to incorrect values being stored or reduce reliability of the extraction pipeline.

Solution

• Added a safe_extract_value function to normalize and clean LLM responses
• Implemented fallback handling:
  • invalid or unclear values are mapped to:
    { "value": None, "requires_review": true }
• Added checks for:
  • prefixed responses
  • extra formatting
  • overly long / noisy outputs
• Ensured plural values are still handled correctly

Impact

• Improves robustness of extraction pipeline
• Prevents unreliable values from being blindly stored
• Enables better downstream validation and review workflows

Testing

Tested with multiple response formats:

• clean values
• prefixed responses
• noisy outputs
• invalid / missing values

The system now handles edge cases without breaking the pipeline.

[Lochit-Vinay]

Resolved merge conflicts with latest main.
Preserved fallback handling logic, prompt engineering updates, and structured error handling while removing duplicate or conflicting code paths.
Ensured the LLM extraction flow and response parsing remain consistent after merge.