feat: Firecrawl web search integration

[kncshw]

Purpose for this enhancement:

1. Perplexity has no free quota so users have to subscribe to enable the web_search tool for LLMs. Firecrawl offers free tier APIs for LLM.
2. Perplexity seems running at a blackbox mode, on the other hand, firecrawl can safely store the web search/crawl outcomes.
3. web_search tool is crucial, without web_search, LLM can only use curl, which is inaccurate and consumes lots of tokens. LLM direct Search requests are typically blocked by CAPTCHA

Test has been completed with Firecrawl but I have not tested the Perplexity as I do not have the API key. The Maintainer needs to do regression tests with Perplexity API key

[0xallam]

Hey @kncshw

Thanks for contributing!

One thing I'm a bit hesitant about: since Firecrawl just returns raw scraped content (unlike Perplexity which does search + LLM summary in one go), this implementation ends up making a separate LLM call inside the tool to synthesize the results. That feels a bit awkward architecturally - "tools" shouldn't really be calling LLMs on their own, and it adds a fair amount of complexity.

Do you think we could simplify by just returning the raw Firecrawl markdown and letting the agent handle it? Or maybe just use another search API with a free tier (like Exa.ai, etc) that works more like Perplexity out of the box?

[kncshw]

Yes, your concern is valid. I think this: 'Do you think we could simplify by just returning the raw Firecrawl markdown and letting the agent handle it? ' could be a better option. thanks.