🌷🤖 tulip agent

A reference implementation for the tulip agent, an LLM-backed agent with access to a large number of tools via a tool library. This approach reduces costs, enables the use of tool sets that exceed API limits or context windows, and increases flexibility with regard to the tool set used.

Key components

🔬 Function analysis
Generates OpenAI API compatible tool descriptions for Python functions via introspection

🌷 Tool library
Combines a vector store for semantic search among tools and tool execution

🤖 Agents
Specifying instructions for an agent completely overrides the base system prompts to avoid contradictions. You can append custom instructions to the default prompts in tulip_agent.prompts.

• Baseline, without tool library
  • BaseAgent: LLM agent without tool access
  • NaiveToolAgent: Includes tool descriptions for all tools available
  • CotToolAgent: Extends the NaiveToolAgent with a planning step that decomposes the user input into subtasks
• Tulip variations with access to a tool library
  • MinimalTulipAgent: Minimal implementation; searches for tools based on the user input directly
  • NaiveTulipAgent: Naive implementation; searches for tools with a separate tool call
  • CotTulipAgent: COT implementation; derives a plan for the necessary steps and searches for suitable tools
  • InformedCotTulipAgent: Same as CotTulipAgent, but with a brief description of the tool library's contents
  • PrimedCotTulipAgent: Same as CotTulipAgent, but primed with tool names based on an initial search with the user request
  • OneShotCotTulipAgent: Same as CotTulipAgent, but the system prompt included a brief example
  • AutoTulipAgent: Fully autonomous variant; can use the search tool at any time and modify its tool library with CRUD operations
  • DfsTulipAgent: DFS inspired variant that leverages a DAG for keeping track of tasks and suitable tools, can create new tools

📊 Evaluation

• math_eval: Math evaluation
• robo_eval: Robotics evaluation using tools created for AttentiveSupport

📝 Examples
See ./examples

Setup

• Make sure to set the environment variables required by the API of your choice. Currently supported:
  • OpenAI: OPENAI_API_KEY, see the official instructions
  • Azure: AZURE_OPENAI_API_KEY, AZURE_API_VERSION, and AZURE_OPENAI_ENDPOINT
  • OpenAI compatible endpoints: OAI_COMPATIBLE_BASE_URL and OAI_COMPATIBLE_API_KEY for OpenAI compatible endpoints, such as Ollama
• Install with poetry install or pip install -e .
• Check out the examples, the robot evaluation in src/eval/robo_eval, and examples/local_examples.py for a local setup

Dev notes

• Python v3.10.11 recommended, higher versions may lead to issues with chroma when installing via Poetry
• Pre-commit hooks - install with (poetry run) pre-commit install
• Linting: ruff
• Formatting: black
• Import sorting: isort
• Tests: Run with (poetry run) python -m unittest discover tests/

Known issues

SQLite version incompatibility

See these troubleshooting instructions

1. On Linux install pysqlite3-binary: poetry add pysqlite3-binary
2. Add the following to lib/python3.10/site-packages/chromadb/__init__.py in your venv

__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

Running the example results in a ModuleNotFoundError

Make sure to install the package itself, e.g., with poetry install or pip install -e .
Then run the example with poetry run python examples/calculator_example.py