Demonstrate memory; correct provider return types

[twilwa]

Would like to include examples of this synthesis of troubleshooting today:

Eliza Codebase: Lessons Learned & Reference Guide

1. Overview

This reference document compiles insights, code examples, and troubleshooting tips gathered from recent discussions, session logs, and code reviews related to the Eliza agent system. Its primary goal is to help developers:

• Understand the roles of Providers, Actions, and Evaluators
• Work effectively with Memory (short-term vs. long-term)
• Integrate Embedding vectors for advanced search and retrieval
• Diagnose and resolve common pitfalls or errors

---

2. Core Concepts

2.1 Providers

• Purpose: Fetch or prepare data for your agent. A provider typically retrieves external/contextual information—such as recent messages, user info, or facts—and formats it so that the runtime (and subsequent actions) can consume it.
• Common Confusion:
  • Provider vs. Action: Providers do not change or act upon the environment. They gather or format data. Actions, on the other hand, are meant to perform something, like storing or modifying data, rebalancing a liquidity pool, etc.

Example (simplified from a session snippet):

typescript

Copy code

import { Provider, IAgentRuntime, Memory, State } from "@ai16z/eliza"; const customProvider: Provider = { get: async (runtime: IAgentRuntime, message: Memory, state?: State) => { // 1. Gather recent memories const memories = await runtime.messageManager.getMemories({ roomId: message.roomId, count: 5, }); // 2. Format or transform them const contextString = formatContextString(memories); // 3. Return the formatted context return contextString; }, };

---

2.2 Actions

• Purpose: Actions do the “work.” They leverage data from providers (or memory) to execute logic—such as storing new facts, calling external APIs, or performing domain-specific tasks (e.g., rebalancing a liquidity pool).
• Key Points:
  • Actions often occur after data is retrieved by a provider.
  • They may store results back into memory, or initiate a further chain of events.

Example:

typescript

Copy code

const customAction = { name: "customAction", execute: async (runtime: IAgentRuntime, message: Memory) => { // 1. Possibly call a provider or fetch from memory const providerOutput = await someProvider.get(runtime, message); // 2. Use the context from providerOutput const result = await runtime.llm.complete({ prompt: "Use this context to respond", context: providerOutput, }); // 3. (Optional) Save new data to memory await runtime.messageManager.storeMemory({ type: "action_result", content: result, roomId: message.roomId, }); return result; } };

---

2.3 Evaluators

• Purpose: To validate conditions, check constraints, or decide whether an action is permissible or needed.
• Usage Pattern: Often used in a pipeline, e.g.:
  1. Provider fetches data
  2. Evaluator checks if the data meets certain criteria
  3. Action executes if evaluator conditions pass

Example:

typescript

Copy code

const memoryEvaluator = { name: "memoryEvaluator", evaluate: async (runtime: IAgentRuntime, message: Memory) => { // For instance, only proceed if we have at least 3 recent user messages const userMessages = await runtime.messageManager.getMemories({ roomId: message.roomId, userId: message.userId, count: 3, }); return userMessages.length >= 3; } };

---

3. Memory Management

3.1 Short-Term vs. Long-Term Memory

• Short-Term/State: Data relevant only to the current turn or immediate session. This is often stored in an in-memory state object, not persisted in a DB.
• Long-Term/Database Memory: Data that’s persisted to a database (e.g., SQLite). This memory allows the agent to recall information across sessions or beyond ephemeral usage.

Storing and Retrieving:

typescript

Copy code

// Storing a piece of knowledge in memory await runtime.messageManager.storeMemory({ type: "knowledge", content: "Useful fact or context", roomId: message.roomId, metadata: { timestamp: Date.now(), source: "session" } }); // Retrieving the last 5 pieces of knowledge const knowledgeMemories = await runtime.messageManager.getMemories({ roomId: message.roomId, count: 5, type: "knowledge" });

---

3.2 Embedding Vectors

• Purpose: Allow semantic search or advanced retrieval by encoding text into a vector.
• Common Issues:
  • Zero-length vector: Ensure your embedding model is properly configured and that you’re actually sending it some text.
  • Dimension mismatch: Verify that the model’s expected dimensionality aligns with how you’ve configured your memory table.

Tips:

1. Check Environment: Confirm the embedding service is running or your local model is accessible.
2. Reset & Re-initialize: If embeddings fail repeatedly, try resetting the DB and re-initializing environment variables.

---

4. Troubleshooting Workflows

1. Version Compatibility

   • Keep your Eliza (or related frameworks) version consistent across all plugins and providers to avoid mismatched APIs.
   • Example: If you recently upgraded to v1.8, ensure that your plugin and memory managers are also updated.

2. Provider Context Errors

   • Typically arise when asynchronous calls aren’t awaited before returning.
   • Double-check that your get methods fully resolve promises.

3. Memory Retrieval Issues

   • Verify that you’re using the correct roomId, userId, or type.
   • Confirm the data was actually persisted (e.g., no early return or silent error blocked storeMemory).

4. Embedding Failures

   • Confirm the local or remote embedding model is running.
   • Look for dimension mismatch logs in your DB or runtime console.

5. Action Logic

   • If an action isn’t executing, see if an evaluator or condition is blocking it.
   • Make sure the action name is spelled correctly and is registered in the runtime.

---

5. Example End-to-End Flow

Below is a simplified flow demonstrating how a user message might travel through providers, evaluators, and an action, all while storing something to memory for future reference:

typescript

Copy code

// 1. User sends a message -> runtime calls provider const memoryProvider: Provider = { get: async (runtime, message) => { const recentMemories = await runtime.messageManager.getMemories({ roomId: message.roomId, count: 5, unique: true, }); return formatMessages(recentMemories); }, }; // 2. Evaluator checks if we have enough context const enoughContextEvaluator = { name: "enoughContextEvaluator", evaluate: async (runtime, message) => { const memoryCount = await runtime.messageManager.countMemories({ roomId: message.roomId, }); return memoryCount > 3; // simple check }, }; // 3. Action uses the provider's output if evaluator passes const handleUserRequestAction = { name: "handleUserRequest", execute: async (runtime, message) => { // Grab context from the provider const context = await memoryProvider.get(runtime, message); // Embedding example const embedding = await embed(runtime, context); // Optionally store the embedding in memory for retrieval await runtime.messageManager.storeMemory({ roomId: message.roomId, type: "embedding", content: JSON.stringify(embedding), }); // Use the context for a response const response = await runtime.llm.complete({ prompt: `Given this context: ${context}\nRespond to user:`, }); // Persist the final answer in memory await runtime.messageManager.storeMemory({ type: "answer", content: response, roomId: message.roomId, }); return response; }, };

1. Provider fetches recent memories and formats them
2. Evaluator determines if we have sufficient memory to proceed
3. Action uses the context and optionally stores new memory (including embeddings)

---

6. Final Recommendations

1. Document Patterns
   • Maintain a concise table or glossary explaining each file’s purpose (topics.ts, wallet.ts, etc.) and the roles of providers, evaluators, and actions.
2. Error-Handling Patterns
   • Implement robust error handling in provider code (try/catch) to handle asynchronous issues gracefully and log helpful messages.
3. Version Synchronization
   • Whenever updating Eliza’s core library, check all plugins and providers for API changes or method signature updates.
4. Testing & Logging
   • Use console/log statements or a structured logging framework to trace the flow between memory retrieval, evaluation, and action execution.

---

7. Conclusion

This comprehensive guide captures the core knowledge shared throughout recent development and troubleshooting sessions. It explains why the codebase is designed as it is, how to utilize providers and actions together, and where common pitfalls might appear—particularly around memory and embedding usage.

By following the outlined best practices, maintaining clean version control, and ensuring consistent documentation, future users and maintainers of the Eliza system should have a smoother experience building and debugging this powerful agent framework.

---

Additional Resources

• GitHub Discussions: Check the plugin’s repository PR comments from contributors like Neo Sofratis for advanced usage patterns.
• Embedding Models: Validate your embeddings against well-known vector libraries or AI providers to confirm dimension and data integrity.
• Community Support: For unresolved or emergent issues, reaching out to the broader developer community can provide fresh insights and solutions.