Overview

Introduction

This framework is a modular agent system designed to harness the power of large language models (LLMs) alongside dynamic tool integration and conversation memory. Its architecture is split into several components, such as memory management, LLM provider integration, and tool management. Additionally, advanced features—like semantic memory, retrieval-augmented generation (RAG), multimodal support, and streaming responses—allow the framework to be extended and customized for various applications.

Key Features

Memory Management

  • Basic Memory: Keeps track of recent conversation history.
  • Semantic Memory: Uses open source libraries (SentenceTransformers, FAISS) for embedding-based retrieval.

LLM Provider Integration

Supports multiple providers (e.g., OpenAI, Anthropic) with a unified interface.

Tool Management

Dynamically loads and executes external tools (like a calculator) based on configuration.

Retrieval-Augmented Generation

Augments responses by retrieving contextually similar documents.

Agent Communication

Enables message passing between agents.

Evaluation Framework

Logs performance metrics for continuous improvement.

Multimodal Support

Processes images and audio using open source libraries.

Streaming Responses

Streams output for more interactive experiences.

Type Safety

Provides input/output validation to prevent errors.

Use Cases

  • Conversational Agents: Build chatbots or virtual assistants that can understand and respond to user queries.
  • Task Automation: Integrate and automate external tools (e.g., calculators, document search) seamlessly.
  • Information Retrieval: Enhance responses with context retrieved from indexed documents.
  • Multimodal Applications: Process and analyze images or audio files as part of the interaction.

Example Code

Below is a basic example of how to create an agent, register a tool, and run a query:

# Example: Creating and running an agent

from agent import create_agent, run_agent
from tool import ToolManager

# Step 1: Set up a basic agent configuration
# This creates the agent configuration file (agent_config.json) and registers a simple calculator tool.
ToolManager.setup_basic_config()

# Step 2: Create an agent using the configuration file.
agent = create_agent("agent_config.json", llm_provider="openai")

# Step 3: Run the agent with a query.
response = run_agent(agent, "What is 2 + 2?")
print("Agent Response:", response)

In this example:

  • ToolManager.setup_basic_config() registers a basic calculator tool and generates the necessary configuration.
  • create_agent() initializes the agent with the settings from agent_config.json.
  • run_agent() sends a query to the agent, which processes the query through the LLM and uses tools as needed.

This framework offers a robust foundation for building conversational systems. With its modular design, developers can easily extend its capabilities, whether by integrating new tools, switching memory modules, or adding multimodal support.