+ ```sh
+ git clone https://github.com/LebToki/MoA.git
+ cd MoA
+ ```
-
+ ```
+ GROQ_API_KEY=your_groq_api_key
+ OPENAI_API_KEY=your_openai_api_key
+ DEBUG=0
+ ```
-## Multi-layer MoA Example
+### Running the Application
-In the previous example, we went over how to implement MoA with 2 layers (4 LLMs answering and one LLM aggregating). However, one strength of MoA is being able to go through several layers to get an even better response. In this example, we'll go through how to run MoA with 3+ layers in `advanced-moa.py`.
-
-```python
-python advanced-moa.py
+ Open your web browser and navigate to the following URL to access the web-based interface:
+```sh
+ http://127.0.0.1:5000/
```
-
-## Interactive CLI Demo
+## Usage
-This interactive CLI demo showcases a simple multi-turn chatbot where the final response is aggregated from various reference models.
+### Interacting with the MoA Grog Chatbot
-To run the interactive demo, follow these 3 steps:
+- **Model Selection**: Choose the model you want to use from the dropdown menu. The default model is `llama3-70b-8192`, which balances performance and speed.
+
+- **Temperature Control**: Adjust the temperature setting to control the randomness and creativity of the chatbot's responses. The default value is `0.7`, providing a good balance between deterministic and varied outputs.
+
+- **Max Tokens**: Define the maximum number of tokens (words or characters) for the response. The default is `2048`, which ensures comprehensive answers without overwhelming verbosity.
+
+- **Create Your Topics**: Easily create new conversation topics by entering your desired topic names in the text field provided. This helps organize your interactions and revisit previous conversations.
+
+- **Choose Your Topic**: Select a topic by clicking on it in the sidebar. The chat interface will load on the right, allowing you to continue your discussion seamlessly.
+
+- **Instruction Input**: Enter your prompt or instruction in the text area. This is where you ask questions or provide commands to the chatbot.
+
+- **Theme Toggle**: Enhance your user experience by switching between light and dark modes. Use the "Switch to Dark Mode" button to toggle themes based on your preference.
+
+- **Submit and View Responses**: After filling in the necessary fields, submit the form to receive a response from the MoA Grog Chatbot. The response will be displayed on the same page, within the chat interface.
-1. Export Your API Key: `export TOGETHER_API_KEY={your_key}`
-2. Install Requirements: `pip install -r requirements.txt`
-3. Run the script: `python bot.py`
+### Additional Features
-The CLI will prompt you to input instructions interactively:
+- **Create New Conversations**: Use the sidebar to create new conversation topics, helping you manage different discussions effectively.
+
+- **Reset All Conversations**: If needed, reset all conversations from the sidebar to start fresh.
-1. Start by entering your instruction at the ">>>" prompt.
-2. The system will process your input using the predefined reference models.
-3. It will generate a response based on the aggregated outputs from these models.
-4. You can continue the conversation by inputting more instructions, with the system maintaining the context of the multi-turn interaction.
+This intuitive interface makes it easy to engage with the MoA Grog Chatbot, providing a seamless and interactive user experience.
-### [Optional] Additional Configuration
-The demo will ask you to specify certain options but if you want to do additional configuration, you can specify these parameters:
+### Planned Features
-- `--aggregator`: The primary model used for final response generation.
-- `--reference_models`: List of models used as references.
-- `--temperature`: Controls the randomness of the response generation.
-- `--max_tokens`: Maximum number of tokens in the response.
-- `--rounds`: Number of rounds to process the input for refinement. (num rounds == num of MoA layers - 1)
-- `--num_proc`: Number of processes to run in parallel for faster execution.
-- `--multi_turn`: Boolean to toggle multi-turn interaction capability.
+- **Uploading Documents**: The MoA Groq Chatbot will support file uploads and interact with the content of the uploaded documents, adding functionality to handle file uploads, process the contents of these files, and integrate the results into the chatbot's conversation flow.
+- **Refine the styling**: With the latest Bootstrap and jQuery, we have room to enhance the UI/UX further.
+- **Chronological ordering**: Move recent chats to the top (DESC order) to cut down on scrolling.
+- **Further enhancement of the output**: Currently, not much has been implemented to control the output except basic styling. This is an area to be worked on based on various use cases.
-## Evaluation
-We provide scripts to quickly reproduce some of the results presented in our paper
-For convenience, we have included the code from [AlpacaEval](https://github.com/tatsu-lab/alpaca_eval),
-[MT-Bench](https://github.com/lm-sys/FastChat), and [FLASK](https://github.com/kaistAI/FLASK), with necessary modifications.
-We extend our gratitude to these projects for creating the benchmarks.
+## File Structure
-### Preparation
+- **MoA/**
+ - `app.py` - Flask application
+ - `bot.py` - Main chatbot logic
+ - `utils.py` - Utility functions
+ - `requirements.txt` - Python dependencies
+ - **templates/**
+ - `index.html` - HTML template for the web interface
+ - `chat.html` - HTML template for the chat interface
+ - **static/**
+ - `style.css` - CSS styles for the web interface
+ - `script.css` - JavaScript for theme switching and UI enhancements
+ - `bot.png` - favicon
-```bash
-# install requirements
-pip install -r requirements.txt
-cd alpaca_eval
-pip install -e .
-cd FastChat
-pip install -e ".[model_worker,llm_judge]"
-cd ..
-# setup api keys
-export TOGETHER_API_KEY={{ conversation.topic }}
+
+ {% for msg in messages %}
+
+
+
+ {{ msg['role'] }}:
+
+ {% endfor %}
+ {{ msg['content']|safe }}
+
+
-```
-bash run_eval_alpaca_eval.sh
-```
+## Acknowledgements
-### Run MT-Bench
+- **Original Creator**: Special thanks to the original creator of this amazing project. [togethercomputer](https://github.com/togethercomputer/MoA)
+- **Matthew Berman**: A special thanks to [Matthew Berman](https://www.youtube.com/@matthew_berman) for showcasing this project and providing valuable insights.
-For a minimal example of MT-Bench evaluation, run:
-
-```
-bash run_eval_mt_bench.sh
-```
-
-### Run FLASK
+## License
-For a minimal example of FLASK evaluation, run:
+Previous Releases
+Initial Release · [01/07/2024] -To run AlpacaEval 2, execute the following scripts: +- Code Organization: Initial setup of the project with organized structure for controllers, models, and views. +- Error Handling: Basic error handling for API requests and user inputs. +- Front-end Enhancements: Initial design of the UI with Bootstrap and FontAwesome integration. Responsive design for better accessibility on all devices. +- Performance Considerations: Basic optimizations for faster loading times. +- Accessibility and Usability: Added alt attributes to all images for better accessibility. +This work by Tarek Tarabichi is licensed under CC BY 4.0
- 
-
-
+# Note for Contributors
+This project is based on Groq's API, not the original Together API.
+If you are contributing, please ensure compatibility and optimizations are aligned with Groq's specifications and guidelines.
-We achieved top positions on both the AlpacaEval 2.0 leaderboard and MT-Bench. Notably, on AlpacaEval 2.0, using solely open-source models, we achieved a margin of 7.6% absolute improvement from 57.5% (GPT-4 Omni) to 65.1% (MoA).
+## Contact
-
- -
- 
-
-
+For any questions or feedback, please open an issue in this repository.
-FLASK offers fine-grained evaluation of models across multiple dimensions. Our MoA method significantly outperforms the original Qwen1.5-110B-Chat on harmlessness, robustness, correctness, efficiency, factuality, commonsense, insightfulness, completeness. Additionally, MoA also outperforms GPT-4 Omni in terms of correctness, factuality, insightfulness, completeness, and metacognition.
+---
+# Get Involved
+Whether you're a developer, system integrator, or enterprise user, you can trust that we did everything possible to make it as smooth and easy as 1,2,3 to set up MoA Groq Chatbot.
-Please feel free to contact us if you have difficulties in reproducing the results.
+⭐ Give us a star on GitHub 👆
-## Credits
+⭐ Fork the project on GitHub and contribute👆
-Notably, this work was made possible by the collaborative spirit and contributions of active organizations in the AI field. We appreciate the efforts of Meta AI, Mistral AI, Microsoft, Alibaba Cloud, and DataBricks for developing the Llama 3, Mixtral, WizardLM 2, Qwen 1.5, and DBRX models. Additionally, we extend our gratitude to Tatsu Labs, LMSYS, and KAIST AI for developing the AlpacaEval, MT-Bench, and FLASK evaluation benchmarks.
+🚀 Do you like to code? You're more than welcome to contribute Join the Discussions!
-## License
+💡 Got a feature suggestion? Add your roadmap ideas
-This project is licensed under the Apache 2.0 License - see the LICENSE file for details.
+
- -
-## Citation +This project is licensed under the Attribution License. +2024 · Tarek Tarabichi from 2TInteractive.com · Made with 💙 -If you find this work helpful, please consider citing: +Thank you for using MoA Groq Chatbot! +We hope you find it useful and look forward to your contributions. -```bibtex -@article{wang2024mixture, - title={Mixture-of-Agents Enhances Large Language Model Capabilities}, - author={Wang, Junlin and Wang, Jue and Athiwaratkun, Ben and Zhang, Ce and Zou, James}, - journal={arXiv preprint arXiv:2406.04692}, - year={2024} -} -``` diff --git a/app.py b/app.py new file mode 100644 index 0000000..f3b24de --- /dev/null +++ b/app.py @@ -0,0 +1,128 @@ +from flask import Flask, request, render_template, redirect, url_for +from flask_sqlalchemy import SQLAlchemy +from flask_uploads import UploadSet, configure_uploads, DOCUMENTS +from werkzeug.utils import secure_filename +from datetime import datetime +from flask_migrate import Migrate +import os + +# Ensure the 'uploads' directory exists +if not os.path.exists('uploads'): + os.makedirs('uploads') + +from utils import generate_with_references, generate_together_stream # Ensure these functions are defined or imported + +app = Flask(__name__) +app.config['SECRET_KEY'] = os.urandom(24) +app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///instance/conversations.db' +app.config['UPLOADED_FILES_DEST'] = 'uploads' +app.config['UPLOADED_FILES_ALLOW'] = DOCUMENTS + +db = SQLAlchemy(app) +migrate = Migrate(app, db) + +files = UploadSet('files', DOCUMENTS) +configure_uploads(app, files) + + +class Conversation(db.Model): + id = db.Column(db.Integer, primary_key=True) + topic = db.Column(db.String(100), nullable=False) + messages = db.Column(db.Text, nullable=False) + created_at = db.Column(db.DateTime, default=datetime.utcnow) + + +@app.route('/') +def home(): + conversations = Conversation.query.order_by(Conversation.created_at.desc()).all() + return render_template('index.html', conversations=conversations) + + +@app.route('/chat/
+
+
+
+
diff --git a/templates/index.html b/templates/index.html
new file mode 100644
index 0000000..10758a2
--- /dev/null
+++ b/templates/index.html
@@ -0,0 +1,85 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/utils.py b/utils.py
index 4651745..cf6acff 100644
--- a/utils.py
+++ b/utils.py
@@ -7,16 +7,17 @@
from loguru import logger
+from dotenv import load_dotenv
+load_dotenv()
DEBUG = int(os.environ.get("DEBUG", "0"))
-
def generate_together(
- model,
- messages,
- max_tokens=2048,
- temperature=0.7,
- streaming=False,
+ model,
+ messages,
+ max_tokens=2048,
+ temperature=0.7,
+ streaming=False,
):
output = None
@@ -25,7 +26,7 @@ def generate_together(
try:
- endpoint = "https://api.together.xyz/v1/chat/completions"
+ endpoint = "https://api.groq.com/openai/v1/chat/completions"
if DEBUG:
logger.debug(
@@ -41,7 +42,7 @@ def generate_together(
"messages": messages,
},
headers={
- "Authorization": f"Bearer {os.environ.get('TOGETHER_API_KEY')}",
+ "Authorization": f"Bearer {os.environ.get('GROQ_API_KEY')}",
},
)
if "error" in res.json():
@@ -75,16 +76,16 @@ def generate_together(
def generate_together_stream(
- model,
- messages,
- max_tokens=2048,
- temperature=0.7,
+ model,
+ messages,
+ max_tokens=2048,
+ temperature=0.7,
):
- endpoint = "https://api.together.xyz/v1"
+ endpoint = "https://api.groq.com/openai/v1/"
client = openai.OpenAI(
- api_key=os.environ.get("TOGETHER_API_KEY"), base_url=endpoint
+ api_key=os.environ.get("GROQ_API_KEY"), base_url=endpoint
)
- endpoint = "https://api.together.xyz/v1/chat/completions"
+ endpoint = "https://api.groq.com/openai/v1/chat/completions"
response = client.chat.completions.create(
model=model,
messages=messages,
@@ -97,10 +98,10 @@ def generate_together_stream(
def generate_openai(
- model,
- messages,
- max_tokens=2048,
- temperature=0.7,
+ model,
+ messages,
+ max_tokens=2048,
+ temperature=0.7,
):
client = openai.OpenAI(
@@ -135,8 +136,8 @@ def generate_openai(
def inject_references_to_messages(
- messages,
- references,
+ messages,
+ references,
):
messages = copy.deepcopy(messages)
@@ -161,21 +162,27 @@ def inject_references_to_messages(
def generate_with_references(
- model,
- messages,
- references=[],
- max_tokens=2048,
- temperature=0.7,
- generate_fn=generate_together,
+ model,
+ messages,
+ references=[],
+ max_tokens=2048,
+ temperature=0.7,
+ generate_fn=generate_together_stream,
):
-
if len(references) > 0:
-
messages = inject_references_to_messages(messages, references)
- return generate_fn(
+ # Generate response using the provided generate function
+ response = generate_fn(
model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
)
+
+ # Check if the response is in the expected format
+ if hasattr(response, 'choices'):
+ return response
+ else:
+ return [{"choices": [{"delta": {"content": response}}]}]
+
+
+
+
+
+ {{ conversation.topic }}
+
+ {% for msg in messages %}
+
+
+
+ {{ msg['role'] }}:
+
+ {% endfor %}
+ {{ msg['content']|safe }}
+