Artificial Intelligence Companion that actually says things.

Malaika isn't just a companion, she's a presence. Inspired by the minimalist elegance of "Her" my favorite movie, and the high energy aesthetic of modern social interfaces.

Tech Stack

Frontend

Backend

Tools & Platforms

Quick Start

Prerequisites

• Python 3.9+
• Node.js 18+
• Docker & Docker Compose
• Hugging Face API Token

Project Structure

• frontend/: Contains the React application with TypeScript and Three.js 3D rendering.
• backend/: Contains the FastAPI backend with ChromaDB memory integration.

Setup and Installation

Backend

1. Navigate to the backend directory:

   cd backend

2. Install the required Python packages:

   pip install -r requirements.txt

3. Set your Hugging Face API key as an environment variable:

   export HF_TOKEN='your_huggingface_token'

4. Run the backend server:

   uvicorn app.main:app --reload --port 8000

Frontend

1. Navigate to the frontend directory:

   cd frontend

2. Install the required npm packages:

   npm install --legacy-peer-deps

3. Start the React development server:

   npm start

Docker Setup (Alternative)

1. Run with Docker Compose:

   docker-compose up --build

Features

• Multi-modal Memory: Integrated with ChromaDB for hybrid semantic/chronological retrieval of conversations.
• Proactive Visual Awareness: Powered by Qwen2.5-VL-7B-Instruct for high-fidelity visual and video context awareness.
• Emotional Depth: Dual-layered emotion engine analyzing facial expressions and message sentiment.
• Natural Speech Synthesis: Edge-TTS with natural prosody and conversational flow.

Usage

1. Open your web browser and navigate to http://localhost:3000.
2. Interact with Malaika through voice or text.
3. Clap your hands to trigger and start voice conversation!

macOS Permissions & Setup

To allow Malaika to control your macOS laptops, you must grant the following permissions:

1. Accessibility: Required for controlling system volume, brightness, and window management via osascript and Quartz.
   • Go to System Settings > Privacy & Security > Accessibility.
   • Add and enable your Terminal (e.g., iTerm2 or Terminal.app) and/or your IDE (e.g., VS Code).
2. Screen Recording: Required for the Vision module to capture frames for analysis.
   • Go to System Settings > Privacy & Security > Screen Recording.
   • Enable for your terminal/browser.
3. Microphone: Required for voice interaction and clap detection.
   • Go to System Settings > Privacy & Security > Microphone.
   • Enable for your browser (Chrome/Safari) and Terminal.
4. Automation: Ensure the terminal is allowed to control "System Events".
   • Usually prompted on first run; can be managed in System Settings > Privacy & Security > Automation.

License

This project is licensed under the MIT License.

Support

If you have any questions or issues, please open an issue on GitHub or contact mwalyangashadrack@gmail.com