ShopSight - AI-Powered E-commerce Analytics

A prototype dashboard for intelligent e-commerce analytics powered by LLMs, demonstrating natural language search, historical sales visualization, and AI-generated insights.

---

⚡ TL;DR - Quick Overview

What: AI-powered e-commerce analytics dashboard with natural language search
Stack: Next.js + FastAPI + OpenAI GPT-4o-mini + DuckDB
Data: Real H&M dataset (476K transactions, 105K products) from S3
Time: ~2 hours development

What's Working (Real):

• ✅ NLP product search → LLM parses queries
• ✅ Sales charts → Real H&M transaction data
• ✅ AI insights → LLM analyzes patterns
• ✅ AI Agent → Conversational analytics interface

What's Mocked: Forecasts & customer segments (with production roadmap documented below)

---

📹 Demo Video

📺 Watch Demo Video - 2 minute walkthrough showing search → analytics → AI agent interaction

🎯 Project Overview

ShopSight transforms e-commerce analytics by making it agentic, search-driven, and intelligent. Users can search for products in natural language and instantly see:

• 📊 Historical Sales Trends (real data visualization)
• 🤖 AI-Generated Insights (LLM-powered analysis)
• 📈 Sales Forecasts (mocked, but production-ready architecture)
• 👥 Customer Segmentation (mocked, demonstrates future capability)

🏗️ Architecture

┌─────────────────────────────────────────────┐
│         Next.js Frontend (TypeScript)       │
│  - React Components (Recharts for charts)   │
│  - Tailwind CSS for styling                 │
│  - Axios for API calls                      │
└──────────────────┬──────────────────────────┘
                   │ REST API
┌──────────────────▼──────────────────────────┐
│         FastAPI Backend (Python)            │
│  ┌──────────────────────────────────────┐   │
│  │  LLM Service (OpenAI GPT-4o-mini)    │   │
│  │  - Natural language search parsing   │   │
│  │  - Insights generation               │   │
│  └──────────────────────────────────────┘   │
│  ┌──────────────────────────────────────┐   │
│  │  Data Processor (DuckDB + Pandas)    │   │
│  │  - H&M dataset processing            │   │
│  │  - Sales aggregation                 │   │
│  └──────────────────────────────────────┘   │
└─────────────────────────────────────────────┘

✨ Features

✅ Fully Implemented (End-to-End Working)

1. Natural Language Product Search 🔍

   • User types: "running shoes", "red dresses", etc.
   • LLM parses query into structured search parameters
   • Returns matching products from H&M dataset
   • Displays friendly interpretation of search intent

2. Historical Sales Visualization 📊

   • Real data from H&M transaction history
   • Interactive line/bar charts (Recharts)
   • Weekly aggregation with sales count & revenue
   • Clean, professional UI with tooltips

3. AI-Generated Insights 🤖

   • LLM analyzes sales patterns
   • Generates human-readable insights
   • Identifies trends, anomalies, recommendations
   • Contextual analysis of performance data

🎨 Smartly Mocked (Production-Ready Architecture)

4. Sales Forecasting 📈

   • Mocked 30-day predictions
   • Shows confidence intervals
   • Production approach: Time series models (ARIMA, Prophet, LSTM)

5. Customer Segmentation 👥

   • Mocked buyer personas with characteristics
   • Shows percentage distribution and avg purchase value
   • Production approach: K-means clustering, RFM analysis

🚀 Getting Started

Prerequisites

• Node.js 18+ (for Next.js frontend)
• Python 3.10+ (for FastAPI backend)
• OpenAI API Key (for LLM features)

Installation

1. Clone the repository

   git clone <your-repo-url>
   cd ShopSight

2. Backend Setup

   cd backend
   
   # Create virtual environment
   python3 -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   
   # Install dependencies
   pip install -r requirements.txt
   
   # Create .env file with your OpenAI API key
   export OPENAI_API_KEY="your_key_here"

   Note: Data loads automatically from S3 on first startup (no preprocessing needed!)

3. Frontend Setup

   cd ../frontend
   
   # Install dependencies
   npm install

Running the Demo

Terminal 1 - Backend:

cd backend
source venv/bin/activate
python -m uvicorn main:app --reload --port 8000

Terminal 2 - Frontend:

cd frontend
npm run dev

Open browser: http://localhost:3000

Quick Test Flow

1. Type "running shoes" in the search bar
2. Click Search → AI parses your query
3. Select a product from results
4. See the magic:
   • 📊 Real sales chart loads
   • 🤖 AI insights appear
   • 📈 Forecast shown (mocked)
   • 👥 Segments displayed (mocked)

🧠 Thought Process & Design Decisions

Why This Scope?

Goal: Demonstrate LLM integration + real data processing in a demoable prototype

Strategy:

• ✅ Perfect one end-to-end flow (search → sales → insights)
• ✅ Use real data where it matters (sales history)
• ✅ Mock intelligently where it doesn't (forecasts, segments)
• ✅ Polish the UX to feel production-ready

Technical Choices

Choice	Rationale
Next.js	Modern React framework, fast development, great DX
FastAPI	Python + async + auto docs = perfect for data/ML APIs
OpenAI GPT-4o-mini	Fast & cheap for demos, easy upgrade to GPT-4o
DuckDB	In-memory SQL engine, blazing fast for analytics queries
Recharts	Beautiful React charts with minimal config
Tailwind CSS	Rapid UI development, clean design system

LLM Integration Strategy

Use Case 1: Semantic Search 🔍

User: "comfortable shoes for running"
↓
LLM extracts: {type: "shoes", activity: "running", attribute: "comfort"}
↓
Maps to: product_type="sneakers", description contains "running"

Use Case 2: Insights Generation 🤖

1. Fetch sales data (real)
2. Calculate aggregates (peaks, trends)
3. LLM synthesizes narrative
4. Return structured insights + summary

Why LLMs Add Value:

• Natural language interface (vs complex filters)
• Contextual understanding (not just keyword matching)
• Human-readable explanations (not just numbers)
• Scalable to more complex queries

🔌 API Reference

Base URL: http://localhost:8000 • Interactive Docs: http://localhost:8000/docs

Core Analytics

• POST /api/search - Natural language product search
• GET /api/analytics/sales/{article_id} - Historical sales data (✅ Real)
• GET /api/analytics/forecast/{article_id} - Predictive sales forecasting (🎭 Mocked)
• GET /api/analytics/segments/{article_id} - Customer demographic analysis (🎭 Mocked)
• GET /api/insights/{article_id} - AI-generated performance insights (✅ Real)

AI Agent

• POST /api/agent/ask - Interactive AI analytics with conversational context (✅ Real)

Example Request

# Search for products
curl -X POST http://localhost:8000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "black dress"}'

# Get sales history
curl http://localhost:8000/api/analytics/sales/123456789

# Ask AI Agent
curl -X POST http://localhost:8000/api/agent/ask \
  -H "Content-Type: application/json" \
  -d '{"article_id": "123456789", "question": "What are the sales trends?", "conversation_history": []}'

📊 Data

H&M Dataset

Source: s3://kumo-public-datasets/hm_with_images/

Automatic Loading:

• Downloads from S3 on first startup
• Caches locally for fast subsequent loads
• No manual preprocessing needed

What's loaded:

• articles/ → 105,542 products (parquet format)
• transactions/ → 476,039 sales records (parquet format)

Processing:

• Products: Loaded into memory as JSON for fast search
• Transactions: Loaded into DuckDB for fast analytics queries
• Cache: Stored in backend/data/cache/ for instant reloads

🤔 Assumptions & Design Decisions

Key Assumptions

1. Data Scale & Performance

   • Assumed ~100K products and ~500K transactions is acceptable for demo
   • Products loaded in memory (fast search, acceptable for demo scale)
   • Transactions in DuckDB (OLAP-optimized for analytics queries)
   • In production: Would use proper DB (PostgreSQL + Redis cache)

2. LLM Usage & Cost

   • Assumed OpenAI API access is available
   • Used GPT-4o-mini for cost efficiency (~10x cheaper than GPT-4)
   • Structured outputs ensure reliability (no hallucinated JSON)
   • Average cost per search: ~$0.001-0.002

3. User Experience

   • Assumed users prefer natural language over complex filters
   • 2-3 second LLM latency is acceptable for better UX
   • Search results filtered to products with sales data (avoid empty charts)
   • Visual loading states to manage expectations

4. Mocking Strategy

   • Forecasts/segments mocked because:
     • Real ML models would take hours to train properly
     • Demo timeline prioritizes working end-to-end flow
     • Architecture is ready for easy swap-in of real models
   • Mocked data is realistic (not random garbage)

5. Demo Constraints

   • Target: 1-2 hour development time (actual: ~2 hours)
   • Prioritized: One perfect flow over many half-baked features
   • Tech stack: Chose familiar tools (Next.js/FastAPI) for speed
   • No authentication (demo scenario, would add in production)

6. Data Quality

   • Assumed H&M dataset is clean (it is!)
   • Sales data is normalized/sparse → Applied 30x scaling for believable demo numbers
   • Revenue scaled 300,000x (original prices were normalized)
   • In production: Would use actual transaction values

Trade-offs Made

Decision	Why	Production Alternative
In-memory products	Fast search, simple	PostgreSQL with full-text search
DuckDB for analytics	Perfect for OLAP, embeddable	ClickHouse or BigQuery
Mock forecasts	ML models take too long to train	Time series models (Prophet, ARIMA)
Mock segments	Real clustering needs customer profiles	K-means on RFM analysis
S3 direct load	Simple, no infra needed	ETL pipeline (Airflow/Dagster)
Monolithic backend	Faster development	Microservices for scale

🔮 Future Enhancements

What I Would Build Next

1. Real Forecasting Models 📈

   • Prophet for seasonality detection
   • LSTM for complex patterns
   • Ensemble methods for robustness
   • A/B test different models

2. Real Customer Segmentation 👥

   • RFM (Recency, Frequency, Monetary) analysis
   • K-means clustering on purchase behavior
   • DBSCAN for anomaly detection
   • Persona generation with LLMs

3. Multi-Turn Conversations 💬

   • "Show me top products"
   • "Compare to last quarter"
   • "What drove the spike in November?"
   • Agentic flow with function calling

4. Comparative Analysis ⚖️

   • Side-by-side product comparison
   • Category-level insights
   • Competitor benchmarking (with external data)

5. Data Pipeline 🔄

   • Incremental ETL from S3
   • Real-time updates
   • Caching layer (Redis)
   • Data quality monitoring

6. Advanced Visualizations 📊

   • Heatmaps for seasonal patterns
   • Cohort analysis
   • Geographic insights
   • Interactive drill-downs

📁 Project Structure

ShopSight/
├── backend/
│   ├── main.py                 # FastAPI app entry point
│   ├── routers/
│   │   ├── search.py           # Product search endpoints
│   │   ├── analytics.py        # Sales & forecasts
│   │   ├── insights.py         # AI insights generation
│   │   └── agent.py            # AI analytics agent
│   ├── services/
│   │   ├── llm_service.py      # OpenAI integration
│   │   └── data_processor.py   # S3 loading & DuckDB queries
│   ├── models/
│   │   └── schemas.py          # Pydantic models
│   ├── data/
│   │   ├── cache/              # S3 cached data (auto-downloaded)
│   │   └── raw/                # Raw S3 parquet files
│   ├── requirements.txt        # Python dependencies
│   └── venv/                   # Virtual environment (gitignored)
├── frontend/
│   ├── app/
│   │   ├── page.tsx            # Main dashboard
│   │   ├── layout.tsx          # App layout
│   │   ├── globals.css         # Global styles
│   │   └── favicon.ico         # Browser icon
│   ├── components/
│   │   ├── SearchBar.tsx       # Search input
│   │   ├── ProductImage.tsx    # Image with lazy loading
│   │   ├── SalesChart.tsx      # Historical sales visualization
│   │   ├── ForecastChart.tsx   # Demand forecast chart
│   │   ├── InsightsPanel.tsx   # AI insights display
│   │   ├── SegmentCards.tsx    # Customer segments
│   │   ├── AIAgent.tsx         # Conversational AI agent
│   │   └── LoadingSpinner.tsx  # Loading states
│   ├── lib/
│   │   └── api.ts              # API client & types
│   ├── package.json            # Node dependencies
│   ├── next.config.ts          # Next.js config
│   ├── tsconfig.json           # TypeScript config
│   └── node_modules/           # NPM packages (gitignored)
├── README.md                   # This file
└── .gitignore                  # Git ignore rules

📄 License

This is a demo project for Kumo.AI interview purposes.

---

💭 Final Thoughts

This prototype demonstrates my ability to:

• ✅ Quickly scope and prioritize features
• ✅ Integrate LLMs thoughtfully (not just chatbots)
• ✅ Build production-quality architecture
• ✅ Balance real implementation vs. smart mocking
• ✅ Create demoable, polished UX
• ✅ Communicate technical decisions clearly

Time spent: ~2 hours (with focus on quality over speed)

What I'm most proud of:

1. AI Analytics Agent - Went beyond requirements with a conversational interface that provides contextual insights
2. Triple LLM Integration - Search parsing, insight generation, AND agent (not just one chatbot)
3. Real Data Pipeline - Actual S3 loading, DuckDB analytics, not just mocks
4. Production Architecture - Clean separation, type-safe, scalable patterns
5. Thoughtful UX - Dark theme, lazy-loaded images, smooth interactions

Key Achievements:

• ✅ End-to-end working flow (search → analytics → insights)
• ✅ Real H&M dataset with 476K transactions processed
• ✅ Three distinct LLM use cases demonstrating agentic patterns
• ✅ Modern, polished UI that feels customer-ready
• ✅ Honest mocking with production roadmap

Thank you for the opportunity to build this! 🚀

---

Contact: Manush Murali • manushpalaniappan@gmail.com • @Manushpm8