QIDK - Qualcomm Innovators Development Kit Project

🎯 Project Overview

This project is a comprehensive solution built for the Qualcomm Innovators Development Kit (QIDK) that leverages hardware-accelerated AI for image processing and generation. The project consists of two main components:

1. Backend: FastAPI-powered server that runs on the QIDK hardware, utilizing Qualcomm's Neural Processing Unit (NPU) for AI model inference.
2. Frontend: React Native mobile application that provides a user-friendly interface for interacting with the AI capabilities.

📋 Features

• User authentication with registration, login, and OTP verification
• AI-powered image generation and transformation
• Real-time model inference on QIDK hardware
• Mobile-friendly UI with intuitive controls
• Hardware-accelerated model execution using Qualcomm NPU

🧠 AI Integration

AI Models Used

The project incorporates several AI models optimized for edge deployment:

1. Fast Neural Style Transfer: Applies artistic styles to user-provided images using a lightweight PyTorch model optimized for NPU execution.
2. SNPE (Snapdragon Neural Processing Engine): Runtime for executing neural networks on Qualcomm hardware.
3. QNN (Qualcomm Neural Network): SDK for hardware-accelerated neural network inference on QIDK.

AI Tools & Integration Points

AI Tool	Location	Purpose
SNPE Runtime	backend/src/inference/snpe_runtime.py	Provides hardware acceleration for neural networks using Snapdragon NPU
QNN Runtime	backend/src/inference/qnn_runtime.py	Implements Qualcomm Neural Network SDK for optimized model execution
Model Manager	backend/src/inference/model_manager.py	Handles model loading, unloading, and switching between different AI models
Preprocessor	backend/src/inference/preprocessor.py	Prepares images for model input (resizing, normalization, etc.)
Postprocessor	backend/src/inference/postprocessor.py	Processes model output to generate final images

🔧 Technologies Used

Backend

• Framework: FastAPI
• Runtime: Python 3.8+
• AI Acceleration: SNPE (Snapdragon Neural Processing Engine), QNN (Qualcomm Neural Network)
• Dependencies: PyTorch, NumPy, OpenCV, Pillow
• API: RESTful endpoints, WebSockets for real-time updates

Frontend

• Framework: React Native with Expo
• Navigation: React Navigation
• UI Components: Custom components with Expo
• Form Handling: Formik + Yup
• Authentication: Custom auth context with secure storage

🚀 Setup Instructions

Prerequisites

• Qualcomm Innovators Development Kit (QIDK)
• Windows/Mac/Linux computer for development
• Node.js 16+ and npm/yarn
• Python 3.8+
• USB cable for connecting QIDK

Backend Setup (On QIDK)

1. Clone the repository

   git clone https://github.com/your-username/qidk-project.git
   cd qidk-project/backend

2. Create and activate Python virtual environment

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies

   pip install -r requirements.txt

4. Install SNPE SDK on QIDK

   # Follow instructions in backend/README.md for SNPE SDK installation

5. Download pre-trained models

   python scripts/download_production_models.py

6. Start the backend server

   python -m src.main

Frontend Setup (On Development Computer)

1. Navigate to the frontend directory

   cd qidk-project/frontend/thereRepo

2. Install dependencies

   npm install
   # or
   yarn install

3. Configure the API endpoint

   • Edit services/api.ts to point to your QIDK IP address or use ADB port forwarding

4. Start the development server

   npm start
   # or
   yarn start

5. Run on a device or emulator

   npm run android
   # or
   npm run ios

📱 Mobile App Features

The mobile application provides:

• User registration and authentication flow
• Gallery of AI style options
• Image upload functionality
• Real-time style transfer preview
• Image saving and sharing capabilities
• Settings for model parameters

🔌 API Documentation

The backend exposes several endpoints:

• POST /api/auth/register: Create new user account
• POST /api/auth/login: User login
• POST /api/auth/verify: OTP verification
• POST /api/generate: Submit image for AI processing
• GET /api/models: List available AI models
• GET /api/status: Get QIDK system status
• WS /api/ws: WebSocket for real-time updates

🛠️ Development Workflow

Adding New AI Models

1. Convert the model to SNPE/QNN format
2. Place model files in the appropriate directory
3. Register the model in the model manager
4. Implement preprocessing and postprocessing functions

Customizing the Frontend

The React Native frontend follows a component-based architecture:

• Screens are in frontend/thereRepo/screens/
• Reusable components are in frontend/thereRepo/components/
• Theme settings are in frontend/thereRepo/constants/theme.ts

📝 Project Structure

├── backend/                  # FastAPI server for QIDK
│   ├── scripts/              # Utility scripts
│   └── src/                  # Source code
│       ├── api/              # API routes
│       ├── inference/        # AI model inference
│       └── utils/            # Helper utilities
│
└── frontend/                 # React Native mobile app
    └── thereRepo/
        ├── app/              # Expo Router app directory
        ├── components/       # Reusable UI components
        ├── screens/          # App screens
        └── services/         # API integration

🧪 Testing

• Backend: Run pytest from the backend directory
• Frontend: Run npm test from the frontend directory

📈 Performance Considerations

The project is optimized for QIDK hardware:

• Models are quantized to INT8 for faster inference
• NPU acceleration is used when available
• Batch processing minimizes memory usage
• Response time is kept under 500ms for real-time feel

🔍 Future Improvements

• Add more AI model options
• Implement user galleries for generated content
• Add social sharing features
• Optimize model loading time
• Support offline mode

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgements

• Qualcomm for providing the QIDK hardware
• The PyTorch team for model optimization tools
• The FastAPI and React Native communities
• Perplexity for problem statement understanding (https://www.perplexity.ai/search/ok-i-am-in-a-hackathon-for-a-c-YKtdTubmTIKeqSPxupZeuQ#0)
• Copilot for quick coding and debugging.

---

This project was developed as part of Megathon 2025 at IIITH