[DMP 2026]: Building a Voice Based Conversational AI System for Government School Students #3
Description
Ticket Contents
Description
Many government school students have limited typing ability and digital literacy, making voice-first interfaces significantly more accessible than text-based ones. TAP currently uses tools such as ElevenLabs to enable voice-based interaction, but for large-scale government deployment, the system must work across multiple Indian languages, be cost-efficient, and integrate with India's digital public infrastructure such as Bhashini. This project involves building and fine-tuning a voice-first conversational AI system that can understand student voice inputs, respond conversationally in regional languages, and guide students through learning experiences.
Goals & Mid-Point Milestone
Goals
- Design the end-to-end voice pipeline (speech input → conversational AI → speech output)
- Research and document the use of Small Multilingual Language Models (SMLs) for specific Indian language conversations
- Explore and implement Bhashini API integration for speech recognition and translation as an alternative
- Fine-tune open-source conversational AI models using TAP's conversational datasets
- Optimize the pipeline for low latency and cost-efficient large-scale deployment
- [Goals Achieved By Mid-point Milestone]: End-to-end voice pipeline functional in at least one language (e.g., Hindi) with a fine-tuned conversational model responding to basic learning queries
Setup/Installation
No response
Expected Outcome
Voice-first conversational learning pipeline
Multilingual conversational interaction using fine tuned language specific SMLs or Bhashini
Fine-tuned conversational model for learning use cases
Documentation for large-scale GovTech deployment
Acceptance Criteria
No response
Implementation Details
- Speech-to-text and text-to-speech: Bhashini APIs or fine-tuned Small Multilingual Language Models (SMLs)
- Conversational AI: open-source LLMs fine-tuned on TAP's conversational datasets using PyTorch
- Multilingual support: Hindi (required), Marathi and Punjabi (stretch goals)
- Voice learning workflows: enable question-answering and conversational exploration of learning content
- Performance optimization for low latency and cost efficiency at scale
Mockups/Wireframes
No response
Product Name
Voice-First Conversational AI System for Government School Students
Organisation Name
The Apprentice Project
Domain
Education
Tech Skills Needed
Natural Language Processing, Artificial Intelligence, Machine Learning, Python
Mentor(s)
TBD
Category
Machine Learning