vocabARy 📷🌐

Learn vocabulary in a natural way by pointing your camera at objects around you.

vocabARy is an augmented reality vocabulary learning application that uses real-time object detection to help users learn words in multiple languages. Simply point your camera at everyday objects, and the app will identify them and teach you their names in your chosen language.

🎓 Course Project

This project was developed as part of the AR and VR (Augmented Reality and Virtual Reality) course.

✨ Features

• 🔍 Real-Time Object Detection - Uses YOLOv5 to identify objects through your webcam
• 🌍 Multi-Language Support - Learn words in Tamil, Telugu, Kannada, Hindi, French, and Japanese
• 🎤 Voice Recognition - Practice pronunciation with speech recognition and get instant feedback
• 📝 Pronunciation Guides - Transliterated pronunciations for non-Latin scripts
• 🎮 Gamified Learning - Earn points for each word you learn correctly
• 📊 Progress Tracking - Track your learning journey with levels and word counts
• 🎉 Confetti Celebrations - Fun animations when you pronounce words correctly
• 💾 Persistent Storage - All detected objects are saved for later review

🛠️ Tech Stack

• Backend: Django 5.1
• Database: SQLite
• Object Detection: YOLOv5 (PyTorch Hub)
• Translation: Google Translator (deep-translator)
• Transliteration: indic-transliteration, pykakasi
• Frontend: HTML, CSS, JavaScript
• Speech Recognition: Web Speech API

📦 Installation

Prerequisites

• Python 3.8 or higher
• Webcam
• pip (Python package manager)

Setup

1. Clone the repository

   git clone https://github.com/yourusername/vocabARy.git
   cd vocabARy

2. Create a virtual environment

   python -m venv venv
   
   # On Windows
   venv\Scripts\activate
   
   # On macOS/Linux
   source venv/bin/activate

3. Install dependencies

   pip install django torch torchvision opencv-python deep-translator indic-transliteration pykakasi pandas

4. Run database migrations

   python manage.py migrate

5. Start the development server

   python manage.py runserver

6. Open your browser and navigate to http://127.0.0.1:8000

🚀 Usage

1. Home Page - Overview of features and your current progress
2. Start Learning - Click the button to open the detection view
3. Detection View - Point your camera at objects; detected items appear in the sidebar
4. My Lists - View all detected objects as cards
5. Learn Words - Click any card to see translation and pronunciation
6. Practice - Hold the microphone button and speak the translation to earn points

📁 Project Structure

vocabARy/
├── events/                 # Main Django app
│   ├── models.py          # DetectedObject & Score models
│   ├── views.py           # View functions & API endpoints
│   ├── urls.py            # URL routing
│   ├── admin.py           # Admin panel configuration
│   └── static/
│       └── styles.css     # Application styles
├── templates/
│   ├── index.html         # Home page
│   ├── det.html           # Detection/camera view
│   └── list.html          # Word list with cards
├── voc/                   # Django project settings
│   ├── settings.py
│   └── urls.py
├── manage.py
├── db.sqlite3
├── LICENSE
└── README.md

🔌 API Endpoints

Endpoint	Method	Description
/	GET	Home page
/detect/	GET	Camera detection view
/list/	GET	Word cards list
/video_feed/	GET	Live video stream
/get_objects/	GET	List of detected objects
/translate/	GET	Translate a word
/update_score_and_learn/	POST	Update score and mark word as learnt
/get_stats/	GET	Get current points and word count

👥 Team

This project was built with ❤️ by:

• Saran
• Shreeja
• Manish
• Nihal

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• YOLOv5 for object detection
• deep-translator for translation services
• indic-transliteration for Indian language transliteration
• pykakasi for Japanese romaji conversion

---

vocabARy - Learning languages, one object at a time 🎯