Machine Learning Internship at visionNLP

Project: Machine Learning Theoretical Interview Chatbot

Overview

This project focuses on building a chatbot for conducting machine learning theory-based interviews. Two models were implemented: a custom Encoder-Decoder RNN and a GPT-2-based model. Both models were trained on a dataset created by web scraping machine learning-related websites and blogs.

Models

1. Custom Encoder-Decoder RNN Model:

   • Developed from scratch
   • Understands and generates text responses
   • Trained on the machine learning theory dataset

2. GPT-2-Based Model:

   • Utilizes pre-trained GPT-2 architecture
   • Leverages powerful text generation capabilities
   • Trained on the machine learning theory dataset

Dataset Creation

Web Scraping Process

1. Data Collection: Gathered machine learning theory-related questions and answers from reputable online sources, including websites, blogs, and forums.

2. Source Selection: Carefully selected websites and platforms known for quality machine learning content.

3. Web Scraping: Automated data extraction using Python libraries like Beautiful Soup and Scrapy. Custom scripts visited selected websites, extracted relevant text content, and saved it for dataset creation.

Training the Models:

• Fine-tune the GPT-2 model using the dataset.
• Train the custom Encoder-Decoder RNN model.

Conclusion

The Machine Learning Theoretical Interview Chatbot project simplifies candidate screening for machine learning roles. The custom and GPT-2 models offer efficient solutions for evaluating candidates' theoretical knowledge in the field.