ElderViews

[dawnglimmer]

Smart AI Glasses Designed for Seniors

Project Background📖

With the advancement of technology, smart wearable devices have become an indispensable part of daily life. AI glasses, as a new type of smart device, not only provide information queries and navigation functions but also enhance the quality and safety of users' lives through artificial intelligence technology. This project aims to develop AI glasses with intelligent interaction, reading assistance, item search, emergency rescue, smart navigation, and obstacle avoidance functions, specifically designed to meet the needs of elderly users, capable of recognizing dialects, and providing functions that facilitate their daily lives.

Introduction📜

Smart AI Glasses: A smart wearable device specifically designed for seniors, integrating natural language processing, computer vision, and machine learning technologies. It offers various functions such as intelligent interaction, reading assistance, item search, emergency rescue, smart navigation, and obstacle avoidance to enhance the quality and safety of seniors' lives.

Project Functions🌟

Intelligent Interaction

• Free Voice Conversation: Supports natural and fluent voice conversations with users, accurately understanding and responding to their needs, whether in Mandarin or local dialects.
• Life Advice: Provides personalized life advice based on the user's voice input, including diet, health, and entertainment.
• Smart Event Reminders: Automatically records and reminds users of important events and schedules.

Reading Assistance

• Smooth Text Reading: Offers high-quality text reading features, helping users easily access books, articles, and other textual content.
• Text Information Summarization: Summarizes textual content, extracting key information for quick comprehension.
• Text-Related Q&A: Allows users to ask questions about the text they are reading, with the system providing answers based on the content.

Item Search

• Record Item Information: Records information and storage locations of commonly used items.
• Inform Item Location: Quickly provides the storage location of items upon user request.
• Describe Colors and Scenes: Recognizes and describes the colors and scenes of the surroundings to help users locate items.

Emergency Rescue

• Automatic Anomaly Detection: Continuously monitors the user's health status and surroundings, automatically detecting anomalies.
• Provide Handling Solutions: Offers emergency handling solutions and guides users in self-rescue or mutual aid during anomalies.
• Voice Dialing/Contact Emergency Contact/Call 120: Automatically dials 120 or contacts preset emergency contacts for voice assistance in emergencies.

Smart Navigation

• Intelligent Route Planning: Provides optimal route planning based on the user's destination needs, including walking and public transportation options.
• Walking and Public Transit Navigation: Real-time navigation guiding users to their destination by walking or using public transit.
• Nearby Information Queries: Offers services for querying nearby information such as restaurants, shopping centers, and attractions.

Obstacle Avoidance

• Road Obstacle Detection: Real-time detection of obstacles on the user's path.
• Obstacle Distance Estimation: Estimates the distance of obstacles and provides distance information.
• Advance Voice Warnings: Issues advance voice warnings when detecting obstacles to avoid collisions and falls.

Project Implementation Technologies🤖

This project utilizes the InternVL version of the large language model and includes the following technical components:

• Natural Language Processing (NLP): To achieve accurate and natural conversation, the system uses advanced NLP algorithms to understand and generate human language.
• Computer Vision: For reading assistance, item search, and obstacle avoidance functions, the system employs state-of-the-art computer vision technology to interpret visual information.
• Machine Learning: Personalized life advice and anomaly detection rely on machine learning models trained on vast amounts of data.
• Speech Recognition and Synthesis: Integrates high-precision speech recognition and synthesis systems to support voice interaction and real-time notifications.

Data Utilization🚀

• Voice Dataset: Includes various voices and dialects for training speech recognition and synthesis models.
• Text Dataset: Contains extensive textual content for training reading assistance and text summarization features.
• Image and Video Dataset: Used to train computer vision models for item search and obstacle avoidance functions.
• Health and Activity Data: Collected from users to improve the accuracy of anomaly detection and personalized advice.

Reference 📚

This project uses information from the following sources:

• XTuner: A Toolkit for Efficiently Fine-tuning LLM

• InternLM2.5: A Multilingual Language Model with Progressively Enhanced Capabilities

• 书生大模型实战营

Acknowledgement 🙏

Special thanks to Shanghai Artificial Intelligence Laboratory for its support of this project.