Add a new WebTool: AI Chat WebTool #264

esatiyev · 2025-07-24T09:07:01Z

This pull request contains the current working version of the AI Chat WebTool being developed under Google Summer of Code 2025.

The implementation builds upon prior work by @rmackay9 and other contributors. My additions are integrated with that foundation and reflect the state of development so far.

All changes have been tested locally. Feedback is welcome.

Copilot

Pull Request Overview

This pull request introduces an AI Chat WebTool for Google Summer of Code 2025 that enables natural language control of ArduPilot vehicles through OpenAI's assistant API. The implementation builds upon prior work and integrates MAVLink communication for real-time vehicle control.

Key changes include:

Creation of a complete chat interface with OpenAI integration for vehicle control commands
Implementation of MAVLink messaging and parameter handling for vehicle communication
Addition of voice-to-text functionality for hands-free operation

Reviewed Changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
Chat/index.html	Main application interface with chat UI, MAVLink integration, and OpenAI assistant functionality
Chat/style.css	CSS styling for user and assistant chat message bubbles
Chat/shared.js	Shared module for exporting MAVLink instances across components
Chat/mavlink_msgs.js	Auto-generated MAVLink message definitions and component mappings
Chat/Readme.md	Basic documentation describing the OpenAI-based chat module
Chat/MAVLink/mavlink_store.js	MAVLink message storage and retrieval system for caching vehicle data
Chat/MAVLink/local_modules/long/*	Local copy of the Long.js library for 64-bit integer operations

Chat/index.html

Chat/mavlink_msgs.js

Chat/index.html

…msg typo; changed label's for to match input id

- Added and Fixed functions to set, get, delete, and check wakeup timers - Implemented get_mode_mapping function - Fixed minor UI design issue - Updated handleTranscription to add FormData field language=en to enforce English in Whisper transcription

…UI change - Added case 22 in mavlink_msg_handler for handling PARAM_VALUE messages - Converted handle_function_call to async - Added functions for use in handle_function_call: - get_parameter - get_all_parameters - get_parameter_description - set_parameter - get_mavlink_message - get_available_mavlink_messages - Made small UI adjustments - Made small UI adjustments

…perties of null (reading 'apiKey') Added a new “Connect AI” button so users can initialize the assistant before sending messages Linked the button to the connection routine and exposed the function globally for other modules Updated audio transcription to ensure a connection to OpenAI exists before sending the audio for processing

- Add "Start New Chat" button and support for restoring previous chat sessions with thread id - Display prompt to continue or reset previous sessions on page load - Introduce chat busy state to prevent race conditions; disable UI controls during chat operations - Remove unnecessary functions for arming/disarming vehicle - Enhance parameter metadata loading - Clean up labels, error handling, and general code structure

khanasif786 · 2025-08-03T13:51:52Z

@esatiyev We can add tool in the Development section "Work is Progress Tools"

esatiyev · 2025-08-05T14:54:57Z

@khanasif786 Done, thanks

rmackay9 · 2025-08-07T00:58:18Z

@esatiyev, could you post a picture of the webtool in the description at the top? That will let people know more quickly what it looks like.

rmackay9

Thanks very much for this!

IamPete1 · 2025-08-12T11:22:57Z

There is a bit of MAVLink re-work in #267 too. It would be good if they could converge to keep the lib in the same place and use it in the same way.

IamPete1

I think Chat.js and mavlink_msgs.js can just be removed?

Chat/Chat.js

esatiyev · 2025-08-14T11:51:50Z

I think Chat.js and mavlink_msgs.js can just be removed?

I have removed the unnecessary files as suggested in commit 06aab8b . Thanks.

and added mavftp (cherry picked from commit 90dfb75)

…Link

esatiyev · 2025-08-14T15:38:37Z

There is a bit of MAVLink re-work in #267 too. It would be good if they could converge to keep the lib in the same place and use it in the same way.

I’ve moved MAVLink to modules, same as in PR #267, and updated my index.html file to use it.

… path

Chat/MAVLink/mavlink_store.js

Chat/index.html

shiv-tyagi · 2025-08-02T15:17:11Z

Chat/index.html

+  <!-- script for Mavlink export  -->
+  <script src="shared.js"></script>
+  <!-- simple script functions local to this page -->
+  <script type="module">


Let's move this script to a separate file. It would improve the performance and readability.

shiv-tyagi · 2025-08-02T15:19:20Z

Chat/index.html

+        eyeIconContainer.innerHTML = eyeClosedSVGPath;
+      }
+    }
+    // listener for toggle api key visibility button click


Suggested change

// listener for toggle api key visibility button click

// listener for toggle api key visibility button click

shiv-tyagi · 2025-08-02T15:58:16Z

Chat/index.html

+    }
+
+    // attach event listener to the connect button
+    if (connect_button) {


Is the if statement needed?

shiv-tyagi · 2025-08-02T16:04:03Z

Chat/index.html

+    document.getElementById("mavlink-disarm-button").addEventListener("click", mavlink_disarm_vehicle);
+
+    // add text to debug text box
+    function add_text_to_debug(text) {


Can we also put an auto-scroll toggle button?

This is not for this PR

shiv-tyagi · 2025-08-02T16:11:22Z

Chat/index.html

+        const new_thread = await openai.beta.threads.create();
+        return new_thread ? new_thread.id : null;
+      } catch (error) {
+        return null;


Should we also log the error to console?

rmackay9 added 2 commits July 24, 2025 17:44

Chat: openai chat tool

d0118e4

Chat: add MAVLink support

4773e63

rmackay9 requested a review from Copilot July 31, 2025 07:24

Copilot AI reviewed Jul 31, 2025

View reviewed changes

IamPete1 and others added 13 commits August 3, 2025 21:24

Chat: mess about with type to mavlink send to work

379fd51

Chat: add mavlink store and handle function calls

5f0a988

Chat: Fix send button by adding proper event listener

873b491

Chat: added get_vehicle_location_and_yaw definition

46fdf1e

Chat: implemented get_vehicle_state function's definition

8dc5a88

Chat: fix error with api key visibility toggle button; fix heartbeat_…

586a848

…msg typo; changed label's for to match input id

Chat: added support for recording functionality

5a741d0

Chat: Refactor chat: update layout, add shared.js and style.css

3a84de6

Chat: add wakeup timers, fix UI bug

817791a

- Added and Fixed functions to set, get, delete, and check wakeup timers - Implemented get_mode_mapping function - Fixed minor UI design issue - Updated handleTranscription to add FormData field language=en to enforce English in Whisper transcription

Chat: Refactor style.css: Remove unused styles

2384c04

Chat: Added find_message_by_name function

dc915ab

esatiyev force-pushed the chat-webtool branch from e27e175 to b9f9e69 Compare August 3, 2025 12:51

esatiyev force-pushed the chat-webtool branch from a69cb18 to 4d1482f Compare August 3, 2025 13:40

esatiyev force-pushed the chat-webtool branch from 8928059 to 4d1482f Compare August 5, 2025 14:45

Dev: add Chat link including icon

31d7b35

rmackay9 approved these changes Aug 7, 2025

View reviewed changes

IamPete1 requested changes Aug 12, 2025

View reviewed changes

Chat/Chat.js Outdated Show resolved Hide resolved

Chat/Chat.js Outdated Show resolved Hide resolved

Chat: Remove unused Chat.js and mavlink_msgs.js

06aab8b

tridge and others added 3 commits August 14, 2025 23:28

modules: moved MAVLink to modules directory

bd416a5

and added mavftp (cherry picked from commit 90dfb75)

modules: moved mavlink_store.js from Chat/MAVLink to modules/MAVLink

8dc58e8

Chat: remove old MAVLink module and update imports to use modules/MAV…

6803bc6

…Link

Chat: align send method with new mavlink module and update mavlink.js…

c8e242f

… path

esatiyev requested a review from IamPete1 August 19, 2025 01:35

esatiyev changed the title ~~AI Chat WebTool for GSoC 2025: Current Progress~~ Add a new WebTool: AI Chat WebTool Aug 21, 2025

shiv-tyagi suggested changes Aug 21, 2025

View reviewed changes

module: enforce singleton pattern for MAVLinkMessageStore

3d12c93

esatiyev mentioned this pull request Aug 22, 2025

Add AI Chat WebTool #269

Open

	// listener for toggle api key visibility button click

	// listener for toggle api key visibility button click

Uh oh!

Add a new WebTool: AI Chat WebTool #264

Are you sure you want to change the base?

Add a new WebTool: AI Chat WebTool #264

Uh oh!

Conversation

esatiyev commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

khanasif786 commented Aug 3, 2025

Uh oh!

esatiyev commented Aug 5, 2025

Uh oh!

rmackay9 commented Aug 7, 2025

Uh oh!

rmackay9 left a comment

Choose a reason for hiding this comment

Uh oh!

IamPete1 commented Aug 12, 2025

Uh oh!

IamPete1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

esatiyev commented Aug 14, 2025

Uh oh!

esatiyev commented Aug 14, 2025

Uh oh!

Uh oh!

Uh oh!

shiv-tyagi Aug 2, 2025

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Aug 2, 2025

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Aug 2, 2025

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Aug 2, 2025

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Aug 21, 2025

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Aug 2, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

esatiyev commented Jul 24, 2025 •

edited

Loading