add achitectural overview

.github/copilot-instructions.md

@@ -0,0 +1,70 @@
+# Copilot / AI agent instructions for the Kusto Explorer VS Code repo
+
+These instructions guide AI coding agents (and the developers driving them) when working **on this
+codebase**. They are not the end-user KQL assistant prompt — that lives in
+[`src/Client/instructions.md`](../src/Client/instructions.md) and is shipped to users at runtime. Keep
+the two separate.
+
+For the full "why" behind everything below, read [`ARCHITECTURE.md`](../ARCHITECTURE.md). When a change
+contradicts these rules, prefer the architecture doc and update both.
+
+## What this project is
+
+A VS Code extension that mimics the desktop **Kusto Explorer** app: edit multi-query `.kql` files, run
+queries, and view results as tables and charts. Scope is intentionally **partial**; the roadmap is
+driven by community feedback, not a fixed plan. Don't invent features — match how the desktop app feels
+and prefer small, focused changes.
+
+## Architecture rules an agent must respect
+
+- **Two processes, on purpose.** A TypeScript extension host (`src/Client`) talks to a C# .NET
+  language server (`src/Server`) over LSP (StreamJsonRpc on stdio) plus custom `kusto/*` JSON-RPC
+  methods. KQL parsing/analysis lives in the **C# server** because the native `Kusto.Language` parser
+  is ~50x faster than the JS translation. **Do not** reimplement KQL parsing/analysis in TypeScript,
+  and add cross-process calls through the existing LSP / `kusto/*` seam — not new side channels.
+- **Data model vs. UI component split.** Each feature is usually two modules: a testable
+  `*Manager`-style **data model** (state + logic, no VS Code UI) and a `*Panel` / `*Editor` / `*Provider`
+  **UI component**. Put new logic where it can be unit-tested without a webview. A filename like
+  `panel`, `editor`, `viewer`, or `statusBar` means it's the UI half.
+- **Provider model for result content.** The results editor delegates visually distinct content
+  (tabular, chart, query text) to providers that each emit HTML into their own `<div>`. Charts sit
+  behind a single `IChartProvider` seam with multiple implementations (`plotly`, `timePivot`, `graph`)
+  joined by `compositeChartProvider`. Add new result/chart surfaces behind this seam — don't bolt them
+  onto the editor.
+- **Webview house style.** Webview UIs build an HTML **string** with inline `<script>`, use
+  `acquireVsCodeApi()` + document-level click delegation, and post messages back to the extension.
+  There is **no React/bundler** for webview content; CDN libraries (Plotly, Cytoscape, simple-datatables)
+  are loaded directly. Match this pattern; don't introduce a webview framework.
+- **Interface-first seams.** `IServer`, `IConnection`, `ISchemaSource`, `IChartProvider`,
+  `IDataTableProvider`, etc. exist so implementations can be swapped or stubbed. Preserve them and
+  provide Null/test variants for new seams.
+- **Schema is cached at two levels.** In-memory (session) in `SchemaManager` plus VS Code storage
+  (cross-session, via the `kusto/getData`/`kusto/setData` → `globalState` bridge) so startup is usable
+  immediately, with background reconciliation updating both copies. Don't bypass this cache.
+
+## Build, run, and test
+
+Client commands run from `src/Client`:
+
+- Build the extension: `npm run compile`
+- Type-check: `npm run type-check`
+- Lint (zero warnings): `npm run lint`
+- Unit tests (vitest): `npm test` (or `npm run test:unit`)
+- Integration tests: `npm run test:integration`
+- Build the debug server: `npm run build-debug-server` (release: `build-release-server`)
+- `F5` builds the client + debug server and launches the Extension Development Host.
+
+Server commands run from `src/Server` (build) and `ServerTests` (tests):
+
+- Build the server: `dotnet build`
+- Run server tests: `dotnet test` from the `ServerTests` project.
+
+Always run the relevant tests and ensure the build passes before considering a change done.
+
+## Working style
+
+- Read [`ARCHITECTURE.md`](../ARCHITECTURE.md) before structural changes; keep it in sync when you
+  change components, seams, or conventions.
+- Keep changes minimal and focused; don't refactor or add abstractions that weren't asked for.
+- Prefer editing existing modules over adding new ones, and keep logic in the testable data-model half.
+- Validate your own work with the build + tests above rather than assuming success.

CONTRIBUTING.md

@@ -81,6 +81,12 @@ Only after that process has resulted in an "open for contribution" status, shoul
 
 If a pull request arrives without prior discussion for a change that needs design or roadmap approval, it may be closed and redirected to Discussions.
 
+## Architecture
+
+For a component-level overview of how the extension works — the client/server split, the LSP +
+custom `kusto/*` RPC seam, how a query runs end-to-end, and the design intent behind the
+structure — see [ARCHITECTURE.md](ARCHITECTURE.md).
+
 ## Repository Structure
 
 | Path | Description |

src/Server/Diagnostics/DiagnosticsManager.cs

@@ -1,15 +1,22 @@
-// Copyright (c) Microsoft Corporation.
-// Licensed under the MIT license.
-
-using System.Collections.Immutable;
-using System.Runtime.CompilerServices;
-using Kusto.Language;
-using Kusto.Language.Editor;
-
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+using System.Collections.Immutable;
+using System.Runtime.CompilerServices;
+using Kusto.Language;
+using Kusto.Language.Editor;
+
 namespace Kusto.Vscode;
 
 public record DiagnosticInfo(Uri Id, string Text, ImmutableList<Diagnostic> Diagnostics);
 
+/// <summary>
+/// Turns document changes into published diagnostics. It listens to
+/// <see cref="IDocumentManager"/> add/change events, recomputes parse + analyzer diagnostics for
+/// the affected document, and raises <see cref="DiagnosticsUpdated"/> for the server to forward as an
+/// LSP <c>publishDiagnostics</c> notification. Recomputation is debounced/serialized so a burst of
+/// keystrokes collapses into a single up-to-date result rather than a flood of stale ones.
+/// </summary>
 public class DiagnosticsManager : IDiagnosticsManager
 {
     private readonly IDocumentManager _documentManager;

src/Server/Documents/DocumentManager.cs

@@ -1,42 +1,50 @@
-// Copyright (c) Microsoft Corporation.
-// Licensed under the MIT license.
-
-using Kusto.Language.Editor;
-using Kusto.Language.Symbols;
-using System.Collections.Immutable;
-using System.Diagnostics.CodeAnalysis;
-
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+using Kusto.Language.Editor;
+using Kusto.Language.Symbols;
+using System.Collections.Immutable;
+using System.Diagnostics.CodeAnalysis;
+
 namespace Kusto.Vscode;
 
-public class DocumentManager : IDocumentManager
-{
+/// <summary>
+/// Tracks every open editor document (keyed by URI) and keeps its parsed
+/// <see cref="Kusto.Language.Editor.CodeScript"/> in sync with two moving inputs: the document text
+/// (as the user types) and the shared symbols (when <see cref="ISymbolManager.GlobalsChanged"/>
+/// fires). It also remembers each document's cluster/database connection context so language
+/// services analyze the query against the right schema. Editor features (diagnostics, completion,
+/// hover, etc.) read documents from here rather than re-parsing, so parsing happens once per change.
+/// </summary>
+public class DocumentManager : IDocumentManager
+{
     /// <summary>
     /// The maximum number of times the resolve operation
     /// can repeat resolving as long as symbols are updating.
-    /// </summary>
+    /// </summary>
     private const int MaxResolveLoopCount = 20;
 
-    private readonly ISymbolManager _symbolManager;
-    private readonly ILogger? _logger;
-
-    private ImmutableDictionary<Uri, DocumentInfo> _idToInfoMap =
-        ImmutableDictionary<Uri, DocumentInfo>.Empty;
-
-    public DocumentManager(
-        ISymbolManager symbolManager, 
-        ILogger? logger = null)
-    {
-        _symbolManager = symbolManager;
-        _logger = logger;
-
-        // update scripts when global symbols change
-        _symbolManager.GlobalsChanged += (_, _) =>
-        {
-            foreach (var id in _idToInfoMap.Keys)
-            {
-                UpdateGlobals(id);
-            }
-        };
+    private readonly ISymbolManager _symbolManager;
+    private readonly ILogger? _logger;
+
+    private ImmutableDictionary<Uri, DocumentInfo> _idToInfoMap =
+        ImmutableDictionary<Uri, DocumentInfo>.Empty;
+
+    public DocumentManager(
+        ISymbolManager symbolManager, 
+        ILogger? logger = null)
+    {
+        _symbolManager = symbolManager;
+        _logger = logger;
+
+        // update scripts when global symbols change
+        _symbolManager.GlobalsChanged += (_, _) =>
+        {
+            foreach (var id in _idToInfoMap.Keys)
+            {
+                UpdateGlobals(id);
+            }
+        };
     }
 
     public event EventHandler<Uri>? DocumentAdded;
@@ -323,7 +331,7 @@ public async Task RefreshReferencedSymbolsAsync(Uri documentId, CancellationToke
                 // Refresh the database symbol
                 await _symbolManager.RefreshAsync(dbRef.Cluster, dbRef.Database, cancellationToken).ConfigureAwait(false);
             }
-        }
+        }
     }
 
     /// <summary>
@@ -384,18 +392,18 @@ private void UpdateGlobals(DocumentInfo info, bool raiseDocumentChanged)
         }
     }
 
-    /// <summary>
-    /// Gets the document for the given id.
-    /// </summary>
-    public bool TryGetDocument(Uri documentId, [NotNullWhen(true)] out IDocument? document)
-    {
-        if (_idToInfoMap.TryGetValue(documentId, out var info))
-        {
-            UpdateGlobals(info, raiseDocumentChanged: true);
-            document = info.Document;
-            return true;
-        }
-        document = null;
-        return false;
-    }
+    /// <summary>
+    /// Gets the document for the given id.
+    /// </summary>
+    public bool TryGetDocument(Uri documentId, [NotNullWhen(true)] out IDocument? document)
+    {
+        if (_idToInfoMap.TryGetValue(documentId, out var info))
+        {
+            UpdateGlobals(info, raiseDocumentChanged: true);
+            document = info.Document;
+            return true;
+        }
+        document = null;
+        return false;
+    }
 }

src/Server/Querying/QueryManager.cs

@@ -10,6 +10,15 @@
 
 namespace Kusto.Vscode;
 
+/// <summary>
+/// Orchestrates query validation, result-typing, and execution. This is the seam where
+/// the three independent subsystems meet for a single request: it asks <see cref="ISymbolManager"/>
+/// to ensure the cluster/database symbols are loaded (so the parser can analyze the query),
+/// resolves an <see cref="IConnection"/> from <see cref="IConnectionManager"/>, then runs the
+/// query and wraps the tables/charts/diagnostics into a single result. It also handles inline
+/// directives (e.g. <c>#connect</c>) before execution. Kept deliberately thin: it coordinates,
+/// it does not parse, cache schema, or own connections.
+/// </summary>
 public class QueryManager : IQueryManager
 {
     private readonly IConnectionManager _connectionManager;

src/Server/Server.cs

@@ -16,6 +16,18 @@
 
 namespace Kusto.Vscode;
 
+/// <summary>
+/// The language server entry point and coordinator. It extends <see cref="LspServer"/> (the
+/// StreamJsonRpc/LSP plumbing) to implement the standard editor features as well as the custom
+/// <c>kusto/*</c> RPC methods (run query, fetch schema, decode connection strings, etc.), delegating
+/// each one to the appropriate manager. It deliberately owns no domain logic itself; instead it wires
+/// the managers together and implements the small host-facing seams
+/// (<see cref="ILogger"/>, <see cref="ISettingSource"/>, <see cref="IStorage"/>,
+/// <see cref="IAuthenticationProvider"/>) that bridge server concerns back to the VS Code client —
+/// logging, configuration, persistence, and AAD token acquisition all live in the host window.
+/// Two constructors exist: one for dependency injection (tests) and one that composes the concrete
+/// manager graph for normal startup.
+/// </summary>
 public class Server : LspServer, ILogger, ISettingSource, IStorage, IAuthenticationProvider
 {
     private readonly ILogger _logger;

src/Server/Symbols/SymbolManager.cs

@@ -1,13 +1,21 @@
-// Copyright (c) Microsoft Corporation.
-// Licensed under the MIT license.
-
-using Kusto.Language;
-using Kusto.Language.Editor;
-using Kusto.Language.Symbols;
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+using Kusto.Language;
+using Kusto.Language.Editor;
+using Kusto.Language.Symbols;
 using System.Collections.Immutable;
 
 namespace Kusto.Vscode;
 
+/// <summary>
+/// Owns the shared <see cref="Kusto.Language.Symbols.GlobalState"/> that the Kusto parser needs to
+/// resolve names (clusters, databases, tables, functions). It turns the raw schema from
+/// <see cref="ISchemaManager"/> into parser symbols and publishes a single, immutable globals
+/// snapshot via <see cref="GlobalsChanged"/>, which downstream consumers (documents, diagnostics,
+/// completion) react to. All loads run through a <see cref="TaskQueue"/> so concurrent requests
+/// can't race while mutating the globals — the global state has exactly one writer.
+/// </summary>
 public class SymbolManager : ISymbolManager
 {
     private readonly ISchemaManager _schemaManager;