Evaluate Apple Core AI models as a local TypeWhisper backend

[SeoFood]

Problem / Motivation

Apple has published the apple/coreai-models repository with Core AI model export recipes, Python tooling, Swift runtime utilities, and ready-to-run model integration examples.

TypeWhisper already supports several local and cloud model paths, including bundled MLX-based LLM plugins. We should evaluate whether Core AI models can become a useful local backend for TypeWhisper, especially for private on-device prompt processing and future local model workflows.

Reference:
https://github.com/apple/coreai-models

Apple’s current documented requirements are macOS/iOS 27.0+ and Xcode 27.0+, so this should start as a compatibility and feasibility spike rather than a user-facing integration.

Proposed Solution

Test the Core AI models repository against TypeWhisper’s local model/plugin architecture:

• Identify which shipped Core AI model families are relevant to TypeWhisper workflows.
• Export or download at least one small model and run it locally on a supported Mac.
• Verify basic Swift integration using the Core AI runtime utilities.
• Compare runtime behavior against the existing local MLX plugin approach:
  • installation/setup complexity
  • first-run specialization or compilation cost
  • model load latency
  • memory footprint
  • inference latency
  • packaging and distribution implications
  • sandboxing/notarization implications
• Decide whether the right next step is:
  • no action for now
  • internal experimental plugin only
  • public plugin support
  • shared abstraction for Core AI local models

Alternatives Considered

• Continue relying on the existing MLX plugin path for local LLMs.
• Wait until Core AI platform requirements are closer to stable user hardware/software.
• Treat Core AI as a separate future backend instead of trying to merge it into current MLX plugin code.

Acceptance Criteria

• A short written feasibility note is attached to this issue or linked from a PR.
• At least one Core AI model is tested end-to-end locally, if the current macOS/Xcode environment supports it.
• The spike records whether Core AI can realistically fit TypeWhisper’s plugin system.
• Any follow-up implementation issue is scoped narrowly around one concrete integration path.