Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
94 changes: 42 additions & 52 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,49 +26,43 @@ npm link

## Adding a New Site Adapter

This is the most common type of contribution. Start with YAML when possible, and use TypeScript only when you need browser-side logic or multi-step flows.

### YAML Adapter (Recommended for data-fetching commands)

Create a file like `clis/<site>/<command>.yaml`:

```yaml
site: mysite
name: trending
description: Trending posts on MySite
domain: www.mysite.com
strategy: public # public | cookie | header
browser: false # true if browser session is needed

args:
query:
positional: true
type: str
required: true
description: Search keyword
limit:
type: int
default: 20
description: Number of items

pipeline:
- fetch:
url: https://api.mysite.com/trending

- map:
rank: ${{ index + 1 }}
title: ${{ item.title }}
score: ${{ item.score }}
url: ${{ item.url }}

- limit: ${{ args.limit }}

columns: [rank, title, score, url]
All adapters use TypeScript. Use the pipeline API for data-fetching commands, and `func()` for complex browser interactions.

### Pipeline Adapter (Recommended for data-fetching commands)

Create a file like `clis/<site>/<command>.ts`:

```typescript
import { cli, Strategy } from '@jackwener/opencli/registry';

cli({
site: 'mysite',
name: 'trending',
description: 'Trending posts on MySite',
domain: 'www.mysite.com',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'query', positional: true, required: true, help: 'Search keyword' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of items' },
],
columns: ['rank', 'title', 'score', 'url'],
pipeline: [
{ fetch: { url: 'https://api.mysite.com/trending' } },
{ map: {
rank: '${{ index + 1 }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
url: '${{ item.url }}',
}},
{ limit: '${{ args.limit }}' },
],
});
```

See [`hackernews/top.yaml`](clis/hackernews/top.yaml) for a real example.
See [`hackernews/top.ts`](clis/hackernews/top.ts) for a real example.

### TypeScript Adapter (For complex browser interactions)
### func() Adapter (For complex browser interactions)

Create a file like `clis/<site>/<command>.ts`:

Expand Down Expand Up @@ -114,7 +108,7 @@ Use `opencli explore <url>` to discover APIs and see [opencli-explorer skill](./
### Validate Your Adapter

```bash
# Validate YAML syntax and schema
# Validate adapter
opencli validate

# Test your command
Expand All @@ -137,16 +131,12 @@ Use **positional** for the primary, required argument of a command (the "what"

Do **not** convert an argument to positional just because it appears first in the file. If the argument is optional, acts like a filter, or selects a mode/configuration, it should usually stay a named option.

YAML example:
```yaml
args:
query:
positional: true # ← primary arg, user types it directly
type: str
required: true
limit:
type: int # ← config arg, user types --limit 10
default: 20
Pipeline example:
```typescript
args: [
{ name: 'query', positional: true, required: true, help: 'Search query' }, // ← primary arg
{ name: 'limit', type: 'int', default: 20, help: 'Max results' }, // ← config arg
]
```

TS example:
Expand Down Expand Up @@ -198,7 +188,7 @@ Common scopes: site name (`twitter`, `reddit`) or module name (`browser`, `pipel
npx tsc --noEmit # Type check
npm test # Core unit tests
npm run test:adapter # Focused adapter tests (if you touched adapter logic)
opencli validate # YAML validation (if applicable)
opencli validate # Adapter validation
```
4. Commit using conventional commit format
5. Push and open a PR
Expand Down
30 changes: 21 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,24 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other

## Why OpenCLI

- **One mental model**: use the same CLI for websites, browser automation, Electron apps, and local tools.
- **Reuse real sessions**: browser-backed commands run against your existing Chrome/Chromium login state instead of reimplementing auth.
- **Deterministic outputs**: adapters return stable, scriptable structures that work well in shells, CI, and AI-agent tool use.
- **AI-agent ready**: `browser` handles live control, `explore` discovers APIs, `synthesize` drafts adapters, and `cascade` probes auth strategies.
- **Low runtime cost**: no model tokens are consumed when running existing commands.
- **Extensible by default**: keep built-ins, register local CLIs, or drop `.ts` / `.yaml` adapters into `clis/`.
---

## Highlights

- **CLI All Electron** — CLI-ify apps like Antigravity Ultra! Now AI can control itself natively.
- **Browser Automation** — `browser` gives AI agents direct browser control: click, type, extract, screenshot — any interaction, fully scriptable.
- **Website → CLI** — Turn any website into a deterministic CLI: 70+ pre-built adapters, or crystallize your own with `opencli record`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **Anti-detection built-in** — Patches `navigator.webdriver`, stubs `window.chrome`, fakes plugin lists, cleans ChromeDriver/Playwright globals, and strips CDP frames from Error stack traces. Extensive anti-fingerprinting and risk-control evasion measures baked in at every layer.
- **AI Agent ready** — `explore` discovers APIs, `synthesize` generates adapters, `cascade` finds auth strategies, `browser` controls the browser directly.
- **External CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, obsidian, docker, etc). Zero setup.
- **Self-healing setup** — `opencli doctor` diagnoses and auto-starts the daemon, extension, and live browser connectivity.
- **Dynamic Loader** — Simply drop `.ts` adapters into the `clis/` folder for auto-registration.
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
- **Deterministic** — Same command, same output schema, every time. Pipeable, scriptable, CI-friendly.
- **Broad coverage** — 79+ sites across global and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, and more), plus desktop Electron apps via CDP.

---

## Quick Start

Expand Down Expand Up @@ -283,9 +295,9 @@ opencli plugin uninstall my-tool

| Plugin | Type | Description |
|--------|------|-------------|
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | YAML | GitHub Trending repositories |
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | TS | GitHub Trending repositories |
| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | TS | Multi-platform trending aggregator |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | YAML | 稀土掘金 (Juejin) hot articles |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | TS | 稀土掘金 (Juejin) hot articles |
| [opencli-plugin-vk](https://github.com/flobo3/opencli-plugin-vk) | TS | VK (VKontakte) wall, feed, and search |

See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
Expand All @@ -298,7 +310,7 @@ See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.

```bash
opencli explore https://example.com --site mysite # Discover APIs + capabilities
opencli synthesize mysite # Generate YAML adapters
opencli synthesize mysite # Generate TS adapters
opencli generate https://example.com --goal "hot" # One-shot: explore → synthesize → register
opencli cascade https://api.example.com/data # Auto-probe: PUBLIC → COOKIE → HEADER
```
Expand Down
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ OpenCLI 可以用同一套 CLI 做三类事情:
- **输出稳定**:适配器命令返回固定结构,适合 shell、脚本、CI 和 AI Agent 工具调用。
- **面向 AI Agent**:`browser` 负责实时操作,`explore` 负责探索接口,`synthesize` 负责生成适配器,`cascade` 负责探测认证路径。
- **运行成本低**:已有命令运行时不消耗模型 token。
- **天然可扩展**:既能用内置能力,也能注册本地 CLI,或直接往 `clis/` 丢 `.ts` / `.yaml` 适配器。
- **天然可扩展**:既能用内置能力,也能注册本地 CLI,或直接往 `clis/` 丢 `.ts` 适配器。

## 快速开始

Expand Down Expand Up @@ -414,7 +414,7 @@ opencli plugin uninstall my-tool # 卸载
# 1. Deep Explore — 网络拦截 → 响应分析 → 能力推理 → 框架检测
opencli explore https://example.com --site mysite

# 2. Synthesize — 从探索成果物生成 evaluate-based YAML 适配器
# 2. Synthesize — 从探索成果物生成 evaluate-based TS 适配器
opencli synthesize mysite

# 3. Generate — 一键完成:探索 → 合成 → 注册
Expand Down
5 changes: 2 additions & 3 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,10 +132,9 @@ npx vitest src/

## 如何添加新测试

### 新增 YAML Adapter(如 `clis/producthunt/trending.yaml`)
### 新增 Adapter(如 `clis/producthunt/trending.ts`)

1. `opencli validate` 的 E2E / smoke 测试会覆盖 adapter 结构校验
2. 根据 adapter 类型,在对应测试文件补一个 `it()` block
1. 根据 adapter 类型,在对应测试文件补一个 `it()` block

```typescript
// 如果 browser: false(公开 API)→ tests/e2e/public-commands.test.ts
Expand Down
36 changes: 36 additions & 0 deletions clis/bilibili/hot.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import { cli, Strategy } from '@jackwener/opencli/registry';

cli({
site: 'bilibili',
name: 'hot',
description: 'B站热门视频',
domain: 'www.bilibili.com',
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of videos' },
],
columns: ['rank', 'title', 'author', 'play', 'danmaku'],
pipeline: [
{ navigate: 'https://www.bilibili.com' },
{ evaluate: `(async () => {
const res = await fetch('https://api.bilibili.com/x/web-interface/popular?ps=\${{ args.limit }}&pn=1', {
credentials: 'include'
});
const data = await res.json();
return (data?.data?.list || []).map((item) => ({
title: item.title,
author: item.owner?.name,
play: item.stat?.view,
danmaku: item.stat?.danmaku,
}));
})()
` },
{ map: {
rank: '${{ index + 1 }}',
title: '${{ item.title }}',
author: '${{ item.author }}',
play: '${{ item.play }}',
danmaku: '${{ item.danmaku }}',
} },
{ limit: '${{ args.limit }}' },
],
});
38 changes: 0 additions & 38 deletions clis/bilibili/hot.yaml

This file was deleted.

28 changes: 28 additions & 0 deletions clis/bluesky/feeds.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { cli, Strategy } from '@jackwener/opencli/registry';

cli({
site: 'bluesky',
name: 'feeds',
description: 'Popular Bluesky feed generators',
domain: 'public.api.bsky.app',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of feeds' },
],
columns: ['rank', 'name', 'likes', 'creator', 'description'],
pipeline: [
{ fetch: {
url: 'https://public.api.bsky.app/xrpc/app.bsky.unspecced.getPopularFeedGenerators?limit=${{ args.limit }}',
} },
{ select: 'feeds' },
{ map: {
rank: '${{ index + 1 }}',
name: '${{ item.displayName }}',
likes: '${{ item.likeCount }}',
creator: '${{ item.creator.handle }}',
description: '${{ item.description }}',
} },
{ limit: '${{ args.limit }}' },
],
});
29 changes: 0 additions & 29 deletions clis/bluesky/feeds.yaml

This file was deleted.

28 changes: 28 additions & 0 deletions clis/bluesky/followers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { cli, Strategy } from '@jackwener/opencli/registry';

cli({
site: 'bluesky',
name: 'followers',
description: 'List followers of a Bluesky user',
domain: 'public.api.bsky.app',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'handle', required: true, positional: true, help: 'Bluesky handle' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of followers' },
],
columns: ['rank', 'handle', 'name', 'description'],
pipeline: [
{ fetch: {
url: 'https://public.api.bsky.app/xrpc/app.bsky.graph.getFollowers?actor=${{ args.handle }}&limit=${{ args.limit }}',
} },
{ select: 'followers' },
{ map: {
rank: '${{ index + 1 }}',
handle: '${{ item.handle }}',
name: '${{ item.displayName }}',
description: '${{ item.description }}',
} },
{ limit: '${{ args.limit }}' },
],
});
Loading
Loading