Dev/impl v0

.gitignore

@@ -12,3 +12,10 @@ dist/
 build/
 *.egg-info/
 
+.venv/
+uv.lock
+
+.tmp_artifacts/
+
+# Example output files generated by running scripts under `examples/`.
+examples/output.csv

README.md

@@ -1,82 +1,184 @@
 # usegolib
 
-**usegolib** lets Python call Go libraries by loading a Go-built shared library directly into the Python process.
+[![CI](https://github.com/TimLai666/usegolib/actions/workflows/ci.yml/badge.svg)](https://github.com/TimLai666/usegolib/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/usegolib.svg)](https://pypi.org/project/usegolib/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/usegolib.svg)](https://pypi.org/project/usegolib/)
+[![License](https://img.shields.io/github/license/TimLai666/usegolib.svg)](LICENSE)
+
+Call Go libraries from Python by loading a Go-built shared library directly into the Python process.
 
 - No C/C++ glue code written by you
 - No background Go process (no daemon / RPC / subprocess)
-- MessagePack-based ABI
+- MessagePack-based ABI with manifest-driven schema validation
 
-## What problem does this solve?
+## Status
 
-- You have a Go library
-- You want Python users to call it like a native Python module
-- You do NOT want:
-  - C/C++ glue code
-  - gopy-style heavy bindings
-  - running a Go server or subprocess
+`usegolib` is early-stage software (v0.x). Expect sharp edges and evolving interfaces.
 
-## Model
+## How It Works
 
-- **Build time (CI / build machine)**: use Go toolchain + an auto-provisioned C compiler (Zig) to build a Go module into a shared library + `manifest.json`.
-- **Runtime (end-user / production)**: Python loads the shared library and calls it via MessagePack. Go is not running as a service.
+- Build time (CI / build machine): use the Go toolchain plus an auto-provisioned C compiler (Zig) to build a Go module into a shared library and emit `manifest.json`.
+- Runtime (end-user / production): Python loads the shared library and calls it via MessagePack. No daemon, no RPC, no subprocess.
 
 ## Requirements
 
 - Python 3.10+
-- Go toolchain (build-time only; not required for runtime if you ship prebuilt artifacts/wheels)
+- Go toolchain
+  - required for building artifacts (CI / build machine)
+  - required in dev if you rely on auto-build (missing artifact triggers a build)
+  - not required for end-user runtime if you ship prebuilt artifacts or wheels
+
+## Installation
+
+```bash
+python -m pip install usegolib
+```
 
-## Quickstart (local build + call)
+From source (dev):
 
 ```bash
-# Build an artifact from a local Go module directory
-python -m usegolib build --module path/to/go/module --out out/artifact
+python -m pip install -e ".[dev]"
 ```
 
+## Quickstart
+
+Build an artifact from a local Go module directory:
+
+```bash
+usegolib build --module path/to/go/module --out out/artifact
+```
+
+Import from the artifact root and call exported functions:
+
 ```python
 import usegolib
 
-# Import from an artifact *root* (may contain many modules/versions/platforms)
 h = usegolib.import_("example.com/mod", artifact_dir="out/artifact")
 print(h.AddInt(1, 2))
 ```
 
-Typed wrappers (when schema is present in the artifact manifest):
+Dev convenience: omit `artifact_dir` to use the default artifact cache root. If the artifact
+is missing, `usegolib.import_` will build it into the default cache when a Go toolchain is
+available.
+
+```python
+import usegolib
+
+h = usegolib.import_("example.com/mod")
+print(h.AddInt(1, 2))
+```
+
+To enforce "no build" behavior, pass an explicit artifact root and set `build_if_missing=False`:
+
+```python
+import usegolib
+
+h = usegolib.import_("example.com/mod", artifact_dir="out/artifact", build_if_missing=False)
+```
+
+## Example: Calling Insyra From Python
+
+Insyra is a Go data-analysis library: `github.com/HazelnutParadise/insyra`.
+
+Auto-build on import (developer convenience; downloads the module + builds into usegolib's default artifact root):
+
+```python
+import usegolib
+
+insyra = usegolib.import_("github.com/HazelnutParadise/insyra", version="v0.2.14")
+
+# DataList (object handle + variadic any)
+dl = insyra.NewDataList()
+dl.Append(1, 2, 3, 4, 5)
+print(dl.Sum(), dl.Mean())
+
+# DataTable (variadic map[string]any + variadic bool)
+dt = insyra.NewDataTable()
+dt.AppendRowsByColIndex({"A": 1, "B": 2}, {"A": 3, "B": 4})
+print(dt.NumRows(), dt.NumCols())
+dt.Show()
+```
+
+Notes:
+
+- `usegolib` can only call functions/methods whose parameter and return types are supported by the current type bridge (see `docs/abi.md`).
+- This example requires a Go toolchain to be installed. (At the time of writing, `insyra@v0.2.14` requires `go >= 1.25` and may cause Go to auto-download a newer toolchain.)
+- If you want to disable auto-build, pass `build_if_missing=False` (and usually an explicit `artifact_dir`).
+- For end-users without Go, ship prebuilt artifacts/wheels (see Packaging).
+
+## Methods (Object Handles)
+
+For exported methods, `usegolib` supports Go-side object handles (opaque ids) to avoid serializing the receiver on every call:
+
+```python
+with h.object("Counter", {"n": 1}) as c:
+    print(c.Inc(2))
+```
+
+Typed object handles (schema required):
 
 ```python
 th = h.typed()
-print(th.AddInt(1, 2))
+types = th.types
+
+with th.object("Counter", types.Counter(N=10)) as c:
+    snap = c.Snapshot()
+    print(snap)
+```
+
+## Generic Functions (Build-Time Instantiation)
+
+Generic functions require explicit build-time instantiation:
+
+```bash
+usegolib build --module example.com/mod --out out/artifact --generics generics.json
+```
+
+At runtime (schema required), use the helper:
+
+```python
+fn = h.generic("Id", ["int64"])
+print(fn(123))
 ```
 
-## Quickstart (generate a distributable Python package project)
+## Packaging (Ship Wheels Without Requiring Go)
+
+Generate a distributable Python package project embedding artifacts:
 
 ```bash
-python -m usegolib package --module path/to/go/module --python-package-name mypkg --out out/
+usegolib package --module path/to/go/module --python-package-name mypkg --out out/
 ```
 
-Then install the generated project:
+Install the generated project:
 
 ```bash
 python -m pip install -e out/mypkg
 ```
 
 ```python
 import mypkg
+
 print(mypkg.AddInt(1, 2))
 ```
 
-## Version Rules (important)
+## Version Rules
 
 - One Go module = one version per Python process
 - If a module is already loaded, all subpackages must use the same version
-- If `version=None` and multiple artifact versions exist under `artifact_dir`, import fails with `AmbiguousArtifactError`
-
-## Policies
-
-- `docs/versioning.md`: ABI + manifest versioning and compatibility rules
-- `docs/compatibility.md`: supported platforms and toolchain pins
-- `docs/security.md`: security boundaries and current hardening
-- `docs/reproducible-builds.md`: reproducible build guidance
+- If `version=None` and multiple artifact versions exist under `artifact_dir`, import fails with `AmbiguousArtifactError` (unless that module is already loaded in the current process, in which case `version=None` follows the already-loaded version)
+
+## Documentation
+
+- `docs/roadmap.md`
+- `docs/abi.md`
+- `docs/cli.md`
+- `docs/versioning.md`
+- `docs/compatibility.md`
+- `docs/testing.md`
+- `docs/security.md`
+- `docs/troubleshooting.md`
+- `docs/releasing.md`
 
-## Releasing
+## License
 
-- `docs/releasing.md`
+MIT. See `LICENSE`.

docs/abi.md

@@ -23,7 +23,10 @@ The ABI is intentionally small: the wire format carries only generic MessagePack
 All requests are MessagePack maps:
 
 - `abi`: integer ABI version (v0 == `0`)
-- `op`: operation name (v0 supports only `call`)
+- `op`: operation name (v0 supports: `call`, `obj_new`, `obj_call`, `obj_free`)
+
+### `op = "call"`
+
 - `pkg`: Go package import path (string)
 - `fn`: exported function name (string)
 - `args`: list of arguments (see Type Bridge below)
@@ -40,6 +43,66 @@ Example (conceptual):
 }
 ```
 
+### `op = "obj_new"`
+
+Create a Go-side object instance and return an opaque id.
+
+- `pkg`: Go package import path (string)
+- `type`: receiver struct type name (string, exported; no leading `*`)
+- `init`: optional record-struct value used to initialize the struct (Level 3); if omitted, the zero value is used
+
+Example (conceptual):
+
+```text
+{
+  "abi": 0,
+  "op": "obj_new",
+  "pkg": "example.com/mod",
+  "type": "Counter",
+  "init": {"n": 1}
+}
+```
+
+### `op = "obj_call"`
+
+Call a method on a previously created object.
+
+- `pkg`: Go package import path (string)
+- `type`: receiver struct type name (string, exported; no leading `*`)
+- `id`: object id returned by `obj_new`
+- `method`: exported method name (string)
+- `args`: list of arguments (see Type Bridge below)
+
+Example (conceptual):
+
+```text
+{
+  "abi": 0,
+  "op": "obj_call",
+  "pkg": "example.com/mod",
+  "type": "Counter",
+  "id": 1,
+  "method": "Inc",
+  "args": [2]
+}
+```
+
+### `op = "obj_free"`
+
+Free a Go-side object id.
+
+- `id`: object id returned by `obj_new`
+
+Example (conceptual):
+
+```text
+{
+  "abi": 0,
+  "op": "obj_free",
+  "id": 1
+}
+```
+
 ## Response Format
 
 All responses are MessagePack maps:
@@ -68,6 +131,9 @@ Supported values crossing the ABI:
   - `string`
   - `bytes`
   - slices/lists of supported values (including lists of bytes)
+- `any`:
+  - `any` is represented on the wire as a plain MessagePack value (no extra envelope)
+  - `[]any` and `map[string]any` are supported recursively
 - Level 2:
   - `map[string]T` where `T` is a Level 1 scalar or a slice of Level 1 scalars
   - Encoded as a MessagePack map from string keys to values
@@ -77,6 +143,19 @@ Supported values crossing the ABI:
 
 Unsupported values MUST fail with `UnsupportedTypeError`.
 
+### Variadic Parameters (`...T`)
+
+Go variadic parameters (`...T`) are represented in the ABI as a single final argument whose value is a list.
+
+Examples:
+- Go: `Append(values ...any)`
+  - Python: `obj.Append(1, 2, 3)` sends `args = [[1, 2, 3]]`
+  - Python: `obj.Append()` sends `args = [[]]`
+- Go: `Data(useNamesAsKeys ...bool)`
+  - Python: `dt.Data(True)` sends `args = [[True]]`
+
+When manifest schema is present, the Python runtime packs varargs automatically.
+
 ### Canonical Keys (Tags)
 
 For record structs, canonical keys follow:
@@ -102,6 +181,7 @@ These are interpreted using the manifest schema; on the wire they are just strin
 
 Artifacts embed a schema in `manifest.json` describing:
 - callable symbols and their parameter/return types
+- callable methods (receiver type + method name) and their parameter/return types
 - named struct types and their fields (including keys, aliases, required/omitempty)
 
 When schema is present, the runtime validates call arguments and successful results against the schema