Skip to content

development.md

Latest commit

 

History

History
362 lines (265 loc) · 9.97 KB

development.md

File metadata and controls

362 lines (265 loc) · 9.97 KB
title description
Development Guide
How to build, test, and contribute to go-scm.

Development Guide

Prerequisites

  • Go 1.26 or later
  • Git (for git/ package tests)
  • gh CLI (for collect/github.go and rate limit checking -- not required for unit tests)
  • SSH access to agent machines (for agentci/ integration -- not required for unit tests)
  • Access to forge.lthn.ai/core/go and sibling modules for the framework dependency

Repository Layout

go-scm/
+-- go.mod              Module definition (dappco.re/go/core/scm)
+-- forge/              Forgejo API client + tests
+-- gitea/              Gitea API client + tests
+-- git/                Multi-repo git operations + tests
+-- agentci/            Clotho Protocol, agent config, security + tests
+-- jobrunner/          Poller, journal, types + tests
|   +-- forgejo/        Forgejo signal source + tests
|   +-- handlers/       Pipeline handlers + tests
+-- collect/            Data collection pipeline + tests
+-- manifest/           Application manifests, ed25519 signing + tests
+-- marketplace/        Module catalogue and installer + tests
+-- plugin/             CLI plugin system + tests
+-- repos/              Workspace registry, work config, git state + tests
+-- cmd/
|   +-- forge/          CLI commands for `core forge`
|   +-- gitea/          CLI commands for `core gitea`
|   +-- collect/        CLI commands for data collection
+-- docs/               Documentation
+-- .core/              Build and release configuration

Building

This module is primarily a library. Build validation:

go build ./...         # Compile all packages
go vet ./...           # Static analysis

If using the core CLI with a .core/build.yaml present:

core go qa             # fmt + vet + lint + test
core go qa full        # + race, vuln, security

Testing

Run all tests

go test ./...

Run a specific test or package

go test -v -run TestName ./forge/
go test -v ./agentci/...

Run with race detector

go test -race ./...

Race detection is particularly important for git/ (parallel status), jobrunner/ (concurrent poller cycles), and collect/ (concurrent rate limiter access).

Coverage

go test -coverprofile=cover.out ./...
go tool cover -html=cover.out

Local Dependencies

go-scm depends on several forge.lthn.ai/core/* modules. The recommended approach is to use a Go workspace file:

// ~/Code/go.work
go 1.26

use (
    ./core/go
    ./core/go-io
    ./core/go-log
    ./core/config
    ./core/go-scm
    ./core/go-i18n
    ./core/go-crypt
)

With a workspace file in place, replace directives in go.mod are superseded and local edits across modules work seamlessly.

Test Patterns

forge/ and gitea/ -- httptest Mock Server

Both SDK wrappers require a live HTTP server because the Forgejo/Gitea SDKs make an HTTP GET to /api/v1/version during client construction. Use net/http/httptest:

func setupServer(t *testing.T) (*forge.Client, *httptest.Server) {
    mux := http.NewServeMux()
    mux.HandleFunc("/api/v1/version", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        json.NewEncoder(w).Encode(map[string]string{"version": "1.20.0"})
    })
    // ... register other handlers ...
    srv := httptest.NewServer(mux)
    t.Cleanup(srv.Close)
    client, err := forge.New(srv.URL, "test-token")
    require.NoError(t, err)
    return client, srv
}

Config isolation -- always isolate the config file from the real machine:

t.Setenv("HOME", t.TempDir())
t.Setenv("FORGE_TOKEN", "test-token")
t.Setenv("FORGE_URL", srv.URL)

SDK route divergences discovered during testing:

  • CreateOrgRepo uses /api/v1/org/{name}/repos (singular org)
  • ListOrgRepos uses /api/v1/orgs/{name}/repos (plural orgs)

git/ -- Real Git Repositories

git/ tests use real temporary git repos rather than mocks:

func setupRepo(t *testing.T) string {
    dir := t.TempDir()
    run := func(args ...string) {
        cmd := exec.Command("git", args...)
        cmd.Dir = dir
        require.NoError(t, cmd.Run())
    }
    run("init")
    run("config", "user.email", "[email protected]")
    run("config", "user.name", "Test")
    // write a file, stage, commit...
    return dir
}

Testing getAheadBehind requires a bare remote and a clone:

bare := t.TempDir()
exec.Command("git", "init", "--bare", bare).Run()
clone := t.TempDir()
exec.Command("git", "clone", bare, clone).Run()

agentci/ -- Unit Tests Only

agentci/ functions are pure (no I/O except SSH exec construction) and test without mocks:

func TestSanitizePath_Good(t *testing.T) {
    result, err := agentci.SanitizePath("myrepo")
    require.NoError(t, err)
    assert.Equal(t, "myrepo", result)
}

jobrunner/ -- Table-Driven Handler Tests

Handler tests use the JobHandler interface directly with a mock forge.Client:

tests := []struct {
    name   string
    signal *jobrunner.PipelineSignal
    want   bool
}{
    {"merged PR", &jobrunner.PipelineSignal{PRState: "MERGED"}, true},
    {"open PR", &jobrunner.PipelineSignal{PRState: "OPEN"}, false},
}
for _, tt := range tests {
    t.Run(tt.name, func(t *testing.T) {
        assert.Equal(t, tt.want, handler.Match(tt.signal))
    })
}

collect/ -- Mixed Unit and HTTP Mock

Pure functions (state, rate limiter, events) test without I/O. HTTP-dependent collectors use mock servers:

srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(mockResponse)
}))
t.Cleanup(srv.Close)

The SetHTTPClient function allows injecting a custom HTTP client for tests.

manifest/, marketplace/, plugin/ -- io.Medium Mocks

These packages use the io.Medium abstraction. Tests use io.NewMockMedium() to avoid filesystem interaction:

m := io.NewMockMedium()
m.Write(".core/manifest.yaml", yamlContent)
manifest, err := manifest.Load(m, ".")

repos/ -- io.Medium with Seed Data

m := io.NewMockMedium()
m.Write("repos.yaml", registryYAML)
reg, err := repos.LoadRegistry(m, "repos.yaml")

Test Naming Convention

Tests use the _Good / _Bad / _Ugly suffix pattern:

Suffix Meaning
_Good Happy path -- expected success
_Bad Expected error conditions
_Ugly Panic, edge cases, malformed input

Coding Standards

Language

Use UK English throughout: colour, organisation, centre, licence (noun), authorise, behaviour. Never American spellings.

Go Style

  • All parameters and return types must have explicit type declarations.
  • Import groups: stdlib, then forge.lthn.ai/..., then third-party, each separated by a blank line.
  • Use testify/require for fatal assertions, testify/assert for non-fatal. Prefer require.NoError when subsequent steps depend on the result.

Error Wrapping

// Correct -- using the log.E helper from core/go-log
return nil, log.E("forge.CreateRepo", "failed to create repository", err)

// Correct -- contextual prefix with package.Function
return nil, fmt.Errorf("forge.CreateRepo: marshal options: %w", err)

// Incorrect -- bare error with no context
return nil, fmt.Errorf("failed")

Context Propagation

  • git/ and collect/ propagate context correctly via exec.CommandContext.
  • forge/ and gitea/ accept context at the wrapper boundary but cannot pass it to the SDK (SDK limitation).
  • agentci/ uses SecureSSHCommand for all SSH operations.

Commit Conventions

Use conventional commits with a package scope:

feat(forge): add GetCombinedStatus wrapper
fix(jobrunner): prevent double-dispatch on in-progress issues
test(git): add ahead/behind with bare remote
docs(agentci): document Clotho dual-run flow
refactor(collect): extract common HTTP fetch into generic function

Valid types: feat, fix, test, docs, refactor, chore.

Every commit must include the co-author trailer:

Co-Authored-By: Virgil <[email protected]>

Adding a New Package

  1. Create the package directory under the module root.
  2. Add package <name> with a doc comment describing the package's purpose.
  3. Follow the existing client.go / config.go / types.go naming pattern where applicable.
  4. Write tests from the start -- avoid creating packages without at least a skeleton test file.
  5. Add the package to the architecture documentation.
  6. Maintain import group ordering: stdlib, then forge.lthn.ai/..., then third-party.

Adding a New Handler

  1. Create jobrunner/handlers/<name>.go with a struct implementing jobrunner.JobHandler.
  2. Name() returns a lowercase identifier (e.g. "tick_parent").
  3. Match(signal) should be narrow -- handlers are checked in registration order and the first match wins.
  4. Execute(ctx, signal) must always return an *ActionResult, even on partial failure.
  5. Add a corresponding <name>_test.go with at minimum one _Good and one _Bad test.
  6. Register the handler in Poller configuration alongside existing handlers.

Adding a New Collector

  1. Create a new file in collect/ (e.g. collect/mynewsource.go).
  2. Implement the Collector interface (Name() and Collect(ctx, cfg)).
  3. Use cfg.Limiter.Wait(ctx, "source-name") before each HTTP request.
  4. Emit events via cfg.Dispatcher for progress reporting.
  5. Write output via cfg.Output (the io.Medium), not directly to the filesystem.
  6. Honour cfg.DryRun -- log what would be done without writing.
  7. Return a *Result with accurate Items, Errors, Skipped, and Files counts.

Remote and Push

The canonical remote is on Forgejo via SSH:

git push origin main
# Remote: ssh://[email protected]:2223/core/go-scm.git

HTTPS authentication to forge.lthn.ai is not configured -- always use SSH on port 2223.

Licence

EUPL-1.2. The licence is compatible with GPL v2/v3 and AGPL v3.