Extract shared mirror publish engine

Task 10479 · Done · Coder · created · updated

Original work item (epic Task 10454, verbatim): - Remove Draft state from anywhere of the jaah-ce, agent-sdk, agent-connectors. It needs to be removed at the source and then propagated using standard mechanism - remove any references on the source of documents (especially do not mention any brands) - agent-sdk. must borrow and share the process from jaah-ce, as well as terms. We want them mirrors from the main repo. Only main branch for now - do we want the same with agent-connector? make sure it is up to date with agent-sdk - Look for any sensitive or internal information or any brands - scrub all old versions from all three repos Full settled inputs: the epic's `prd` and `architecture` artifacts (Task 10454; read with `task_read`/`artifact_read`). Decisions quoted below are verbatim from them. Child A — Mirror engine (architecture §1.1–§1.3, §1.9 `--root`). Requirements served (PRD verbatim): R2 "No published tree or mirror commit names the source of a document: no "adapted from …", no n8n, no `jaah-ai/jaah` link or sha, no `SOURCE_COMMIT` file (or any file whose purpose is to point at the private source), no "Jaah Local". Commit messages of every mirror are neutral (e.g. "Jaah Agent SDK <version>")." R3 "agent-sdk is published from `packages/agent-sdk/` only by a manual, environment-gated workflow run, one snapshot commit onto `main`, tagged with the repo-wide version; it is never pushed on a develop merge and never pushed to any branch but `main`. The publish mechanism is shared with jaah-ce's (one implementation; the two may differ only in data: source path, destination, message, environment) — per the context-architecture "two copies" ceiling." Build: - `tools/mirror/publish.py` (today's `tools/community_edition/publish.py` generalized: title from the manifest `[mirror] title`, env `MIRROR_DEST` / `MIRROR_DEPLOY_KEY`, never writes a source sha, fast-forward-only), `tools/mirror/build.py` (generic copy, `[[layout]]`, `[[rewrite]]`, `generate` steps; takes a manifest path), `tools/mirror/check.py` (generic checks: excluded paths, symlinks, gitleaks, instance data; `CHECKS` list seam for B), `tools/mirror/manifest.py` (moved from `tools/community_edition/manifest.py`, plus `[mirror] name, title, source_root, generate = [[argv…]]` and `[sensitive] allow = [{category, path_glob, pattern}]`). - CE wrappers: `tools/community_edition/build_tree.py` = generic build + CE-only steps (`edition_removed_routes.py`, `[[drop_dependency]]`); `tools/community_edition/check_tree.py` = generic checks + CE checks 1, 5, 6, 7. CE ships `tools/mirror/**` except `publish.py` (`import_roots` += `tools/mirror`). - No provenance: `build.py` stops writing `SOURCE_COMMIT`; the marker becomes a sidecar `<out>.mirror-build` (sibling of the tree, never inside it; holds manifest name + source sha for CI logs). Commit message exactly `Jaah Community Edition v<ver>` / `Jaah Agent SDK v<ver>` / `Jaah Agent Connectors v<ver>`, no second line; author/committer `Jaah` <actions@github.com>. - `.github/actions/mirror-publish/action.yml` composite action (inputs `mirror`, `dest`, `deploy-key`; steps: branch guard `master` (ce-staging also `develop`), `tools/version.sh --publish`, build, strict check, publish). Every action pinned by SHA. - `community-edition.yml` publish job calls the action with `dest: ${{ vars.CE_DEST }}`, `deploy-key: ${{ secrets.CE_DEPLOY_KEY }}`; environments unchanged. - New `agent-sdk.yml` (`workflow_dispatch` only, `publish` choice `none` | `agent-sdk-mirror`, check-only default) + `packages/agent-sdk/mirror.toml`. Delete `.github/workflows/agent-sdk-mirror.yml` (no publish on a develop push). - `--root` flag on `publish.py` and a `root_reset` dispatch input on each mirror workflow (parentless commit, `--force` to `main` only when `root_reset` equals the destination repo name); guarded and tested against a bare repo; H's last PR deletes both. - Build-determinism test: two builds of each mirror from one sha give the same `git write-tree` hash, `generate` included. - Tests: rewrite `tools/tests/test_community_edition.py:85-92` (SOURCE_COMMIT assertion); new `tools/tests/test_mirror.py` (publish, sidecar, message, `--root` guard, determinism); move the `jaah-ce` grep at `:640` there. - Docs: process prose in `packages/README.md`, `tools/community_edition/README.md`, `packages/agent-sdk/README.md` describe the one shared mechanism (single-home: one doc owns the mechanism, others point to it). Pinned seams others build on: `check.py <tree> --manifest <path> [--baseline <path>]`, `CHECKS: list[fn(tree: Path, man: Manifest, *, baseline: Baseline | None) -> list[Finding]]`, `Finding(file, line, category, message)`; action `with: {mirror, dest, deploy-key}`, caller job sets `environment:`. Credentials & reach: - AC3/AC5/AC7 read oracles (branches, commits, tags, releases, file contents) | the runner's `jaah-forge` (task forge auth), GitHub API | **verified**: `jaah-forge api get repos/jaah-ai/agent-connectors/git/trees/develop` and `repos/jaah-ai/jaah/environments` answered - `./dc check full`, unit and gate oracles | nothing beyond the worktree | n/a Out of scope: - Do NOT create or change GitHub environments, secrets or deploy keys (Child H). Do NOT run any publish or write to any mirror repo. Do NOT add the sensitive check (Child B) or connectors content (Child C). - Do NOT rename or change the product's `draft` task state or the SDK spec's "draft task" wording (D1). - Do NOT change any repo's visibility (D8); do NOT publish packages to PyPI/npm. - Do NOT relicense agent-sdk or agent-connectors (Apache-2.0 stays); do NOT change the CE license's terms beyond removing the draft banner and origin line. - Do NOT remove functionally required third-party names (S1) — provider CLIs, protocols, dependencies, forge names. - Do NOT edit monorepo files that ship to no mirror for §5 hygiene (S2) (the `skynet` rename alone is exempt, A2); do NOT rewrite the monorepo's own history. - Do NOT change jaah-ce's `ce-staging` environment or the edition seam / API cut. - Do NOT edit the mirrors directly except the one-time history reset and branch/tag/release/package deletion in R5; every content change lands in the monorepo and reaches the mirror by its publish workflow. - Do NOT change the SDK v2 API or the `jaah.ai` extension spec semantics. - Do NOT write to any mirror repo before the final publish child (A5). Done means: - AC3(c, d) by code and test (PRD verbatim): "(c) run the publish workflow by hand from GitHub Actions for agent-sdk → one commit on `main` whose tree equals `packages/agent-sdk/` (plus generated bundles), tag = repo-wide version; (d) code review: jaah-ce and agent-sdk (and connectors) publish through one shared implementation." — oracle: `tools/tests/test_mirror.py` publishes the SDK manifest into a bare repo and asserts one commit on `main`, tree == built tree, tag == `tools/version.sh --publish`; review shows CE and SDK workflows both call `.github/actions/mirror-publish` → `tools/mirror/publish.py` (no second publish implementation; `grep -rn "git push" .github/workflows` finds none outside the action). - AC3(b) by code: `test ! -e .github/workflows/agent-sdk-mirror.yml`; no workflow under `.github/workflows` has an `on: push` trigger that publishes a mirror. - AC2 for the commit and sidecar: a built tree has no `SOURCE_COMMIT` (`test ! -e <tree>/SOURCE_COMMIT`), the sidecar sits beside it; the published commit message is exactly `Jaah Agent SDK v<ver>` (no sha, no `jaah-ai/jaah` URL) — asserted in the test. - AC9 except environments (PRD verbatim): "Every publish is reachable by the operator: the three workflows appear in the monorepo's Actions tab, each runnable by hand from the branch its environment admits, each with its environment and deploy key set (missing key → the job fails before any push, as jaah-ce today)." — oracle: `agent-sdk.yml` and `community-edition.yml` are `workflow_dispatch`; a test/dry-run with an empty `MIRROR_DEPLOY_KEY` exits non-zero before any push. - `--root` and `root_reset` exist, are guarded (refuse unless `root_reset` == destination repo name) and are tested on a bare repo; without `--root` publish stays fast-forward-only (test). - Determinism test passes for the CE and SDK manifests. - Quality: `tools/mirror/` reads as one coherent engine (no CE-specific names in it); `./dc check quick` green locally, `./dc check full` green in the merge lane.

Outcome

Shared mirror engine tools/mirror (build/check/publish) + mirror-publish action; CE and new dispatch-only agent-sdk.yml use it; agent-sdk-mirror.yml deleted; no SOURCE_COMMIT; --root guarded. Quick gate + 71 tests green; PR #7077 queued in lane. Deferred: Child H env setup. Memory: none applied.

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)