Build governance core shared library

Task 5315 · Done · Coder · created · updated

Part of epic 5303 "Living Governance foundation + Portfolio Compliance". Settled inputs — READ FIRST: PRD = Task 5304 artifact `prd`; Architecture = Task 5305 artifact `architecture` (fetch with task_read 5304/5305 + task_read_artifact). § refs are the architecture doc's; R-/AC-/D-/I- ids are the PRD's / architecture §1's. The pinned seam contracts S1–S10 (architecture §11) are BINDING: field names exact, additive fields allowed. Epic vocabulary (verbatim): "Vocabulary decision: the unit is a Rule (not "norm"). A versioned set of rules is a Pack. A rule whose subject is the repo itself (rather than a change) is a structural rule. Portfolio Compliance = structural rules + packs + levels + the matrix + remediation." Design principles (verbatim, "binding for every child task"): - First result in about a minute, with zero configuration. Everything else is progressive disclosure. - Every red cell says why and has a fix action. No score without an explanation. - Rules are data, not prose: a rule without oracles cannot be enforced. - Stack-conditional expectations everywhere; "n/a for this stack" is a visible state. - The data model must be the general one (rules, graph, attestations, twin), even though the first product only uses structural rules. - Readable at phone width; the matrix and the per-repo view are the two screens that matter most. ## Requirement (verbatim) Epic: "Checks are deterministic first (file presence, CI definition parsing, forge settings over the API, skill manifests); an LLM judge is used only where nothing mechanical can decide, and its verdicts carry a confidence that is never rendered as green." / "Inference does the work: the scan works out each repo's stack, deploy target, data stores and existing agent instrumentation, and the expectations follow from that (a Terraform repo is not asked for unit-test coverage; a Django repo is expected to carry a migrations skill)." / "The one thing to preserve is the existing checks' logic where it maps cleanly onto structural rules." PRD R-D.1: "A repo's level is the highest level L for which every level ≤ L is unlocked. A level is unlocked when at least 80% of its applicable rules pass ... n/a rules and rules with an approved exception are excluded from the denominator, and the exceptions are shown as debt. A judge verdict below the pass threshold or of low confidence counts as not passing." ## Scope (architecture §4, contracts S1–S4) New `packages/governance` (stdlib py3.12; never imports an app): - `states.py` — S1, THE single home of cell states (`pass, fail, na_stack, out_of_scope, off, observed, waived, judge, needs_ledger_data, not_scanned, scanning, error, not_scannable_no_repo, not_scannable_no_node`), overlays (`queued, in_progress, pr_open, fix_merged_rescan, fix_failed`), classes GREEN/FAILING/EXCLUDED, `JUDGE_PASS_CONFIDENCE = 0.7`, judge encoding per §4.1. - `predicates.py` — S2 `evaluate(pred, facts)`, `describe(pred)`, `validate(pred)`. - `packfile.py` — S3 TOML pack/rule shape + `validate_pack()` (≥1 pass & ≥1 fail oracle per non-ledger rule, judge_reason on judge rules, primitives exist with valid params, unique keys, pillar/level in pack). Expose a machine-readable primitive catalogue (name + param schema) for the console API (child "Add compliance authoring and proposals API" serves it). - `scoring.py` — S4 `score(cells, rules, pack) -> RepoScore(level, pillars, debt_count)`; zero-applicable level counts unlocked; below L1 = 0. - `primitives/` — file_exists, file_contains, readme_sections, ci_defines_job (GitHub Actions/GitLab/Bitbucket/Azure; PyYAML imported lazily → `error` "yaml unavailable", never pass), instruction_refs_valid, instrumentation_present, skills_match_stack (seeded from `docs/policies/agentic-instrumentation.md` golden baseline), hooks_enforce_gate, mcp_pinned, forge_setting (injected callable over `jaah-forge api get` so oracles stub it), legacy_check (wraps `portfolio/engine` REPO_RULE_CHECKS/rulechecks with profile=None; CHECK_VALID→pass, else fail with note as evidence). A raising primitive → `error`, never pass. `RepoView` abstracts a directory or an in-memory files dict. - `inference.py` — languages, frameworks, package managers, deploy targets (incl. `*.tf`→terraform), data stores (incl. alembic/ and Django migrations/ → has_migrations=true), instrumentation (CLAUDE.md/AGENTS.md, skills, agents, hooks, .mcp.json), each fact with evidence. - `oracles.py` (run_oracles) and `cli.py` skeleton for `jaah-audit` (subcommands are the "Build jaah-audit node CLI and verbs" child's). - Fixture repos under `packages/governance/tests/fixtures/`: Python/FastAPI+Postgres, React/TS, Terraform-only, Django-with-migrations. - Wiring: `infra/local/Dockerfile.api` copies `packages/governance` and makes `governance` importable; `tools/check.sh` runs its tests; `tools/build_jaahd_bundle.py` MANIFEST ships it to nodes; PyYAML added to `infra/jaah/requirements.txt`. Update `packages/README.md` (today says "Empty by design today") with a short pointer-style description (single-home; no copied prose). ## Acceptance (PRD, verbatim — the parts this child owns) - AC-8 (engine half): "engine tests over fixture repos (a Python/FastAPI+Postgres repo, a React/TS repo, a Terraform-only repo) assert the inferred facts" - AC-9 (engine): "The Terraform fixture repo is not asked for unit-test coverage (its cell reads "n/a for this stack" with the reason). A Django/Python-with-migrations fixture is expected to carry a migrations skill. n/a is excluded from the level." - AC-10 (unit): "The level follows the 80% unlock rule in R-D.1. Verify: unit tests on the level computation, including n/a, exceptions, judge low-confidence and needs-ledger-data cases" - AC-13 (primitives): "Every deterministic rule's verdict is produced by code, and an engine test runs it with no LLM available." Out of scope: Epic out of scope (verbatim): "Out of scope: change-level governance (session attestations, guards at tool-call time, the judge over transcripts), the simulator and back-testing, rule effectiveness and context budgeting, the rule miner, earned autonomy and trust bands, regulatory control mapping and evidence packs, the pack marketplace, exports to IDPs. Each of these is a later epic on the same foundation; the data model built here must not make any of them a rewrite." Settled fences (PRD §7 + architecture §1): no gate/lock enforcement or merge seam (I1); no attestation signing (I2); no git↔DB rule sync (D4); no campaign budget (D5); no forge-side checks (D13); no push/merge-triggered rescans (D6); the CP never reads repo content (D2); no parallel plan-flag store (D7); do NOT touch registry `[[asset]]` propagation / `portfolio/engine` sync, context routing (`.claude/context-routes.json`, `tools/context_router.py`), `maturity_sweep.py`, persona licences, or the task lifecycle state machine. This child: no DB tables, no API routes, no UI, no packs, no daemon verbs or prepare/submit logic; do not modify `portfolio/engine` check logic — wrap it. Done means: - `python -c "import governance"` succeeds inside the API container AND in a node-bundle layout built by `tools/build_jaahd_bundle.py` — show both outputs. - Inference tests over the four fixtures assert exact fact sets (FastAPI+Postgres → language python, framework fastapi, data_store postgres; React/TS → typescript, react; Terraform-only → deploy_target terraform and NO language python; Django → has_migrations=true) — cite test names + pass output. - Scoring unit tests cover: n/a excluded; waived excluded + debt counted; judge pass 0.95 counts passing; judge pass 0.4 counts NOT passing; needs_ledger_data excluded; zero-applicable level unlocked; 79% vs 80% boundary — cite. - Every primitive has a pass and a fail test via RepoView fixtures, plus tests that a raising primitive returns `error` and that ci_defines_job with PyYAML unimportable returns `error` (never pass). - legacy_check test runs `adr` and `context_architecture` against a fixture with no registry profile. - `./dc check full` green — cite summary.

Outcome

Built packages/governance (S1–S4, 11 primitives, inference, oracles, jaah-audit skeleton) + API image/bundle/PyYAML wiring. Full gate green; PR #4639 merged. Deferred: TS states mirror (C7), regex sandbox (C4a).

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)