Add tenant Gmail connections and plan switch
## Original request (verbatim)
> Migrate the emails system from GMail API to AWS SES. Use all the best practices. The quality of communication is of highest priority. Spare no time or expense.
>
> [jaah.ai](http://jaah.ai) MX is currently at GMail and will remain so, but we can carve out whatever we need
Epic context (BA PRD Task 9352, Architecture Task 9353) — settled: all system mail (send + receive) moves to SES on `mail.jaah.ai` (dev: `mail.dev.jaah.ai`, dev account, sends/receives real mail) with suppression, operator-only delivery log, RFC 8058 unsubscribe, DMARC monitoring, operator-approved template redesign; tenant Gmail (agent send, automation action, receive) is built in this epic behind one plan feature. All system AWS calls use the hand-signed SigV4 seam (`apps/api/app/aws/common.py` `signed_call`), no boto3. SES events and inbound mail are PULLED from SQS by system crons (precedent `apps/api/app/alarm_relay.py`, `apps/api/app/aws/sqs.py`); the only new public path is unsubscribe on the hooks edge.
## Scope of this child
C7 Tenant Gmail connections: plan switch, `gmail_connections` table, OAuth (send-only) + service-account connect, the Gmail API client moved out of `gmail_mailbox.py` into `app/tenant_gmail/client.py`, the tenant send function (S7) with its `tenant_email_sends` audit table, the shared MIME builder `mail_message.py`, console *Org settings > Email*, the `apps/docs` reader page, and `docs/integrations/gmail.md` rewritten as the tenant-Gmail owner doc. Delivers PRD R9 and the tenant-Gmail part of R13; ships the S7 seam C8/C9 build on.
## Requirements (verbatim from the PRD)
### R9 — Tenant Gmail connection (A6, Q6, Q10, Q15, Q16)
Behind a new plan feature ON in every plan's defaults (operator-overridable per org), an org can connect a Gmail mailbox by (a) per-user Google OAuth requesting `gmail.send` only, or (b) a Workspace service account with domain-wide delegation naming the mailbox (send + read). The credential lives on the control plane; it never reaches a node.
AS1. Assumed tenant `mail.jaah.ai` trigger addresses sit behind the **same plan feature** as tenant Gmail (one "tenant email" term).
AS9. Assumed tenant Gmail sends go **to any recipient** (no verified-address rule) since Q14 = no guardrails and the tenant is the sender of record; system mail keeps the verified-address rule.
### R13 — Documentation (part)
`docs/integrations/gmail.md`'s "Why Gmail, not SES" rationale and the module docstrings stating it are replaced; tenant Gmail has a reader doc in `apps/docs`.
## Approach (from the settled architecture)
- Plan switch `SwitchSpec("tenant_email_allowed", "Tenant email", True)` (`apps/api/app/plan_terms.py:176`) + billing column + `orgs.tenant_email_allowed_override`; gates connections **and** hosted addresses (AS1).
- `gmail_connections` (org-scoped, standard entity shape per `docs/standards/coding-standards.md`): kind `oauth|service_account`, mailbox address, `sealed_secret` (via `credential_crypto.seal(..., aad=connection id)`, precedent `siem_export.seal_secret:107`), can_receive (SA only), history_id, last_sync_at/error. A sealed column, not a credential row — credential rows are dispatched to runners; a column no dispatch path reads makes "never on a node" structural.
- OAuth connect: operator knobs `tenant_gmail_oauth_client_id/secret` (operator tier); flow via `oauth_flow_cookie`; scope exactly `https://www.googleapis.com/auth/gmail.send` (+ `openid email` to learn the address). SA connect: upload key JSON + mailbox; validated by a token exchange with `gmail.send gmail.readonly`.
- `tenant_gmail.send(...)` per S7 writes `tenant_email_sends` (org-scoped: connection, task_id, automation_run_id, actor, to, cc, subject, status, error, gmail id) + an audit event; visible to org admins under *Org settings > Email > Sent*. MIME via `apps/api/app/mail_message.py` `build(sender, to: list[str], cc: list[str], ...)` — **the one MIME builder** shared with SES system mail. This child (phase 1, always first) CREATES `mail_message.py`: From/To/Cc/Subject/Reply-To, multipart/alternative text+HTML, extra headers, header-injection-safe `_header_text` moved from `gmail_mailbox.py` (which then imports it from `mail_message`); tenant Gmail sets its own `Message-ID`. The SES send child (C2a) extends it for S4 — never two builders.
- Gmail API client code (token minting, HTTP calls) moves to `app/tenant_gmail/client.py`, which imports nothing from `gmail_mailbox`. Until C2a/C3 retire the system Gmail path, `gmail_mailbox.py` keeps working by importing the moved functions from `tenant_gmail/client.py` (one copy, single-home); C3 deletes `gmail_mailbox.py` (§9.9).
- Blank operator knobs `tenant_gmail_oauth_client_id/secret` ⇒ the OAuth connect option is hidden (SA connect still offered) and the API answers 409 "not configured"; they are filled when the operator supplies the Google OAuth client (I4, deferred).
- Plan switch OFF (§9.14): sends refuse `PlanOff`; connections and history are kept and resume when turned on again; the connect option is absent and the API refuses connecting.
- Retention (§9.15) registered in `app/cron_retention.py`: `tenant_email_sends` 365 d; a `gmail_connections` row and its sealed secret deleted on disconnect.
- Console: *Org settings > Email* (connections list → tabbed detail: Overview, Sent, Inbound, Addresses — Inbound/Addresses tabs are filled by C10; render them only when they have content or leave them to C10), per `docs/standards/ui-style-guide.md`; copy per `docs/standards/ui-copy-standards.md`.
- Docs: `apps/docs` reader page for tenant Gmail; `docs/integrations/gmail.md` rewritten as the tenant-Gmail owner doc (no "Why Gmail, not SES"; system mail points to `docs/integrations/ses.md`).
Pinned seams (build against these exactly; other children build against them too):
- S7 tenant send (C7 → C8/C9): `TenantOutboundEmail(to: list[str], cc: list[str] = [], subject: str, text: str, html: str | None = None)` (1–50 recipients total); `tenant_gmail.send(session, *, org_id: UUID, connection_id: UUID | None, message: TenantOutboundEmail, actor: Actor, task_id: UUID | None = None, automation_run_id: UUID | None = None) -> SendResult`; `SendResult(send_id: UUID, gmail_message_id: str, from_address: str)`. `connection_id=None` = the org's only send-capable connection, else `AmbiguousConnection` (→ 409). Errors: `NoConnection` (→ 409), `PlanOff` (→ 403), `GmailRefused(.status)` (→ 502). C7 creates `tenant_email_sends` and writes it inside `send`; C8/C9 only pass `task_id`/`automation_run_id`. Both builders feed `mail_message.build(sender, to: list[str], cc: list[str], ...)`.
Credentials & reach:
| Oracle | Credential / identity / environment | Status |
|---|---|---|
| AC10 (non-operator 403), AC17 tenant side | `org/console/dev-orgb-login` (org B admin) | **verified** — `credential_get` ok (its recorded base_url is legacy; use `https://app.dev.jaah.ai`) |
| AC14/AC15/AC16/AC17(a) live (AC16 needs a connected Gmail too) | Jaah Google OAuth client (gmail.send) + a test Workspace SA with DWD | **deferred** — operator: "Build now, verify later" (I4) |
| AC5, AC18, unit/integration suites | worktree only | **verified** — nothing beyond the worktree |
Out of scope:
- Do NOT build the agent MCP tool (C8), the automation action (C9) or tenant receive/hosted addresses/poll cron (C10).
- Do NOT touch the system send path or `notify_outbox` (C2a) beyond creating the shared `mail_message.py`.
- Do NOT store a tenant secret in a credential row or anywhere a dispatch path reads.
Epic out-of-scope (PRD §7, verbatim):
- MUST NOT change the apex `jaah.ai` MX, SPF or `_dmarc.jaah.ai` record, nor any Workspace mailbox, alias or routing.
- MUST NOT add SES Virtual Deliverability Manager, BIMI/VMC, or a dedicated IP / dedicated IP pool.
- MUST NOT request Google's restricted `gmail.readonly` (or any restricted Gmail scope) via OAuth; no CASA assessment.
- MUST NOT add open/click tracking pixels or link rewriting to system mail.
- MUST NOT let tenant Gmail replace system mail for an org's members (Q9 option not chosen).
- MUST NOT expose the delivery log or suppression list to tenant admins or users (operator only, Q7).
- MUST NOT relax the verified-address rule or invitation budgets for system mail (AS6).
- MUST NOT add boto3/botocore to `apps/api/requirements.txt` unless the Architect records why the hand-signed SigV4 seam (`apps/api/app/aws/`) cannot serve SES (the repo guards its absence).
- MUST NOT put a tenant Gmail credential, OAuth token or service-account key on any node or in a runner's environment.
- MUST NOT change Slack/Telegram delivery, the support chat, or site enquiries (no email today).
Done means:
- **AC14 (R9) Tenant connects Gmail.** → verify on dev as an org admin of a non-operator org: connect Gmail via OAuth (consent screen shows send-only scope) and, separately, via a service-account key naming a mailbox; both show connected. With the plan feature OFF for that org (operator override), the connect option is absent and the API refuses. (deferred — live: operator "Build now, verify later" (I4); Jaah Google OAuth client + test Workspace SA do not exist yet.) Substitute now: tests against recorded Google responses cover both connect paths, the exact requested OAuth scope string (`gmail.send` + `openid email`, nothing else), the SA token-exchange validation, and plan-OFF refusal (API 403 + connect option absent in the console, checked live on dev with `org/console/dev-orgb-login` after flipping the override as operator).
- Secret confinement: a test proves no dispatch/compose path can read `gmail_connections.sealed_secret` (e.g. the runner env/compose builders contain no reference to the table), and the sealed value round-trips only with the matching connection id as AAD.
- S7 contract: unit tests for `tenant_gmail.send` covering `NoConnection` (409), `AmbiguousConnection` (409), `PlanOff` (403), `GmailRefused` (502), and that every send (success or failure) writes a `tenant_email_sends` row and an audit event.
- *Org settings > Email* renders the connections list and detail without layout breakage at 390 px and desktop (Playwright screenshot in the report).
- AC18 (part): `docs/integrations/gmail.md` no longer contains "Why Gmail, not SES"; the reader page exists in `apps/docs` and is reachable per `apps/docs/README.md`.
- Quick gate `./dc check quick` green and changed-area tests green; `python3 tools/check_context.py` green (any doc this child adds/edits is routed and single-homed).
Outcome
Built tenant Gmail: the Tenant email plan switch, Google OAuth (send only) and service-account connect, a sealed secret, the S7 send seam with an audit row per attempt, the shared MIME builder, Settings › Email, and docs. Merged as #6749 (10a9b63) after the review lane and the lane's full gate. Visual check done at 390px and desktop. Live Google checks deferred (I4). The deploy was superseded by a later merge and left alone. Memory: none applied.
Artifacts
Activity
- Draft
- Ready
- Runner started
- Runner ended (done)