Provision SES mail infrastructure in Terraform

## Original request (verbatim) > Migrate the emails system from GMail API to AWS SES. Use all the best practices. The quality of communication is of highest priority. Spare no time or expense. > > [jaah.ai](http://jaah.ai) MX is currently at GMail and will remain so, but we can carve out whatever we need Epic context (BA PRD Task 9352, Architecture Task 9353) — settled: all system mail (send + receive) moves to SES on `mail.jaah.ai` (dev: `mail.dev.jaah.ai`, dev account, sends/receives real mail) with suppression, operator-only delivery log, RFC 8058 unsubscribe, DMARC monitoring, operator-approved template redesign; tenant Gmail (agent send, automation action, receive) is built in this epic behind one plan feature. All system AWS calls use the hand-signed SigV4 seam (`apps/api/app/aws/common.py` `signed_call`), no boto3. SES events and inbound mail are PULLED from SQS by system crons (precedent `apps/api/app/alarm_relay.py`, `apps/api/app/aws/sqs.py`); the only new public path is unsubscribe on the hooks edge. ## Scope of this child C1 Mail infra: Terraform for SES on a delegated `mail.<env>` zone in dev and prod, deploy-policy refit, hooks-edge path swap, SES production-access requests for both accounts, the SES owner runbook `docs/integrations/ses.md`, and the ADR entry. Delivers PRD R2, AS10 and the infra/runbook/ADR part of R13. ## Requirements (verbatim from the PRD) ### R2 — Authentication and DNS (A2, A5, Q5) For each of `mail.jaah.ai` and `mail.dev.jaah.ai`: SES identity verified with Easy DKIM (2048-bit); custom MAIL FROM `bounce.<sub>` (SPF aligned); `_dmarc.<sub>` published `p=reject` with `rua` to a mailbox Jaah ingests; MX for `<sub>` → SES inbound; all records in Terraform. The apex `jaah.ai` MX, SPF and DMARC are unchanged. AS10. Assumed the SES **production-access request** (sandbox exit) for each account is a step the infra build performs and AWS approves; until approved, the epic is not done. ### R13 — Documentation and decision record (part) `docs/integrations/gmail.md`'s "Why Gmail, not SES" rationale and the module docstrings stating it are replaced; the SES owner setup has one runbook; tenant Gmail has a reader doc in `apps/docs`; the decision is an ADR entry. (This child: the SES runbook + the ADR entry + the ADR-0084 public-path edit.) ## Approach (from the settled architecture) New `infra/jaah/terraform/mail.tf` (+ module wiring), per environment (`local.name`, `var.mail_domain` = `mail.jaah.ai` | `mail.dev.jaah.ai` in the tfvars): - `aws_route53_zone.mail` + NS delegation record in the parent zone (`data.aws_route53_zone.cp` for main; dev's `dev.jaah.ai` zone). `prevent_destroy`. Reason: the prod deploy role is denied MX/TXT changes under `*.jaah.ai` in the shared apex zone (`modules/deploy-policy/main.tf:341-348`); NS is not in the denied set. - `aws_sesv2_email_identity` for `var.mail_domain`, Easy DKIM **RSA_2048_BIT**; the three DKIM CNAMEs. - `aws_sesv2_email_identity_mail_from_attributes`: `bounce.<mail_domain>`, `behavior_on_mx_failure = REJECT_MESSAGE`; MX `10 feedback-smtp.us-east-1.amazonses.com`; TXT `v=spf1 include:amazonses.com -all`. No SPF on `<mail_domain>` itself. - `_dmarc.<mail_domain>` TXT `v=DMARC1; p=reject; adkim=s; aspf=r; rua=mailto:dmarc-reports@<mail_domain>; fo=1` (`aspf=r` required: MAIL FROM is a sub-subdomain). - MX `<mail_domain>` → `10 inbound-smtp.us-east-1.amazonaws.com`. - `aws_sesv2_configuration_set` `<name>-system`: reputation metrics on, sending on, suppression `BOUNCE, COMPLAINT`, TLS policy `OPTIONAL`; event destination → SNS `<name>-ses-events` → SQS `<name>-ses-events` (+ DLQ, maxReceiveCount 5), event types SEND, DELIVERY, BOUNCE, COMPLAINT, DELIVERY_DELAY, REJECT, RENDERING_FAILURE. - `aws_sesv2_account_suppression_attributes` `BOUNCE, COMPLAINT`. - SNS topic policies admit `ses.amazonaws.com` with `aws:SourceAccount` (+ `aws:SourceArn`); SQS queue policies admit only their topic ARN. DLQ-depth > 0 alarms on both DLQs publish to the CP alarm topic the alarm relay already drains; redrive steps in `docs/integrations/ses.md`. - Inbound: S3 `<name>-inbound-mail` (SSE-S3, block public, lifecycle expiry 30 d, bucket policy admitting `ses.amazonaws.com` with SourceAccount + SourceArn of the rule), SNS `<name>-ses-inbound` → SQS `<name>-ses-inbound` (+ DLQ). `aws_ses_receipt_rule_set` + `aws_ses_active_receipt_rule_set` + one rule: recipients `[<mail_domain>]` (catch-all), `scan_enabled = true`, `tls_policy = Optional`, action S3 (prefix `inbound/`) with `topic_arn` = the inbound topic. - CP task role (`modules/ecs/main.tf:476`): `ses:SendEmail` and `ses:SendRawEmail` on the identity + configuration-set ARNs, conditioned `ses:FromAddress` = `*@<mail_domain>`; `ses:GetSuppressedDestination`, `ses:DeleteSuppressedDestination`, `ses:PutSuppressedDestination`; `sqs:ReceiveMessage`/`DeleteMessage`/`GetQueueAttributes` on the two queues; `s3:GetObject` on `<name>-inbound-mail/inbound/*`. - API env (beside `api_hooks_environment`, `modules/ecs/main.tf:1460`): the S1 variables (region is the existing `AWS_REGION`). - Hooks edge (`modules/ecs/hooks-ingress.tf:32-36`, `apps/web/nginx.conf.template`, `infra/jaah/test_hooks_ingress.py`): replace `/api/public/email/gmail-push` with `/api/public/email/unsubscribe/*` (keeps the ALB rule within its 5-condition cap). Edit ADR-0084's public-path allow-list in place. - Deploy policy (`modules/deploy-policy`): add `ses` to `id_services`, `sqs` for both envs (today only when `manages_alarm_relay`). Size caps are tight (`tests/policy.tftest.hcl:76-81`; project memory mem-17803 says measure with a scratch root and `length(...)`): if the boundary overflows, move the CP task role's SES/SQS statements into a second boundary-compatible managed policy rather than widening wildcards. Never grant kms:Decrypt on a key pattern (mem-17804). - Not in Terraform: the SES production-access request — run `sesv2 put-account-details --production-access-enabled` per account with `me/jaah/system/bootstrap-admin` (assume `OrganizationAccountAccessRole`), mail type TRANSACTIONAL, website `https://jaah.ai`, use-case text naming verified-only recipients, bounce/complaint suppression and one-click unsubscribe. File both on day one; record the case ids in the runbook's status, not the values of any credential. - ADR register entry (`docs/adrs/README.md`): SES for system mail, pull model (SNS→SQS crons, no SES HTTP push), delegated `mail.*` zone per environment. - Terraform discipline: `terraform fmt -recursive && terraform validate`, plan before apply, never paste state. Pinned seams (build against these exactly; other children build against them too): - S1 env (C1 → C2/C3): `SES_MAIL_DOMAIN`, `SES_CONFIGURATION_SET`, `SES_EVENTS_QUEUE_URL`, `SES_INBOUND_QUEUE_URL`, `SES_INBOUND_BUCKET`; inbound objects under prefix `inbound/`. - S2 send tags (C2 ↔ C1 event destination): `EmailTags` `delivery_id=<uuid hex>`, `mail_type=<template key>`. - S3 queue payloads: SNS envelope `{Type:"Notification", Message:"<json>"}` (raw delivery OFF, as the alarm relay expects). Events: SES event-publishing JSON (`eventType`, `mail.messageId`, `mail.tags`, `bounce.bounceType`, `complaint`). Inbound: SES receipt notification (`notificationType:"Received"`, `receipt.recipients`, `receipt.{spam,virus,spf,dkim,dmarc}Verdict.status`, `receipt.dmarcPolicy`, `receipt.action.{bucketName,objectKey}`, `mail.messageId`, `mail.commonHeaders`). Credentials & reach: | Oracle | Credential / identity / environment | Status | |---|---|---| | AC6, terraform plan, SES state, production-access filing (C1, C11) | `me/jaah/system/bootstrap-admin` (management 155534476372) assuming `OrganizationAccountAccessRole` in jaah-dev 905905585302 and jaah-main 366008913850 | **verified** — `credential_get` ok; read-only probe from this node: `sts get-caller-identity`, `sesv2 get-account`, `route53 list-hosted-zones` in both accounts answered | | SES production access (sandbox exit) — gates AC2, AC3, AC11, AC13 and C11 | AWS approval of `PutAccountDetails` in jaah-dev and jaah-main (filed by C1) | **deferred** — reason: AWS reviews it; both accounts probed `ProductionAccessEnabled=false`. Owner C1 files on day one; dependent oracles run after approval | | AC5, AC18, unit/integration suites | worktree only | **verified** — nothing beyond the worktree | Out of scope: - Do NOT write API code (send, inbound, crons, tables) — C2/C3 own it; this child only provides the env and AWS resources they build against. - Do NOT touch the apex `jaah.ai` MX/SPF/`_dmarc.jaah.ai` records; only add the NS delegation record for `mail.jaah.ai` (and `mail.dev.jaah.ai` in `dev.jaah.ai`). - Do NOT widen deploy-policy wildcards to make room. Epic out-of-scope (PRD §7, verbatim): - MUST NOT change the apex `jaah.ai` MX, SPF or `_dmarc.jaah.ai` record, nor any Workspace mailbox, alias or routing. - MUST NOT add SES Virtual Deliverability Manager, BIMI/VMC, or a dedicated IP / dedicated IP pool. - MUST NOT request Google's restricted `gmail.readonly` (or any restricted Gmail scope) via OAuth; no CASA assessment. - MUST NOT add open/click tracking pixels or link rewriting to system mail. - MUST NOT let tenant Gmail replace system mail for an org's members (Q9 option not chosen). - MUST NOT expose the delivery log or suppression list to tenant admins or users (operator only, Q7). - MUST NOT relax the verified-address rule or invitation budgets for system mail (AS6). - MUST NOT add boto3/botocore to `apps/api/requirements.txt` unless the Architect records why the hand-signed SigV4 seam (`apps/api/app/aws/`) cannot serve SES (the repo guards its absence). - MUST NOT put a tenant Gmail credential, OAuth token or service-account key on any node or in a runner's environment. - MUST NOT change Slack/Telegram delivery, the support chat, or site enquiries (no email today). Done means: - **AC6 (R2) DNS.** → verify: `dig +short TXT _dmarc.mail.jaah.ai` → `v=DMARC1; p=reject; …; rua=mailto:…`; `dig +short MX mail.jaah.ai` → SES inbound host; `dig +short TXT bounce.mail.jaah.ai` → SPF including amazonses; same for `mail.dev.jaah.ai`; `dig +short MX jaah.ai` still `1 smtp.google.com.` and `_dmarc.jaah.ai` unchanged. `terraform plan` shows no drift. (This child proves the dev half live after the dev deploy; the prod half is proven by C11 after promotion.) - SES identity `mail.dev.jaah.ai` shows `VerifiedForSendingStatus=true`, DKIM `SUCCESS` (2048-bit) and MAIL FROM `SUCCESS` — verify: `aws sesv2 get-email-identity` in jaah-dev via `me/jaah/system/bootstrap-admin`. - `terraform test` in `infra/jaah/terraform/modules/deploy-policy` green (size caps hold) and `pytest infra/jaah/test_hooks_ingress.py` green with the unsubscribe glob replacing gmail-push. - AS10: production-access requests filed in jaah-dev and jaah-main — verify: `aws sesv2 get-account` in each shows the request recorded (`Details.ReviewDetails.Status` present). Approval itself is AWS's (deferred — AWS reviews it). - A loopback probe works in the sandbox: `aws sesv2 send-email` from jaah-dev to `probe@mail.dev.jaah.ai` produces an object under `inbound/` in the inbound bucket and a message on the inbound queue; a send through the configuration set produces SEND/DELIVERY events on the events queue. - DLQ alarms (§9.8): both DLQs have a depth > 0 CloudWatch alarm whose action is the CP alarm topic the alarm relay drains — verify: `aws cloudwatch describe-alarms` in jaah-dev lists both with that topic ARN as `AlarmActions`. - `docs/integrations/ses.md` (owner runbook incl. DLQ redrive) and the ADR entry exist and are routed from the root instruction file's router. - Quick gate `./dc check quick` green and changed-area tests green; `python3 tools/check_context.py` green (any doc this child adds/edits is routed and single-homed).

SES system-mail infra live on dev (mail.dev.jaah.ai verified DKIM 2048/MAIL FROM/DMARC, loopback + events proven, DLQ alarms). PR #6746 merged 4a3103abd2, dev deploy green. Prod half on promotion; SES prod access filed+answered, AWS review deferred. Memory: proposed 2 gotchas (mem-24624, mem-24625).