API Audit Engine

Status

in-flight

Tier

Tier 2 — Platform

Owner

Ryan Colston

Started

2026-05-01

Target launch

2026-06-01

One-line description. Verified reference docs for every API I publish to often, kept honest by a monthly drift detector that alerts when reality diverges.

Why

I was trying to see who opened my first Listing Legends agent newsletter in Kit. The MCP gave aggregate stats only, then 404'd on per-recipient endpoints. The frustration ("every time I try to do something with the Kit API, we can't") surfaced a real gap: I publish weekly to Ghost, send a Kit broadcast every Wednesday, deploy to Cloudflare for every site I ship — and didn't have a verified map of what those APIs can and can't do.

Five reference docs solve the immediate gap. A monthly drift detector keeps them honest. The cost of stale docs that quietly become wrong is invisible until something breaks in production at 9 UTC on a Sunday — this system pays for itself the first time it catches a breaking API change before code ships against the new shape.

Current state

In-flight. Five audits shipped to main on 2026-05-02 via !640. Drift detector + GitLab CI job live. First scheduled run fires 2026-06-01 at 9 UTC. First manual run triggered 2026-05-06 to validate the pipeline.

• Status: in-flight
• Last update: 2026-05-06
• Blocked on: nothing — verifying first manual CI run completes cleanly (#191)

Next 3 actions

1. Confirm first manual CI run completes successfully — slim image + pip install + GCP_SA_KEY auth all work end-to-end (#191)
2. Close the (unverified) markers in the BigQuery and Cloudflare reference docs using the now-verified spec snapshots (#189)
3. Decide whether .strikes state stays in git or moves to BigQuery — defer until first real drift cycle proves git churn is real (#190)

Decisions log

• 2026-05-02 — Cloudflare spec source = first-party developers.cloudflare.com/openapi.json. Eliminates GitHub dependency. Verified live 2026-05-02.
• 2026-05-02 — GitLab issue auth = Project Access Token (or user PAT fallback) with api scope, masked CI var DRIFT_DETECTOR_TOKEN. CI_JOB_TOKEN rejected because its allowed-endpoint list doesn't reliably include issue creation.
• 2026-05-02 — BigQuery heartbeat reuses existing GCP_SA_KEY from .gitlab-ci.yml. Schema-corrected: function_name + last_run only, status encoded via name suffix.
• 2026-05-02 — Doc-page strategy uses 2-strikes rule + cleaned/normalized text snapshots. Phase-3 fix (post-merge) closed two false-positive bugs surfaced on first wild run.
• 2026-05-06 — Single hub for the whole audit engine instead of one hub per API. The 5 audits + drift detector are one system, not six.

Open issues

• GitLab: glab issue list --label area/api-audit -R rylobasic/ccpj is the punch list.
• Open as of 2026-05-06:
• #189 — Phase 5: close (unverified) gaps in audit docs
• #190 — Move .strikes state out of git
• #191 — Trigger first manual CI run + verify slim image works

References

• Implementation plan: API Drift Detector v2
• Code home — drift detector: ~/CCPJ/projects/api-drift-detector/
• Code home — audit docs: ~/CCPJ/APIs/{Kit,Ghost,Cloudflare,BigQuery,FUB}-API-Implementation/
• CI job: .gitlab-ci.yml::api-drift-check (line 409)
• Heartbeat target: pos_raw.function_heartbeats (function_name = api-drift-detector clean / api-drift-detector-drift on alert)
• Shipped MR: !640
• Related projects: BQ Bulletproof (heartbeat infra, monitoring patterns)

What this covers

Why this hub exists

Five separate hubs would fragment a single system. Future me (or a new contributor) lands on one page that says "this is the audit engine, here's what it covers, here's what's open, here's the code" — instead of having to assemble that picture from six places.