Post

The counterfactual: what should have caught this

On June 6, 2025, a contractor with three-minute-old live backend access rewrote NCBA Rwanda’s mobile money integration so that 70 ghost accounts could withdraw without balance checks, and the system reported “normal” for eight days. The previous post dissected the full anatomy of the attack. Here’s the uncomfortable part: every step of that attack is preventable with technology that exists today — AI, automation, AI-enabled security, notifications, and CI/CD. Not one of them was in place.

This playbook walks the attack timeline and maps each step to the control that stops it, then lists the best practices every fintech should adopt.

Step 1 — Privileged access granted before trust is established

What happened: live backend privileges granted at 5:30 AM; abuse at 5:33 AM. No break-in period, no approval workflow, no session recording.

The fix — Privileged Access Management (PAM) + just-in-time access:

  • Grant privileged access just-in-time via an approval workflow, with a short expiry, never standing access.
  • Require two-person control (dual authorization) for production access; a second approver gets pinged automatically.
  • Record sessions (terminal + screen) and stream them to a SIEM for playback on suspicion.
  • Use break-glass accounts with immediate alerting when used — including who, when, and what commands ran.

AI layer: UEBA (User and Entity Behavior Analytics) models learn each engineer’s normal behaviour — the systems they touch, the hours they work, the commands they run. A contractor touching core integration logic at 5:33 AM on their first day scores as a massive anomaly the moment it happens, not eight days later.

Step 2 — A single developer rewrote core validation logic

What happened: one person altered mobile integration logic that controlled balance checks and success status — straight to production, no review.

The fix — CI/CD gates, four-eyes, and environment separation:

  • No direct production deploys. Ever. Code moves feature branch → PR → automated checks → staging → approved promotion to prod (exactly the main → staging → approved promote model we run at YucanPay).
  • Mandatory code review by a different engineer, enforced by branch protection (1+ approval, status checks required).
  • AI code review as a second pair of eyes: tools that flag “this diff disables a validation check” or “this change introduces a hardcoded account allowlist” — semantic diff analysis catches what a tired human reviewer misses.
  • Immutable artifacts: build once, sign it, promote the same artifact. If someone changes code in production, the deployed artifact hash no longer matches the signed release — that mismatch itself is an alert.
  • SAST + secrets scanning in CI so “bypass validation” patterns and embedded account lists are flagged before merge.

The three-minute test: with CI/CD, a code change at 5:33 AM doesn’t reach production until it passes review. The exploit’s entire premise — instant, unreviewed deployment — is structurally impossible.

Step 3 — Validation could be switched off for a hardcoded list of accounts

What happened: the modified logic skipped balance checks for exactly 70 specific accounts. The system never checked whether those accounts were real, or how much they held.

The fix — defense in depth on the money path:

  • Server-side validation that cannot be bypassed by application code: enforce account existence, status, and balance checks in a separate control plane (database constraints, banking-core validations) that an app-layer change can’t switch off.
  • Database triggers and audit trails: every state-changing query writes an immutable audit row (who, what, when, before/after). A “Success without debit” pattern becomes visible in the ledger, not just in code.
  • Ghost account detection: a data-quality job that periodically validates that every account receiving funds exists in the core system and is KYC-verified. 70 fabricated profiles would be flagged on the first run.

Step 4 — Dashboards showed “normal” for eight days

What happened: operational dashboards tracked availability, not integrity. A withdrawal that returned “Success” without a corresponding debit looked like a normal transaction.

The fix — AI-enabled anomaly detection + real-time reconciliation:

  • Realtime reconciliation, not weekly: cross-check internal ledgers against telecom/switch settlement files continuously. The NCBA deficit took eight days to surface because reconciliation ran on a weekly cycle — an automated daily (or hourly) reconciliation would have flagged the first Ksh discrepancy within hours.
  • AI fraud models tuned for this exact pattern: velocity anomalies (70 accounts withdrawing repeatedly in a short window), consistency checks (Success without debit), ratio anomalies (payouts vs. settlement), and unsupervised anomaly detection that catches novel patterns models weren’t trained on.
  • Graph analytics: the 70 ghost accounts almost certainly share wallets, devices, or counterparties. Transaction graph analysis connects them into a cluster that a rule engine would never see.
  • Model drift monitoring: if the fraud model’s predictions shift for a specific channel, that’s an alert — the attack was designed to be invisible to static rules but not to statistical shifts.

Step 5 — No one was notified

What happened: the exploit produced no alerts. The first signal was a human running a weekly settlement.

The fix — notification & alerting pipelines:

  • Every anomaly event routes to a notification pipeline (SMS, WhatsApp, Slack, PagerDuty) with severity-based escalation: informational → risk team → on-call engineer → executive for large-value anomalies.
  • Threshold-based real-time alerts: single-transaction limits, daily cumulative withdrawal limits per account, and “withdrawal without debit” integrity alerts — the exact signature of this attack.
  • Reconciliation break alerts: any gap between internal ledger and settlement partner triggers an immediate incident, not a weekly spreadsheet review.
  • Test the notification path like you test code — chaos-engineer the alerting pipeline monthly. A notification system nobody receives is a nice dashboard, not a control.

The best-practices checklist for fintech systems

Access & identity

  • Just-in-time privileged access with dual approval and expiry
  • PAM + session recording; break-glass accounts with instant alerting
  • Suspension-aware auth: revoked/suspended users lose sessions immediately
  • Rate limiting on login and OTP (per account + per IP)

Change & delivery

  • No direct production access; feature branch → PR → staging → approved promote
  • Branch protection: 1+ approvals, status checks, signed commits
  • Immutable, signed, versioned artifacts with hash verification at deploy
  • AI-assisted code review + SAST + secrets scanning in CI
  • Automatic rollback and canary deploys for production changes

Detection & response

  • Real-time (not weekly) reconciliation against every settlement partner
  • AI anomaly detection on transactions, privileged users, and balances
  • Database audit triggers; immutable change log
  • Alerting pipeline with escalation + monthly chaos tests
  • SIEM with UEBA on privileged accounts

Data & code integrity

  • Ghost account / data-quality validation jobs
  • Validation enforced in a control plane the app layer cannot bypass
  • Secrets management (vault), never secrets in code or CI logs
  • Encryption in transit and at rest; tokenization of PII

Why this matters beyond NCBA

The NCBA case is not an outlier — it’s the pattern. Flutterwave’s 2023 incident (~N2.7B attempted), the Sidian Bank attempted Sh80M heist in 2022, and the M-PESA agent fraud rings all share the same DNA: privilege + unchecked change + monitoring that looks at the wrong metrics. The countermeasure is a stack, not a tool: AI and analytics to see the anomaly, automation to make unauthorized change structurally impossible, CI/CD to gate every line of code, and notifications to make silence itself an alarm.

The bottom line: the three-minute window isn’t the vulnerability. The vulnerability is an environment where three minutes of access can change core money logic without review, and where eight days can pass before a mismatch of 57.5 million shillings produces a single notification. Fix the environment — AI, automation, AI security, notifications, CI/CD — and the three-minute exploit becomes a three-minute false alarm.

References

This post is licensed under CC BY 4.0 by the author.