AI in Regulated FinTech: The Engineering Constraints

Most writing about shipping AI features assumes a startup: pick a provider, call the API, iterate in production. Everything I've described in the other posts on this blog gets significantly more complicated when the software is a bank's.

I work on digital onboarding and deposit products in a regulated environment. This is the engineering view of what changes — not legal advice, and not a compliance checklist. Your legal and compliance teams own the actual answers; my point is that a lot of what looks like a compliance problem is really an engineering problem you can solve before the conversation starts.

---

Constraint 1: Some Data Cannot Leave

The first question is never "which model" — it's what may be sent, and where does it physically go.

In practice that means classifying data before you design anything:

• Public / synthetic — documentation, sample data, code with no secrets. Easy.
• Internal — code, architecture, tickets. Usually fine under an appropriate agreement.
• Customer data — names, account numbers, transaction history, identity documents. This is where the real work is.

For the third category, the engineering answers are the ones worth knowing:

Don't send it. A shocking amount of value doesn't need real data. Summarising a policy document, generating test fixtures, drafting internal documentation, reviewing code — none of it needs a customer's name.

Redact before you send. Deterministic PII removal in front of the model, with placeholders (<ACCOUNT_1>) restored on the way back. Local, auditable, testable. Test it like a security control, because it is one.

Check the retention and training terms. Whether inputs are retained, for how long, and whether they can be used for training are contract questions with different answers per provider and plan. Get them in writing rather than from a blog post — mine included.

Ask where inference happens. Data residency is often the binding constraint, not the model's quality. Regional deployment options exist across the major providers and cloud platforms; that's a procurement conversation to have early rather than after you've built.

---

Constraint 2: You Have to Be Able to Explain It

Regulated processes need to be explainable and reproducible. A system that produces a different output for the same input, for reasons no one can articulate, is a bad fit for anything that decides something about a customer.

Two consequences shape the architecture:

Log everything, immutably. For any AI-assisted decision: the exact prompt, the model and version, the parameters, the retrieved context, the raw output, and what the system did with it. If you can't reconstruct why the system said what it said six months ago, you have a problem you'll only discover during an audit.

Draw a hard line at decisions. The distinction that matters is between AI that drafts and AI that decides. Drafting a customer email that an agent reviews and sends — straightforward. Deciding whether an application is approved — an entirely different regulatory category, and under the EU AI Act, things like creditworthiness assessment of individuals fall into the high-risk bucket with obligations attached. Know which side of that line your feature is on before you build it, not after.

Human-in-the-loop is not a UX preference here. It's frequently the thing that keeps a feature in the tractable category.

---

Constraint 3: A Model Upgrade Is a Change

Outside a regulated environment, swapping to a newer model is a config change. Inside one, it's a change to a validated system: it needs testing, evidence, and sign-off, exactly like a library upgrade in a payment path.

The practical implication is that you cannot use a floating model alias in production. Pin an exact model version. Know when it's deprecated. Plan the migration as a project with a test cycle, not as an afternoon.

This is where an eval suite stops being an engineering nicety and becomes the artifact that makes the change management possible: run both models against a fixed set of cases, record the results, attach them to the change request. Without it you're asking someone to approve a change with no evidence. (I wrote about building one in the post on evals.)

---

Constraint 4: Third-Party Risk Is Real Work

Adding a model provider means adding a critical third party, and that comes with questions engineers are not used to answering: What's the contractual availability? What happens during an outage? Are subprocessors disclosed? What's the exit plan if the terms change?

The engineering-side answers:

• Abstract the provider behind your own interface. Not to be clever about swapping models weekly — to make "we must move" a two-week project rather than a rewrite.
• Have a degraded mode. What the feature does when the provider is down is a product decision, and in a bank it's often a required one.
• Watch the cost tail. Per-token pricing plus unbounded user input is an incident waiting to happen. Cap input size, cap output tokens, rate-limit per user, alert on spend.

---

Where It's Actually Easy

The picture above sounds discouraging, so here's the part that isn't: internal developer tooling has almost none of these constraints.

Code review, test generation, refactoring, understanding legacy services, drafting documentation, writing migration scripts — no customer data, no automated decisions, no explainability requirement, no customer impact if it's wrong. Just faster engineers.

That's where the value has been for us, by a wide margin. The customer-facing AI feature is a project with a legal workstream, a security review, and a change management process. The internal tooling is available on Monday.

If you're trying to introduce AI in a regulated company, start there. It builds the track record and the institutional comfort that the harder conversations later will need — and it delivers actual value in the meantime.