Skip to content
Ballo Innovations2026 — present
In buildaiarchitecturemobile

Penda

A money app whose AI assistant can act on your finances but cannot quietly rewrite them

A personal finance app with a chat assistant that reads and acts on a user's own ledger. The interesting risk is not the model saying something wrong — a user can discount that. It is the model being confidently wrong about a number and silently overwriting history, because the entire value of a finance tool is the user believing the balance.

Built for
Ballo Innovations
My role
Architecture and implementation of the assistant's tool layer, the confirmation and staging model, and the row-level security boundary it runs inside.
When
2026 — present
  • Deno
  • Supabase Edge Functions
  • PostgreSQL
  • Row Level Security
  • Gemini
  • Groq
  • TypeScript
Related wallet experiments
01The engineering

What made it hard

The problems worth reading about. Everything else in this system was ordinary work.

  1. 01

    The assistant can create a transaction but cannot edit or delete one — not because the prompt says so, but because the function holding the model contains no code that performs an update. A prompt is a suggestion a model can misread; it cannot be a security boundary.

  2. 02

    Destructive intents are written to a staging table and executed by a different function that only runs when the user taps confirm. Tiering is by blast radius rather than by SQL verb, so a large or multi-field create is staged too.

  3. 03

    The assistant runs under the user's own credentials, so database row-level security — not application code — bounds everything it can reach.

02Numbers

What it measures

Each figure says where it came from, so you can judge how much weight to give it. Some are measurements and some are chosen thresholds; the basis line tells you which.

0
Tools that can reach an update or delete

Structural, and the point of the design: the chat function's tool dispatch has no case that performs an update or a delete. Verifiable by reading the dispatch rather than by sampling behaviour.

2
Independent allowlists a write must satisfy

One at staging time in the chat function, one at execution time in the confirm function. A third mirror exists client-side for undo.

40 s
Turn budget

Chosen after working out the unbounded case: a twelve-second model timeout across two providers over four tool iterations stacked to roughly 160 seconds. The wall-clock budget bounds it directly.

Write-up

The longer version

The problem

An assistant that can only answer questions about your money is a search box. An assistant that can act on your money is useful, and is also one bad turn away from destroying data the user cannot reconstruct. Creates are recoverable — a spurious row is visible in a list and can be removed. Updates and deletes are where a confused agent overwrites a number that no longer exists anywhere, and the user may not notice for a month.

What I built

The confinement model: two functions rather than one, with the model-facing function structurally unable to perform a destructive write. Everything else — the staging table, the snapshots for undo, the graduated trust flag that lets a consistently-correct user stop confirming small creates — follows from wanting the guarantee to be a property of the code rather than of the model's behaviour on a given day.

What I would not claim

There is no prompt-injection defence here. What exists is capability confinement, which is a different and in some ways stronger property, but it is not the same claim: nothing tags untrusted text as untrusted. The argument is only that the model's reachable actions are too narrow for an injected instruction to do much with.

The allowlist is also enforced at more than one layer, across two runtimes. Each copy is deliberate defence in depth; keeping them in agreement is not free, and it is the next thing to fix.

Decision records

The close calls, written up

Where a choice was genuinely arguable, I wrote it up the way a team records a design decision internally: the constraint, the options I turned down and why, and what it would cost to reverse.

  • ADR-004acceptedAI Safety30 Jul 2026

    Confining what the assistant can write

    The chat assistant in Penda can create a transaction, but it cannot edit or delete one. Not because the prompt tells it not to — because the function holding the model has no code that performs an update. Destructive intents are written to a staging table and executed by a different function that only runs when the user taps confirm.

    4 options evaluated