Giving an LLM write access to someone’s finances without giving it a write path
The chat assistant in Penda can create a transaction, but it cannot edit or delete one. Not because the prompt tells it not to — because the function holding the model has no code that performs an update. Destructive intents are written to a staging table and executed by a different function that only runs when the user taps confirm.
- Deno
- Supabase Edge Functions
- PostgreSQL
- Row Level Security
- Gemini
- Groq
- TypeScript
What forced a decision
Penda is a money app whose assistant can read and act on a user's finances. The interesting risk is not the model saying something wrong, which a user can discount. It is the model being confidently wrong about a number and silently rewriting a ledger, because the value of a personal finance tool is entirely in the user believing the balance. A hallucinated edit that lands in the database is not a bad answer; it is data loss. Creates are recoverable, because a new row is visible and can be removed. Updates and deletes are where a confused agent destroys history and takes the user's trust with it.
Hard constraints
- No prompt instruction may be the only thing preventing a destructive write.
- A delete must always require an explicit human confirmation, with no exception path.
- The assistant runs under the user's own credentials, so database row-level security bounds everything it can reach.
- A confirmation that is tapped twice, or replayed from an offline queue, must execute exactly once.
- A single model turn must not be able to spend unbounded time or tokens.
What was on the table
Each option carries the one sentence that justifies its verdict.
- Option 01rejected
Let tool calls execute directly
The model calls a tool, the handler performs the corresponding database write, and the result is reported back to the conversation.
For
- Simplest possible implementation; the conversation stays fluent with no interruption.
- No staging state to model, migrate, or clean up.
Against
- A misparsed amount permanently overwrites real data.
- The blast radius of a bad turn is unbounded.
- Nothing structural distinguishes a safe operation from a destructive one.
BecauseThis makes every write exactly as reliable as the model's worst turn. Given that a wrong amount on an update is indistinguishable from a right one until the user notices their balance is off, the failure is both silent and destructive.
- Option 02rejected
Instruct the model to ask before destructive actions
Keep direct execution, but add system-prompt rules requiring the assistant to confirm with the user before editing or deleting anything.
For
- Almost free to implement; no architectural change at all.
- Works most of the time, which makes it dangerously reassuring.
Against
- Unenforceable. There is no mechanism, only an instruction.
- Fails silently and unpredictably rather than loudly.
- Cannot be tested; you can only sample it and hope.
BecauseA prompt is a suggestion the model can misread or ignore, and it fails exactly when you need it most — on an unusual phrasing or a long context. A safety property that depends on a language model choosing to honour it is not a safety property.
- Option 03rejected
One global "let the AI act without asking" toggle
A single user setting that switches the assistant between propose-only and autonomous.
For
- Easy to explain and to build.
- Puts the choice in the user's hands.
Against
- Grants delete authority to a user who only wanted convenience on creates.
- No way to express "trusted for small things, never for large ones".
BecauseIt collapses operations with very different consequences onto one switch. A user who wants to stop confirming every small expense is not thereby saying the assistant may delete a debt record.
- Option 04chosen
Tier by blast radius, and put execution in a different function
Reads and low-impact creates run inline. Updates, deletes, and high-impact creates are staged as pending rows, and only a separate confirm function invoked by a user tap can execute them.
For
- The strongest guarantee available: the capability simply is not present where the model runs.
- Confirmation is a compare-and-swap on the pending row's status, so a double tap or a replayed offline confirm executes once.
- Column allowlists per domain mean even an approved update cannot touch a structural field.
- Trust can be earned for small, reversible operations without ever extending to deletes.
Against
- The same allowlist now exists in three places — the staging function, the execution function, and the client's undo logic — and nothing tests that they agree.
- Conversations are less fluent: the assistant has to say it has proposed something rather than done it, and is explicitly instructed not to claim otherwise.
- A staging table is extra schema, extra migrations, and extra states to reason about.
- Correctness leans on row-level security rather than on scoping inside every query, so an RLS policy mistake would be the single point of failure.
BecauseIt converts a behavioural hope into a structural fact. The function that holds the model contains no code that updates or deletes a record, so no amount of prompt manipulation or model confusion can produce one. The gate is the absence of a code path, not the presence of an instruction.
What each option costs
Scored on the axes that mattered here. The option I chose is weaker than the alternatives on at least one row, which is usually where the interesting conversation starts.
| Criterion | Let tool calls execute directly | Instruct the model to ask before destructive actions | One global "let the AI act without asking" toggle | Tier by blast radius, and put execution in a different functionChosen |
|---|---|---|---|---|
| Guarantee is structural rather than behavioural | Weak | Weak | Adequate | Strong |
| Conversational fluencyThis is the axis the chosen option gives up. Staging means the assistant must describe an intention rather than report a completed action. | Strong | Strong | Strong | Weak |
| Implementation and schema cost | Strong | Strong | Adequate | Weak |
| Blast radius of one bad model turn | Weak | Weak | Weak | Strong |
| Expresses graduated trust | Weak | Weak | Weak | Strong |
What I chose
The chat function exposes seventeen tools. Queries and summaries run inline. Low-impact creates run inline. Updates, deletes, and creates judged high-impact are written to a pending-actions table together with a snapshot of the row's prior state, and the tool returns a string telling the model it has staged rather than applied the change and must not claim otherwise. A separate function executes pending actions, and it does so behind a conditional update on the row still being pending, so concurrent confirmations resolve to one winner and the losers are handed the winner's terminal status. Editable columns are defined per domain in an allowlist, and anything not listed is dropped rather than rejected — a wallet can be renamed but its currency cannot be touched, and a wallet cannot be deleted at all. Auto-apply is gated by an earned trust flag, but deletes and high-impact changes are refused before trust is even consulted, and one undo resets the trust state.
→ Tier by blast radius, and put execution in a different function
Cost to undo
Loosening this is easy and dangerous, which is worth stating plainly: allowing the chat function to execute updates directly would be a small change, so the discipline has to be maintained deliberately. Tightening is also cheap — the allowlists and the high-impact rule are data, and the staging boundary already exists for every mutation kind. What is expensive to reverse is the decision to run the assistant under the user's own credentials rather than a service role. Every query in the agent, including the one that loads a row before staging an edit, omits its own tenant filter because row-level security supplies it. Moving to a privileged connection would mean auditing and rewriting all of that, and until then an RLS policy error is the one mistake that would cross tenants.
What happens when it breaks
The ways this design can break, and what it is set up to do when they happen.
The model calls a tool that does not exist, or emits malformed arguments
minor- Blast radius
- That tool call only. The conversation continues.
- Detection
- A catch-all in the tool dispatch; a JSON parse guard on the provider's tool-call payload.
- Mitigation
- An unknown tool returns a benign string saying no action was taken. Malformed arguments become empty arguments for that call alone rather than failing the turn.
A tool handler throws
minor- Blast radius
- One tool call. No partial write survives.
- Detection
- Caught at the dispatch boundary.
- Mitigation
- The error is returned to the model as a tool result that explicitly tells it not to claim success, instead of surfacing as a server error. Where a staged action was already written and execution then failed, the pending row is removed.
A confirmation is tapped twice, or replayed from an offline queue
major- Blast radius
- Would apply the same mutation twice if unguarded.
- Detection
- The conditional update on status matches zero rows for the loser.
- Mitigation
- Exactly one caller wins the claim. Losers receive the winner's terminal status, which is what lets the client's offline queue drop the item rather than retry forever. A failure during execution resets the row to pending so it can be retried.
A user asks the assistant to do something structural, like change a wallet's currency
minor- Blast radius
- None. The field is not in the allowlist.
- Detection
- None needed; unlisted fields are dropped when the patch is built.
- Mitigation
- Silently ignoring unlisted columns rather than erroring means a creative request degrades to a partial edit instead of an error the model might try to work around.
AI-derived data feeds back into AI-generated advice
major- Blast radius
- Would compound one extraction error into every subsequent insight.
- Detection
- Rows created from receipt scanning are written unconfirmed.
- Mitigation
- Every proactive job — insights, burn-rate nudges, morning summaries — reads only rows a human has confirmed. Machine-extracted data cannot influence generated advice until someone has looked at it.
Untrusted text in a transaction description attempts to redirect the model
major- Blast radius
- Bounded by what the tools can do, which for destructive operations is nothing.
- Detection
- Not detection-based by design. This threat is answered by bounding what the model can reach rather than by classifying the text it reads.
- Mitigation
- Confinement rather than instruction hardening: no write path for updates or deletes, enum values for categories and accounts generated from the user's own live data so an out-of-scope value is unrepresentable, search terms stripped to alphanumerics before reaching a query filter, and identifiers rejected at the door unless they match a UUID pattern. This is a real mitigation but it is not injection defence, and it should not be described as such.
What it costs to run
Each figure says where it came from, so you can judge how much weight to give it.
- 0
- Tools that can reach an update or delete
- 2
- Independent allowlists a write must satisfy
- 10
- Clean confirmations before auto-apply unlocks
- 40 s
- Turn budget
Structural, and the point of the design: the chat function's tool dispatch has no case that performs an update or a delete. Both write to the pending-actions table instead. Verifiable by reading the dispatch, not by sampling behaviour.
One at staging time in the chat function, one at execution time in the confirm function, kept in sync by hand. A third mirror exists client-side for undo. Defence in depth, at the cost of three copies and no test asserting they match.
Chosen threshold. Ten confirmations with no undo graduates a user to auto-applying low-impact mutations; a single undo resets it. Deletes and high-impact changes never auto-apply regardless.
Chosen after working out the unbounded case: a twelve-second model timeout across two providers over four tool iterations stacked to roughly 160 seconds. The wall-clock budget bounds it directly rather than trusting the arithmetic.
The longer version
Why this record exists
There is a lot of writing about getting language models to call tools. There is much less about what happens when a tool call is wrong and the tool writes to a database people depend on.
The framing I settled on is that a prompt cannot be a security boundary. Prompts are suggestions the model can misread, and they fail on exactly the inputs you did not anticipate. If the requirement is "the assistant must never delete a record without the user agreeing", then the only implementation that actually satisfies it is one where the code holding the model contains no deletion.
That reframing is what produced the two-function split. Everything else — the staging table, the snapshots, the trust flag — follows from wanting the guarantee to be a property of the code rather than of the model's behaviour on the day.
Where the tiering line falls, and why
Creates run inline and updates do not, which looks inconsistent until you consider recoverability. A spurious new transaction is visible in a list and can be deleted; the user loses thirty seconds. A spurious edit overwrites a number that no longer exists anywhere, and the user may not notice for a month.
High-impact creates are the exception that proves the rule: a large amount, or a change touching more than one money field at once, gets staged even though it is a create. The dimension that matters is not the SQL verb, it is how much damage survives the mistake.
Two things I would not claim in an interview
There is no prompt-injection defence here. What exists is capability confinement, which is a different and in some ways stronger thing, but it is not the same claim and I would rather say so than be caught conflating them. Nothing tags untrusted text as untrusted; the argument is simply that the model's reachable actions are too narrow for an injected instruction to do much with.
And the allowlist is enforced at more than one layer, across two runtimes. Each copy is deliberate defence in depth, but keeping several copies of one rule in agreement is a maintenance burden rather than a free win, and tightening that into a single asserted source of truth is the next thing I would do here.