Lesson 1 of 8 · 8 min read · last verified 2026-09-04
The text you did not write
In this lesson you will:
- Map every surface where text enters a workflow you have built
- Classify each surface by who controls the bytes, not by how likely harm seems
In P2 you built something that reads. You chose the documents, cut them up, ranked them and put the good ones in front of a model.
Every one of those documents was yours.
That is the assumption this module removes. Nearly every useful system ends up reading text that somebody else wrote, and the moment it does, a question you never had to ask becomes the most important one in the design: who decided what this says?
Untrusted is about control, not suspicion
“Untrusted” sounds like an accusation. It is not. It is a statement about who holds the pen.
A support ticket written by a customer is untrusted. So is a page your assistant fetches, a PDF a colleague uploads, a wiki anyone at your company can edit, and a product review. None of that implies the people involved are hostile. It means you cannot enumerate what the text might say, because you are not the one choosing.
The distinction matters because the usual instinct — “our users are fine, this is an internal tool” — answers the wrong question. It estimates likelihood. The design question is capability: if this text said the worst thing it could say, what would my system do?
Where text actually gets in
Most people list two or three surfaces and stop. The real list is longer, and the forgotten entries are where trouble lives.
Things a person types. Chat messages, form fields, search queries, filenames.
Things a person supplies. Uploaded documents, pasted text, linked URLs, email forwarded to an address your system watches.
Things your system fetches. Web pages, RSS feeds, API responses, database rows written by somebody else, files from shared storage.
Things that arrive as metadata. PDF titles, image alt text, HTML comments, spreadsheet cell notes, code comments, commit messages.
Things another model produced. The output of one assistant becoming the input of the next, which inherits everything the first one read.
That last one deserves its own moment. Chaining models is ordinary now, and each hop carries forward whatever the previous hop absorbed. A summariser that read a hostile page hands its summary to the next step as though it were your own text.
The map is the deliverable
Before any defence is worth discussing, you need the list. Not a general sense of it — the actual list, written down, with a name against each entry.
For each surface, three facts:
- Who can write here? You, a named colleague, any signed-in user, anyone on the internet.
- What does the system do with it? Display it, summarise it, retrieve it, pass it to a tool.
- What can happen next? Nothing, an answer, a database write, an email.
That third column is the one that turns an interesting problem into an urgent one, and L4 is entirely about it.
Our own map, honestly
Lrnon’s assistant reads an index built from Lrnon’s lessons. Today that content is written by a small team and reviewed before it merges, which makes it about as trusted as content gets.
It is not permanently so. The project accepts contributions, and a lesson is a document like any other. The day an outside contribution lands in that index, the assistant is reading text a stranger wrote — with no change to a single line of retrieval code.
That is the shape of most of these problems. The system does not become vulnerable when the code changes. It becomes vulnerable when the inputs do, which is a change nobody deploys and nobody reviews.
Try it now (10 minutes)
Take one workflow you have built or are planning. Write the table:
| Surface | Who can write it | What we do with it | What happens next |
|---|
Fill it in completely, including the boring rows. Then mark every row where column two is anything other than “me”.
Those rows are what the rest of this module is about. If the table has no such rows today, add one more: what would have to change for it to? On most projects the honest answer is “one feature request”.
Check your understanding
Recap
Untrusted means somebody else controls what the text says, which is a question about authorship rather than about how suspicious anyone seems. The surfaces are more numerous than they first appear — metadata, tool responses and other models’ output all count. Write the map before designing any defence, and pay attention to the last column, because what a system can do next is what turns this from interesting into urgent.
🗂 3 flashcards from this lesson join your daily review.