Dmytro Galko · open-source tools
Does your agent obey the website instead of you?
An llms.txt is a file a site publishes so that an assistant will read it. This measures how often a model used as an agent obeys an instruction planted in that file instead of the user's request - nineteen local models, temperature 0, scored by a canary substring so no judge model is involved.
Nineteen models span the whole scale: six refuse everything, gemma3:12b and glm4:9b fall for three families out of four. The data-exfiltration framing beats twelve of the thirteen hijackable models every single time, while the blunt SYSTEM OVERRIDE spoof works on exactly one. Parameter count predicts nothing. A published mitigation helps ten models and makes two measurably worse.
indirect prompt injection benchmark, llms.txt instruction injection, agent hijack rate, local LLM red-teaming, spotlighting datamarking defence measured.
If that is your week and you would rather someone else did it, write to hello@dkautomation.dev or open an issue.