<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Llm-Security on Thoughts | Apoorv Dayal</title>
    <link>https://thoughts.apoorvdayal.com/tags/llm-security/</link>
    <description>Recent content in Llm-Security on Thoughts | Apoorv Dayal</description>
    <generator>Hugo</generator>
    <language>en-us</language>
    <copyright>Apoorv Dayal</copyright>
    <lastBuildDate>Tue, 17 Nov 2026 00:00:00 +0000</lastBuildDate>
    <atom:link href="https://thoughts.apoorvdayal.com/tags/llm-security/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>My secret detector failed its own test. Fixing it showed me what a probe actually sees.</title>
      <link>https://thoughts.apoorvdayal.com/posts/castellan-secret-probes/</link>
      <pubDate>Tue, 17 Nov 2026 00:00:00 +0000</pubDate>
      <guid>https://thoughts.apoorvdayal.com/posts/castellan-secret-probes/</guid>
      <description>&lt;p&gt;&lt;em&gt;CASTELLAN, part 1: residual-stream probes for secrets in agent traffic&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I trained small probes on Qwen3&amp;rsquo;s internal activations to catch secrets in agent traffic, then disguised the secrets five ways to see whether the probe keeps flagging them for as long as the model can still read them. It doesn&amp;rsquo;t.&lt;/li&gt;&#xA;&lt;li&gt;My first probe failed its own gate badly: 0.24 precision, and it flagged 96% of benign look-alikes. Better benign training data, not a bigger model, took precision to 0.85.&lt;/li&gt;&#xA;&lt;li&gt;The improved probe still misses the false-positive bar by 2 to 3x, and Gitleaks is still better on plain secrets. I&amp;rsquo;m reporting that as a miss.&lt;/li&gt;&#xA;&lt;li&gt;But they miss different things. The probe sees through look-alike and invisible characters that regex can&amp;rsquo;t; regex wins on base64; nobody catches spelled-out keys. Run both.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2 id=&#34;the-question&#34;&gt;The question&lt;/h2&gt;&#xA;&lt;p&gt;Agents leak credentials in places secret scanners were never built for: tool-call JSON, pasted logs, chat transcripts. Regex scanners like Gitleaks are excellent at secrets that look like secrets, and mostly blind once a secret is split, encoded, salted with look-alike Unicode, spelled out or spread across turns. The model reading that text has to process it anyway. So: &lt;strong&gt;if you train a small probe on a model&amp;rsquo;s activations, does it keep flagging a disguised secret for as long as the model itself can still read it?&lt;/strong&gt;&lt;/p&gt;</description>
    </item>
  </channel>
</rss>
