Scanning agent transcripts for secrets, without sending them anywhere
An in-depth look at building a 100% local PII detection and masking system for AI coding agent sessions.
Stock photo for illustration only, not from the actual event
- Coding agent transcripts store sensitive credentials in plain text on local disk storage.
- Cloud-based scanning creates leak risks, necessitating a 100% local processing solution.
- A composite detector combines Regex and a 1.6 GB quantized ONNX model for high accuracy.
- Constrained Viterbi decoding ensures precise token span masking without formatting errors.
You paste a .env into a Claude Code session to debug a failing deploy. The agent reads it, fixes the config, and moves on. Twenty minutes later the task is done and you close the terminal. The secret is still there. Not in the terminal, in the transcript. Every agent session writes a JSONL file to disk, and that file now holds your production database URL in clear text, sitting in a directory you will never think about again.
Multiply that by a few hundred sessions. That is the actual state of most machines running coding agents today. The obvious fix is to scan those transcripts and mask what you find. The obvious way to scan them is to send them to something that is good at finding secrets, which in 2026 means a cloud deployed model. To find out whether your transcripts contain credentials, you would upload all of your transcripts, including the ones that contain credentials, to a third party. The scan itself becomes the leak, which drove the requirement for a fully local solution.
Running AI models locally for security-critical tasks like credential detection is becoming an essential paradigm. As enterprises place a higher premium on strict data privacy, relying on third-party cloud infrastructure for sensitive artifacts like API tokens and database keys introduces unacceptable compliance and security surfaces.
Going local means balancing a model small enough to ship, fast enough to feel instant, and accurate enough to be worth turning on. The team's first instinct was regex, which successfully captures structured items like AWS access keys, JWTs, and credit card numbers with microsecond latency.
Stock photo for illustration only, not from the actual event
However, unstructured data like names, street addresses, and passphrases require a neural approach. The team integrated OpenAI's privacy filter, quantized to int8 and exported to ONNX, running through ONNX Runtime. Weights and tokenizer total just over 1.6 GB on disk, which presents a footprint challenge for a menu bar utility.
To mitigate the size impact, the feature remains off by default, downloading weights on demand with SHA-256 verification. Furthermore, decoding raw token outputs utilizes a constrained Viterbi pass to eliminate illegal state transitions, ensuring that every detected credential span is fully formed and properly formatted before string-level masking takes place.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment