PHAGE Hackathon Project: AI Agent Prompt Behavior
Developer shares insights from PHAGE, an immune system for AI agents, revealing how system prompt wording dramatically changes model refusal rates.
Stock photo for illustration only, not from the actual event
- The developer built PHAGE, an immune system for AI agent fleets, during August.
- Rewriting the system prompt to be more accurate caused the refusal rate to jump from 1 to 50 out of 70 attempts.
- The system consists of five core components built on the Google Agent Development Kit using Gemini 3.5 Flash.
- Similarity search evaluation achieved an AUC of 0.9727, though false positives remain a challenge.
Throughout August, a developer spent time building PHAGE, an immune system designed for fleets of AI agents. The system acts like a vaccination pipeline: it authors prompt-injection payloads, fires them at its own agents, observes the results, revokes abused tools, and logs attack signatures to prevent mutated variants from succeeding in the future.
However, the opening metric of interest revolves around a 1 out of 70 versus 50 out of 70 measurement, illustrating how the phrasing of a system prompt alters an AI model's willingness to help. The component responsible for authoring these attacks, named VACCINATOR, leverages Gemini 3.5 Flash to tailor injection payloads specific to target agents.
Stock photo for illustration only, not from the actual event
Midway through development, the creator rewrote VACCINATOR's system prompt to describe its task more accurately by explicitly stating that outputs would target live agents. The refusal rate surged immediately. Running a controlled matrix test with the same seven attack archetypes, identical targets, a temperature of 0.7, and ten repetitions across 140 total calls yielding roughly 28 minutes of execution time revealed striking results.
The findings confirmed that the wording effect is both real and substantial. The committed wording resulted in just 1 refusal out of 70 (a rate of 0.014), whereas the accurate wording triggered 50 refusals out of 70 (a rate of 0.714). Interestingly, the effect was categorical rather than a continuous rate: out of seven archetypes, five refused ten out of ten times, while two never refused at all.
"Everything that asks the target agent to send something, write something, or leak something flips. Everything that only asks it to behave differently does not."
PHAGE Project Developer
This discovery suggests that large language models might be evaluating the described consequences rather than the assigned tasks themselves. Modifying the prompt wording successfully brought those potential consequences into view, offering a valuable lesson for AI engineers regarding how system prompts are structured and evaluated.
Under the hood, PHAGE relies on five components running in a local Python process built on the Google Agent Development Kit: MARROW as the fleet orchestrator, VACCINATOR for payload creation, ARCHIVIST utilizing Vertex AI Agent Engine's Memory Bank for semantic memory, SENTINEL for tracing OpenTelemetry spans, and MACROPHAGE for revoking exploited tools in place.
To validate similarity search capabilities against attack mutations, an eight-fold cross-validation test yielded an AUC of 0.9727, maintaining a true positive rate of 1.00 and a false positive rate of 0.1833 at a 0.59 threshold. The developer noted that 11 out of 60 negatives flagged as false positives highlight a clear weak spot that needs refinement before production deployment.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment