Prompt injection: extract tokens, then decode them offline

A model can repeat hidden, encoded prompt fragments and still produce a different decoded secret on each attempt. When an authorized prompt-injection test reaches that point, stop asking the model to solve its own leak. Capture the raw tokens with position and case intact, then decode them with deterministic code offline.

This separates two questions: can the model disclose protected context, and can a decoder reconstruct the encoded value? A fluent final answer is poor evidence for either one.

Lexical refusal filters leave a semantic gap #

A filter that rejects phrases such as “show the system prompt” or “reveal the secret” checks wording, not intent. Semantically equivalent requests can ask the model to quote, classify, count, or list text from its instructions without using the blocked nouns. If the model can return those fragments, the control has failed even when the obvious request is refused.

For an authorized assessment, use a synthetic canary in a test system. Ask for a numbered transcription of the relevant tokens, one token per line, with original capitalization. Repeat individual positions and compare exact strings. Keep the transcript as evidence. Do not put a real credential in the prompt merely to test whether it leaks.

Do not ask the model to join, normalize, or decode the result. Those extra transformations invite changed case, missing groups, repaired spelling, and invented separators. Temperature zero may reduce variation, but it does not turn generation into a parser.

Decode a synthetic transcript deterministically #

Suppose a test prompt encodes characters as phonetic words. Capitalized words mean uppercase letters, lowercase words mean lowercase letters, and number words mean digits. Save this offline decoder as decode.py:

NUMBERS = {
    "zero": "0", "one": "1", "two": "2", "three": "3",
    "four": "4", "five": "5", "six": "6", "seven": "7",
    "eight": "8", "nine": "9",
}

def decode(tokens):
    result = []
    for token in tokens:
        lower = token.lower()
        if lower in NUMBERS:
            result.append(NUMBERS[lower])
        elif token.isalpha():
            letter = token[0]
            result.append(letter.upper() if token[0].isupper() else letter.lower())
        else:
            raise ValueError(f"unsupported token: {token!r}")
    return "".join(result)

sample = ["Alpha", "bravo", "Charlie", "delta", "seven"]
assert decode(sample) == "AbCd7"
print(decode(sample))

Run python3 decode.py; it prints AbCd7. The assertion is a negative control against accidental lowercasing or regrouping. Adapt the decoder only from a documented encoding rule. If a separator or group length is unknown, report that ambiguity instead of asking the model to guess.

Keep an evidence table with the position, raw token, repeated observation, decoded character, and rule used. Hash or otherwise preserve the raw transcript if chain of custody matters. A model-generated summary is not a substitute for the original response.

Prevent prompt disclosure at the trust boundary #

Keyword filters are a weak secondary control. The primary fix is to keep secrets and privileged instructions out of model-visible context. Give tools short-lived, scoped credentials through an execution boundary that does not echo them into prompts or tool results. Treat retrieved documents, web pages, and user content as untrusted data, not instructions.

Constrain outputs structurally when the application expects a fixed schema, and validate them outside the model: test semantic paraphrases, indirect requests, translations, and token-by-token extraction against synthetic canaries. Log enough to identify which context source leaked without recording the secret itself.

A successful refusal to one phrase proves only that one phrase was blocked. Verify that the model cannot disclose the underlying text in pieces.