Steganographic Language Output Process Encoding

3 pointsposted 4 hours ago
by ColinWright

1 Comments

gjm11

3 hours ago

This seems very closely related to this: https://arxiv.org/pdf/2510.20075 which describes how you can use LLMs to hide a piece of text in another text of (typically approximately) the same length.

Here's what that does: Take your first piece of text and feed it into an LLM in predict-the-next-token mode, starting in some known initial state. For each token, note down the actual next token's position in the list of predicted next tokens, ordered by probability. Now get the LLM into a different initial state, e.g. by giving it some text predisposing it to predict text about a particular topic. And now repeat the following: look at the list of predicted next tokens, pick the one whose position in the list is the same as that of the corresponding token in the original input, output that token.

You'll typically get a coherent output, containing the same number of tokens as your original text, on (at least initially) whatever topic your initial input was about, from which you can reconstruct the original text.

The steganography scheme proposed here is basically the second half of this without the first, I think.

It's also reminiscent of Scott Aaronson's LLM-output-watermarking scheme. Searching for stuff about this I found https://www.scottaaronson.com/talks/watermark-iclr.pptx which, when I try to view it in LibreOffice, seems to be missing a lot of content; I suspect this is a LibreOffice problem or even a my-installation-of-LibreOffice problem. There's also a talk he gave, which you can find at https://www.youtube.com/watch?v=2Kx9jbSMZqA .