Can LLMs be used in reverse?

When you use an AI model, the chat is just one long continuous string. LLMs are only trained to generate what comes after a special token like <assistant_start>, and they get stopped when it's the user's turn to type.

So, what happens if we don't stop them and we let them write the user message? (no finetuning allowed, obviously!)

Chat with reversed model Tell me more! (scroll down)

Models read a single big block of text, where special tokens mark which messages come from the assistant and which come from the user.

When you send a message, it gets wrapped between tokens like <user_start> and <user_end>. The assistant, on the other hand, keeps generating until it produces something like <assistant_end>, which stops it so that you (the user) can finally send another message.

The exact format/tokens used depend on the specific model.

So, we can just use the template in reverse. After <assistant_end>, we add <user_start> and let the model generate from there, so it writes the user message.

I also wanted to test the other obvious way to simulate human messages: simply asking the model to "be human". So there are two configurations: native, which uses the chat template in reverse, and roleplay, which asks the model to "be human please".

After some testing I chose GPT 6 Astra as a judge (see ). Astra sees the chat so far and two replies, the real one and the AI one, and has to say which one the person wrote (each pair is shown in both orders). A model's score is how often it fools the judge: 50% means the judge can't tell. I also made a quick test so you can try to be the judge →

So, finally, here are the results! It's clear that no model comes close to emulating humans well, not even UserLM, which was trained to do exactly that. For reference, the shaded band is how often a real human message taken from another chat fools the judge. It's also clear that native is much better than roleplay in pretty much all the tested models. Also, notice the difference in scores between follow-ups and first messages: it's much easier for models to perform well in the former (as expected).

Bigger is not better. More parameters don't make a model better at fooling the judge; the family a model comes from matters much more.

Roleplay ruins message variety. Real people opened the conversations in (counting the first three words). Models in native mode are surprisingly varied in their openings. In roleplay mode they are WAY less varied.

Here, each square is a conversation: stronger shades mean more models fooled the judge in both orders. Click on the squares to see the details.

Caveats:

Astra does not support logprobs, so the results are less precise than they could have been.
Messages are cut at tokens or the first blank line. This is due to the fact that models almost never print end tokens in the user slot.
UserLM was used without its "intent" prompt. I checked the perplexity over the chats with and without it, and it's pretty much the same. However, the prompt significantly improved end token generation. Using it would require re-running the generation + judging, but I no longer have a ChatGPT subscription to do that.