Can we use LLMs in reverse?

When you use an AI model, the chat is just one long continuous string. LLMs are only trained to generate what comes after a special assistant_starts_here token, and they get stopped when it's the user's turn to type.

So I tried to see what happens if we let LLMs generate the user messages without stopping them (no finetuning allowed, obviously!).

Chat with reversed model Tell me more! (scroll down)

Models read a single big block of text, where special tokens mark which messages come from the assistant and which come from the user.

When you send a message, it gets wrapped between tokens like <user_start> and <user_end>. The assistant, on the other hand, keeps generating until it produces something like <assistant_end>, which stops it so that you (the user) can finally send another message.

The exact format/tokens used depend on the specific model.

So, we can just use it in reverse. Instead of letting the model write between its special assistant start and end tokens, we put the user's messages there and let the model generate right after the user start token.

I also wanted to test the other obvious way to simulate human messages: simply asking the model to "be human". So there are two configurations: native, which uses the chat template in reverse, and roleplay, which asks the model to "be human please".

After some testing I chose GPT 6 Astra as a judge (see sanity checks below). Astra chooses between an AI response and a human response, and sees each in both orders. I also made a quick test so you can try to be the judge →

So, finally, here are the results! It's clear that no model comes close to emulating humans well, not even UserLM which was trained to do exactly that. Its also clear that native is much better than roleplay in pretty much all the tested models. Also, notice the difference in scores between follow-ups and first messages, its much easier for models to perform well in the former (as its expected).

Bigger is not better. More parameters do not fool better, the family a model comes from matters much more.

Roleplay ruins message variety. Real people opened the conversations in different ways (counting the first three words). Models in native mode are surprisingly varied in their openings. In roleplay mode they are WAY less varied.

Here, each square is a conversation: stronger shades mean more models fooled the judge into taking their message for the human one. Click on the squares to see the details.

Caveats:

Astra does not support logprobs, so the results are less precise than they could have been.
Messages are cut at tokens or the first blank line. This is due to the fact that models almost never print end tokens in the user slot.
UserLM was used without its "intent" prompt. I checked the perplexity over the chats with and without it, and it's pretty much the same. However, it significantly improved end token generation. It would require re-running the generation + judging, but I no longer have a ChatGPT subscription to do that.