I said it before, but generative LLMs are not to be blindly trusted. I am not just talking about hallucinations, I am talking about understanding a problem, showing you its intermediate logic and then coming to a 90% accurate conclusion. See below for a recent interaction with GPT-4:

Me: Decode the following ciphertext

ChatGPT Sure, I can help with that. Please provide the ciphertext that you would like me to decode.

Me: Pm ol ohk hufaopun jvumpkluaphs av zhf, ol dyval pa pu jpwoly, aoha pz, if zv johunpun aol vykly vm aol slaalyz vm aol hswohila, aoha uva h dvyk jvbsk il thkl vba huk 1.

ChatGPT: The ciphertext appears to be encoded with a simple substitution cipher, likely a Caesar cipher, given the structure of the text. One common method to start decoding such ciphers is by trying out different shifts. Let's apply a common Caesar cipher shift of 7 and see if it decodes into readable text.

Here's how we can decode it:

1. Identify the shift used in the Caesar cipher.
2. Shift each letter in the ciphertext backward by the same amount to get the plaintext.

Let's try shifting backward by 7 positions (since 'Pm' starts at 'C' with a backward shift of 7):

Given the apparent regularity, let's decode the whole ciphertext with a shift of 7:

Pm ol ohk hufaopun jvumpkluaphs av zhf, ol dyval pa pu jpwoly, aoha pz, if zv johunpun aol vykly vm aol slaalyz vm aol hswohila, aoha uva h dvyk jvbsk il thkl vba huk 1.

Decoding using the shift of 7:

If he had explained sufficiently to say, he would be in secret, that is, by so changing the order of the letters of the alphabet, that not a word could be made out and 1.

The problem is, here is ChatGPT's result followed by the actual, correct result:

If he had explained sufficiently to say, he would be in secret, that is, by so changing the order of the letters of the alphabet, that not a word could be made out and 1.

If he had anything confidential to say, he wrote it in cipher, that is, by so changing the order of the letters of the alphabet, that not a word could be made out and 1.

I am confounded as to why it got it wrong. Did it not actually decode it but instead tried to build it from fragments of its memory? That phrase (slightly modified) is a quote from Suetonius so it is on the internet, but not in the form the LLM spat it out.

Update: I tried Copilot (which is based on GPT-4) and set it to More accurate. Wow.