It's a good thing you've already decided what answer you want, so you can safely dismiss the generalization of all possible evidence on the basis of "that specific scenario didn't convince me so nothing could possibly happen". You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, x.com/OwainEvans_UK/status/189443663705… , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.

LLMs are modelling the world, not just "predicting the next token". They are certainly not akin to parrots. Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. [1] arxiv.org/abs/2405.15943 [2] x.com/OwainEvans_UK/status/189443663705… [3] anthropic.com/research/tracing-thoughts…...

It's linked in the twitter thread from the authors: x.com/OwainEvans_UK/status/189443663705… specifically: x.com/ESYudkowsky/status/18944533762153…

>But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes. This is the very point of contention. You don't get to just assume it. > it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. Another highly contentious point you are just outright assuming. LLMs are modelling the world, not just "predicting the next token". Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. It's interesting how the people with the strongest opinions about LLMs don't seem to understand them. [1] arxiv.org/abs/2405.15943 [2] x.com/OwainEvans_UK/status/189443663705… [3] anthropic.com/research/tracing-thoughts…...

LLMs are modelling the world, not just "predicting the next token". They are not akin to "stochastic parrots". Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. There are so many interesting things to say about LLMs, yet somehow the conversation about them is stuck in 2021. [1] arxiv.org/abs/2405.15943 [2] x.com/OwainEvans_UK/status/189443663705… [3] anthropic.com/research/tracing-thoughts…...