Why Hacker News recommends it
4 comments from 4 distinct recommending readers, between 2023 and 2023.
- Recommends
The Twitter thread is worth looking at. Very Fascinating twitter.com/wesg52/status/1709551516577…...
Language Models Represent Space and TimeRead the thread - Recommends
No. We know that LLMs make world models. (Often literally: twitter.com/wesg52/status/1709551516577…). I should also point out that your tone is aggressive, so it makes me think you're not interested in learning. But I'm going to proceed anyway. It cannot understand these properties of words well because it does not operate on words, it operates on tokens. You can see examples here: platform.openai.com/tokenizer This makes it much more difficult to learn how to reason about particular data that appears inside the token, because it does not ever receive that information. The only way it could get that information is if the training data explicitly attempted to work around this limitation. But I would also appreciate insight from someone with a deeper understanding of the internals of these big LLMs.
LLMs can't self-correct in reasoning tasks, DeepMind study findsRead the thread - Recommends
While one might well argue over whether they "must have" this, it's clear that they do end up with internal models of the world. Even quite literally: twitter.com/wesg52/status/1709551516577…
A case for the capacity to reason of GPT-4Read the thread - Recommends
They've already researched this and have found model inside the LLM such as a map of the world - x.com/wesg52/status/1709551516577902782. Understanding is key to how so much data can be compressed into a LLM. There really isn't a better way to store all of it better than plain understanding it.
OpenAI researchers warned board of AI breakthrough ahead of CEO ousterRead the thread