Why AI can't spell "strawberry".
https://techcrunch.com/2024/08/27/why-ai-cant-spell-strawberry/
I've said before that what we are currently calling AI models, which is really just Large Language Models or LLMs, don't understand anything at all except language. They're language models. That's what they do, and it's all they do.
Except that's not quite true, because they don't understand language in any real way either.
>The failure of large language models to understand the concepts of letters and syllables is indicative of a larger truth that we often forget: These things don't have brains. They do not think like we do. They are not human, nor even particularly humanlike.
>Most LLMs are built on transformers, a kind of deep learning architecture. Transformer models break text into tokens, which can be full words, syllables, or letters, depending on the model.
>"LLMs are based on this transformer architecture, which notably is not actually reading text. What happens when you input a prompt is that it's translated into an encoding," Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, told TechCrunch. "When it sees the word 'the,' it has this one encoding of what 'the' means, but it does not know about 'T,' 'H,' 'E.'"
Typeahead with delusions of grandeur.
Real Aetherness /
npub13y…gwy87
2024-08-28 10:25:13
Author Public Key
npub13y47q8rfttwrtssl3677d5jg4ghh93hr6ygwp4qjv8nvewt65kvqzgwy87Published at
2024-08-28 10:25:13Event JSON
{
"id": "f173bfd4272164231c2d8d3bbec2ca6c76169a961d7063498487f6a57c14b735",
"pubkey": "892be01c695adc35c21f8ebde6d248aa2f72c6e3d110e0d41261e6ccb97aa598",
"created_at": 1724840713,
"kind": 1,
"tags": [
[
"proxy",
"https://poa.st/objects/37697f17-35f5-4fb5-8076-78d2732629b3",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://poa.st/objects/37697f17-35f5-4fb5-8076-78d2732629b3",
"pink.momostr"
],
[
"-"
]
],
"content": "Why AI can't spell \"strawberry\".\n\nhttps://techcrunch.com/2024/08/27/why-ai-cant-spell-strawberry/\n\nI've said before that what we are currently calling AI models, which is really just Large Language Models or LLMs, don't understand anything at all except language. They're language models. That's what they do, and it's all they do.\n\nExcept that's not quite true, because they don't understand language in any real way either.\n\n\u003eThe failure of large language models to understand the concepts of letters and syllables is indicative of a larger truth that we often forget: These things don't have brains. They do not think like we do. They are not human, nor even particularly humanlike.\n\n\u003eMost LLMs are built on transformers, a kind of deep learning architecture. Transformer models break text into tokens, which can be full words, syllables, or letters, depending on the model.\n\n\u003e\"LLMs are based on this transformer architecture, which notably is not actually reading text. What happens when you input a prompt is that it's translated into an encoding,\" Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, told TechCrunch. \"When it sees the word 'the,' it has this one encoding of what 'the' means, but it does not know about 'T,' 'H,' 'E.'\"\n\nTypeahead with delusions of grandeur.",
"sig": "68b2b5f6d3a4c20bb8541433ef9fa58eac7dab2c863978913e3abb8f911005b99805c12640eb6479b6d96c1f0119d48494b986b37368863a14c1d47d60ae373b"
}