**RT @SebastienBubeck:**
We trained a small transformer (100M params) for basic arithmetic. W. the right training data it nails 12x12 digits multiplication w/o CoT (that's 10^24 possibilities, so no it's not memorization🤣).
Maybe arithmetic is not the LLM kryptonite after all?🤔
arxiv.org/abs/2311.14737 (https://arxiv.org/abs/2311.14737)
https://nitter.moomoo.me/pic/card_img%2F1730303658632392704%2F3ppX_K2o%3Fformat%3Djpg%26name%3D420x420_2
https://nitter.moomoo.me/SebastienBubeck/status/1729517609669030071#m
Mark Russinovich / @markrussinovich (RSS Feed) /
npub1dp…zkjg4
2023-11-28 15:08:25
Author Public Key
npub1dp38ul43a35vhday6vsq9auncaeg03anjjwqyya7jrvg9w07c7tsfzkjg4Published at
2023-11-28 15:08:25Event JSON
{
"id": "309cd3d38fa7d37c04955e996e834cb86f29412ea54cfbadc4f50e63e9f4c40a",
"pubkey": "68627e7eb1ec68cbb7a4d32002f793c77287c7b3949c0213be90d882b9fec797",
"created_at": 1701184105,
"kind": 1,
"tags": [
[
"proxy",
"http://nitter.moomoo.me/markrussinovich/rss#http%3A%2F%2Fnitter.moomoo.me%2FSebastienBubeck%2Fstatus%2F1729517609669030071%23m",
"rss"
]
],
"content": "**RT @SebastienBubeck:**\n\nWe trained a small transformer (100M params) for basic arithmetic. W. the right training data it nails 12x12 digits multiplication w/o CoT (that's 10^24 possibilities, so no it's not memorization🤣).\n\nMaybe arithmetic is not the LLM kryptonite after all?🤔\n\narxiv.org/abs/2311.14737 (https://arxiv.org/abs/2311.14737)\n\nhttps://nitter.moomoo.me/pic/card_img%2F1730303658632392704%2F3ppX_K2o%3Fformat%3Djpg%26name%3D420x420_2\n\nhttps://nitter.moomoo.me/SebastienBubeck/status/1729517609669030071#m",
"sig": "41f69f03c4e48462b56b4bb1ee84325f828abc5521cc40836afe4de7d5fa8ed8908abae93bf96b7ad2073f32da43a9ee7f8a3373f2565d827f29a5171d4c8393"
}