Today in "LLMs can't do even simple reasoning":
Prompt: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?
See 60 LLMs fail: https://benchmarks.llmonitor.com/sally
tante /
npub1de…gtq8a
2023-09-12 10:00:56
Author Public Key
npub1dedm8axlwewsysrxlw93kz43j6mgwf4cpync9gu8e95cl5za3s6s4gtq8aPublished at
2023-09-12 10:00:56Event JSON
{
"id": "73cdf9ff68a1106cf9b22a6f3be20393a028a1ad750b1db9ca5303fc766d943d",
"pubkey": "6e5bb3f4df765d024066fb8b1b0ab196b68726b8092782a387c9698fd05d8c35",
"created_at": 1694512856,
"kind": 1,
"tags": [
[
"proxy",
"https://tldr.nettime.org/users/tante/statuses/111051594557347916",
"activitypub"
]
],
"content": "Today in \"LLMs can't do even simple reasoning\":\n\nPrompt: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?\n\nSee 60 LLMs fail: https://benchmarks.llmonitor.com/sally",
"sig": "17eef5da70abe0e90e493b9882a5fd1a5075bd2d478f314c4053adc702326c44a504b332dffb943772b55692cb48f4297fae019b922056a59c0067057db77f00"
}