Right, with respect to how many RLHF pairs, you can compare it to prior result of ~0%. But what I don't understand, and especially with the hype claims the paper makes about "autonomous super-human reasoning", is why can't they just keep running it and get much higher than 50%? Seems like there's another aspect that is preventing getting higher scores, and makes me wonder if these architectures are really just plateauing.
Don't get me wrong, it's some good work; it's just the language of the paper has some ridiculous hype.
Dustin / Dustin Dannenhauer
npub1mg…gpdjc
2025-05-12 21:43:38
in reply to nevent1q…3l6r
Author Public Key
npub1mgvwnpsqgrem7jfcwm7pdvdfz2h95mm04r23t8pau2uzxwsdnpgs0gpdjcPublished at
2025-05-12 21:43:38Event JSON
{
"id": "3db8ca926c637c8faf80cb27a465d8b1c5eb3f4c51d0aeeb31de9ba7458a1ed6",
"pubkey": "da18e9860040f3bf493876fc16b1a912ae5a6f6fa8d5159c3de2b8233a0d9851",
"created_at": 1747086218,
"kind": 1,
"tags": [
[
"e",
"23182d6a78752d9428cd77052fe666b28ac3d14b06497823d2781411c0ae5996",
"wss://relay.primal.net",
"root"
],
[
"e",
"d95c381db5fff384336216eb9b12c4fa09a08d67e69ea760ec4cdf1d35750d91",
"wss://relay.primal.net",
"reply"
],
[
"p",
"576d23dc3db2056d208849462fee358cf9f0f3310a2c63cb6c267a4b9f5848f9",
"",
"mention"
],
[
"p",
"82341f882b6eabcd2ba7f1ef90aad961cf074af15b9ef44a09f9d2a8fbfbe6a2",
"",
"mention"
],
[
"p",
"da18e9860040f3bf493876fc16b1a912ae5a6f6fa8d5159c3de2b8233a0d9851"
],
[
"r",
"wss://relay.getalby.com/v1"
],
[
"r",
"wss://nostr.wine/"
],
[
"r",
"wss://purplepag.es/"
],
[
"r",
"wss://relay.supertech.ai/"
],
[
"r",
"wss://relay.damus.io/"
],
[
"r",
"wss://bevo.nostr1.com/"
],
[
"r",
"wss://nostr-pub.wellorder.net/"
]
],
"content": "Right, with respect to how many RLHF pairs, you can compare it to prior result of ~0%. But what I don't understand, and especially with the hype claims the paper makes about \"autonomous super-human reasoning\", is why can't they just keep running it and get much higher than 50%? Seems like there's another aspect that is preventing getting higher scores, and makes me wonder if these architectures are really just plateauing. \n\nDon't get me wrong, it's some good work; it's just the language of the paper has some ridiculous hype. ",
"sig": "abf6b753a69fc2f4857cc293fe9a18e4bf5802c19f047831b6fd4669d8dc98ccfbc4379ce4daf4f04290738ee893fccdd867b52126897ba9d19f45d6b165fcca"
}