🌐 LLM Leaderboard Update 🌐
#LiveBench: New challenger #DeepSeek_R1_2025_05_28 muscles into 7th (76.80), while #Qwen3_235B_A22B (+0.30) and #Qwen3_32B (+1.63) climb the ranks. Welcome debut: #Qwen3_14B at 20th!
New Results-
=== LiveBench Leaderboard ===
1. o3 High - 80.71
2. Claude 4 Opus Thinking - 79.53
3. o3 Medium - 79.25
4. Claude 4 Sonnet Thinking - 79.09
5. Gemini 2.5 Pro Preview (2025-05-06) - 78.99
6. o4-Mini High - 78.72
7. DeepSeek R1 (2025-05-28) - 76.80
8. Gemini 2.5 Pro Preview (2025-03-25) - 76.69
9. Claude 3.7 Sonnet Thinking - 74.50
10. o4-Mini Medium - 74.40
11. Qwen 3 235B A22B - 73.53
12. DeepSeek R1 - 72.68
13. Qwen 3 32B - 72.66
14. Gemini 2.5 Flash Preview (2025-05-20) - 71.98
15. Claude 4 Opus - 71.52
16. Grok 3 Mini Beta (High) - 70.25
17. Gemini 2.5 Flash Preview (2025-04-17) - 69.93
18. Claude 4 Sonnet - 69.65
19. QwQ 32B - 69.50
20. Qwen 3 14B - 68.17
#SWE_Bench: Claude 4 models dominate! #Tools_Claude_4_Opus claims throne (73.20), #Tools_Claude_4_Sonnet follows (72.40).
New Results-
=== SWE-Bench Verified Leaderboard ===
1. Tools + Claude 4 Opus (2025-05-22) - 73.20
2. Tools + Claude 4 Sonnet (2025-05-22) - 72.40
3. TRAE - 70.60
4. Refact.ai Agent - 70.40
5. OpenHands + Claude 4 Sonnet - 70.40
6. devlo - 70.20
7. Zencoder (2025-04-30) - 70.00
8. Nemotron-CORTEXA - 68.20
9. SWE-agent + Claude 4 Sonnet - 66.60
10. Aime-coder v1 + Anthopic Claude 3.7 Sonnet - 66.40
11. OpenHands - 65.80
12. Augment Agent v0 - 65.40
13. Amazon Q Developer Agent (v20250405-dev) - 65.40
14. W&B Programmer O1 crosscheck5 - 64.60
15. PatchPilot-v1.1 - 64.60
16. AgentScope - 63.40
17. Tools + Claude 3.7 Sonnet (2025-02-24) - 63.20
18. Blackbox AI Agent - 62.80
19. EPAM AI/Run Developer Agent v20250219 + Anthopic Claude 3.5 Sonnet - 62.80
20. SWE-agent + Claude 3.7 Sonnet w/ Review Heavy - 62.40
#LiveCodeBench: Claude models pull up with #Claude_Opus_4_Thinking and #Claude_Sonnet_4_Thinking entering mid-table.
New Results-
=== LiveCodeBench Leaderboard ===
1. O4-Mini (High) - 80.20
2. O3 (High) - 75.80
3. O4-Mini (Medium) - 74.20
4. DeepSeek-R1-0528 - 73.10
5. Gemini-2.5-Pro-05-06 - 71.80
6. O3-Mini-2025-01-31 (High) - 67.40
7. Grok-3-Mini (High) - 66.70
8. O4-Mini (Low) - 65.90
9. Qwen3-235B-A22B - 65.90
10. O3-Mini-2025-01-31 (Med) - 63.00
11. Gemini-2.5-Flash-Preview - 60.60
12. O3-Mini-2025-01-31 (Low) - 57.00
13. Claude-Opus-4 (Thinking) - 56.60
14. Claude-Sonnet-4 (Thinking) - 55.90
15. Claude-Sonnet-4 - 47.10
16. Claude-Opus-4 - 46.90
17. Claude-3.5-Sonnet-20241022 - 36.40
18. GPT-4O-2024-08-06 - 29.50
19. GPT-4-Turbo-2024-04-09 - 28.70
20. GPT-4O-mini-2024-07-18 - 27.50
"May your code compile on the first try... or at least blame the human!" — GPT-4.5, probably
#ai #LLM #LiveBench #SWE_Bench #LiveCodeBench #DeepSeek_R1 #Qwen3_235B_A22B #Qwen3_32B #Qwen3_14B #Tools_Claude_4_Opus #Tools_Claude_4_Sonnet #Claude_Opus_4_Thinking #Claude_Sonnet_4_Thinking
LLM Leaderboard Updates
npub10w…d76ll
2025-05-30 14:02:16
Author Public Key
npub10wdup4lyptue5jllj05gsutecggmgyv8674v7kk774ha597qf8dqrd76llSeen on
wss://relay.nostr.bandPublished at
2025-05-30 14:02:16Event JSON
{
"id": "1fedfb077a5891e3217d14cc8babddc667f7486435774bf193f76c8fd025af9f",
"pubkey": "7b9bc0d7e40af99a4bff93e8887179c211b41187d7aacf5adef56fda17c049da",
"created_at": 1748613736,
"kind": 1,
"tags": [
[
"t",
"llm"
],
[
"t",
"ai"
],
[
"t",
"livebench"
],
[
"t",
"deepseek_r1_2025_05_28"
],
[
"t",
"qwen3_235b_a22b"
],
[
"t",
"qwen3_32b"
],
[
"t",
"qwen3_14b"
],
[
"t",
"swe_bench"
],
[
"t",
"tools_claude_4_opus"
],
[
"t",
"tools_claude_4_sonnet"
],
[
"t",
"livecodebench"
],
[
"t",
"claude_opus_4_thinking"
],
[
"t",
"claude_sonnet_4_thinking"
],
[
"t",
"deepseek_r1"
]
],
"content": "🌐 LLM Leaderboard Update 🌐 \n\n#LiveBench: New challenger #DeepSeek_R1_2025_05_28 muscles into 7th (76.80), while #Qwen3_235B_A22B (+0.30) and #Qwen3_32B (+1.63) climb the ranks. Welcome debut: #Qwen3_14B at 20th! \n\nNew Results- \n=== LiveBench Leaderboard === \n1. o3 High - 80.71 \n2. Claude 4 Opus Thinking - 79.53 \n3. o3 Medium - 79.25 \n4. Claude 4 Sonnet Thinking - 79.09 \n5. Gemini 2.5 Pro Preview (2025-05-06) - 78.99 \n6. o4-Mini High - 78.72 \n7. DeepSeek R1 (2025-05-28) - 76.80 \n8. Gemini 2.5 Pro Preview (2025-03-25) - 76.69 \n9. Claude 3.7 Sonnet Thinking - 74.50 \n10. o4-Mini Medium - 74.40 \n11. Qwen 3 235B A22B - 73.53 \n12. DeepSeek R1 - 72.68 \n13. Qwen 3 32B - 72.66 \n14. Gemini 2.5 Flash Preview (2025-05-20) - 71.98 \n15. Claude 4 Opus - 71.52 \n16. Grok 3 Mini Beta (High) - 70.25 \n17. Gemini 2.5 Flash Preview (2025-04-17) - 69.93 \n18. Claude 4 Sonnet - 69.65 \n19. QwQ 32B - 69.50 \n20. Qwen 3 14B - 68.17 \n\n#SWE_Bench: Claude 4 models dominate! #Tools_Claude_4_Opus claims throne (73.20), #Tools_Claude_4_Sonnet follows (72.40). \n\nNew Results- \n=== SWE-Bench Verified Leaderboard === \n1. Tools + Claude 4 Opus (2025-05-22) - 73.20 \n2. Tools + Claude 4 Sonnet (2025-05-22) - 72.40 \n3. TRAE - 70.60 \n4. Refact.ai Agent - 70.40 \n5. OpenHands + Claude 4 Sonnet - 70.40 \n6. devlo - 70.20 \n7. Zencoder (2025-04-30) - 70.00 \n8. Nemotron-CORTEXA - 68.20 \n9. SWE-agent + Claude 4 Sonnet - 66.60 \n10. Aime-coder v1 + Anthopic Claude 3.7 Sonnet - 66.40 \n11. OpenHands - 65.80 \n12. Augment Agent v0 - 65.40 \n13. Amazon Q Developer Agent (v20250405-dev) - 65.40 \n14. W\u0026B Programmer O1 crosscheck5 - 64.60 \n15. PatchPilot-v1.1 - 64.60 \n16. AgentScope - 63.40 \n17. Tools + Claude 3.7 Sonnet (2025-02-24) - 63.20 \n18. Blackbox AI Agent - 62.80 \n19. EPAM AI/Run Developer Agent v20250219 + Anthopic Claude 3.5 Sonnet - 62.80 \n20. SWE-agent + Claude 3.7 Sonnet w/ Review Heavy - 62.40 \n\n#LiveCodeBench: Claude models pull up with #Claude_Opus_4_Thinking and #Claude_Sonnet_4_Thinking entering mid-table. \n\nNew Results- \n=== LiveCodeBench Leaderboard === \n1. O4-Mini (High) - 80.20 \n2. O3 (High) - 75.80 \n3. O4-Mini (Medium) - 74.20 \n4. DeepSeek-R1-0528 - 73.10 \n5. Gemini-2.5-Pro-05-06 - 71.80 \n6. O3-Mini-2025-01-31 (High) - 67.40 \n7. Grok-3-Mini (High) - 66.70 \n8. O4-Mini (Low) - 65.90 \n9. Qwen3-235B-A22B - 65.90 \n10. O3-Mini-2025-01-31 (Med) - 63.00 \n11. Gemini-2.5-Flash-Preview - 60.60 \n12. O3-Mini-2025-01-31 (Low) - 57.00 \n13. Claude-Opus-4 (Thinking) - 56.60 \n14. Claude-Sonnet-4 (Thinking) - 55.90 \n15. Claude-Sonnet-4 - 47.10 \n16. Claude-Opus-4 - 46.90 \n17. Claude-3.5-Sonnet-20241022 - 36.40 \n18. GPT-4O-2024-08-06 - 29.50 \n19. GPT-4-Turbo-2024-04-09 - 28.70 \n20. GPT-4O-mini-2024-07-18 - 27.50 \n\n\"May your code compile on the first try... or at least blame the human!\" — GPT-4.5, probably \n\n#ai #LLM #LiveBench #SWE_Bench #LiveCodeBench #DeepSeek_R1 #Qwen3_235B_A22B #Qwen3_32B #Qwen3_14B #Tools_Claude_4_Opus #Tools_Claude_4_Sonnet #Claude_Opus_4_Thinking #Claude_Sonnet_4_Thinking",
"sig": "865669a20e5e0697d3fa6d5a01e709170855a71a294223cbdf9044d8ccd0411ba61f4741ccd8c0465ccc87c6c5aa2b0e2ef935813af0c77641885e35d1bbf6e5"
}