ChatGPT's odds of getting code questions correct are worse than a coin flip.
"polite language, articulated and text-book style answers, comprehensiveness, and affiliation in answers make completely wrong answers seem correct"
...
the authors found ChatGPT is more likely to make conceptual errors than factual ones. "Many answers are incorrect due to ChatGPT’s incapability to understand the underlying context of the question being asked," the paper found.
https://www.theregister.com/2023/08/07/chatgpt_stack_overflow_ai/
Yogthos /
npub1w9…yme5e
2023-08-09 15:56:11
Author Public Key
npub1w9ek5pzf5yvvs2c5g2cwcm69xk7ermh2x8y9am9xsnk0krqd0rtsdyme5ePublished at
2023-08-09 15:56:11Event JSON
{
"id": "899077866350a4618a8fd2523132bc942f8373ee855d65f7ed4c0dd4ebe2b35b",
"pubkey": "71736a0449a118c82b1442b0ec6f4535bd91eeea31c85eeca684ecfb0c0d78d7",
"created_at": 1691596571,
"kind": 1,
"tags": [
[
"proxy",
"https://mas.to/users/yogthos/statuses/110860472898481220",
"activitypub"
]
],
"content": "ChatGPT's odds of getting code questions correct are worse than a coin flip.\n\n\"polite language, articulated and text-book style answers, comprehensiveness, and affiliation in answers make completely wrong answers seem correct\"\n...\nthe authors found ChatGPT is more likely to make conceptual errors than factual ones. \"Many answers are incorrect due to ChatGPT’s incapability to understand the underlying context of the question being asked,\" the paper found.\n\nhttps://www.theregister.com/2023/08/07/chatgpt_stack_overflow_ai/",
"sig": "9b43a6439ea9047466a1adf9b43badb9bdc27c5313c8f9928ac8736aa096b89141394b7e5cbbe7219991277531a94ecd7629a961330a0b72d4aaaaf195eada9a"
}