#AI #GenerativeAI #ChatGPT #Hallucinations: "Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain."
https://www.nature.com/articles/s41598-023-41032-5
Miguel Afonso Caetano /
npub1pw…r95z7
2023-09-11 09:51:01
Author Public Key
npub1pwuvltfvfme0d987k6rs3an6jnv9k2w32zqdlqt5hpy9u3cmv6ps2r95z7Published at
2023-09-11 09:51:01Event JSON
{
"id": "db755c647ebdd535a418fe7c88aaf3f1752a5fa7de26f4d865917fe426679a3a",
"pubkey": "0bb8cfad2c4ef2f694feb68708f67a94d85b29d15080df8174b8485e471b6683",
"created_at": 1694425861,
"kind": 1,
"tags": [
[
"t",
"ai"
],
[
"t",
"generativeAI"
],
[
"t",
"chatgpt"
],
[
"t",
"hallucinations"
],
[
"proxy",
"https://tldr.nettime.org/users/remixtures/statuses/111045893272778128",
"activitypub"
]
],
"content": "#AI #GenerativeAI #ChatGPT #Hallucinations: \"Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses (hallucinations) limit their utility. This study evaluates one particular type of hallucination: fabricated bibliographic citations that do not represent actual scholarly works. We used ChatGPT-3.5 and ChatGPT-4 to produce short literature reviews on 42 multidisciplinary topics, compiling data on the 636 bibliographic citations (references) found in the 84 papers. We then searched multiple databases and websites to determine the prevalence of fabricated citations, to identify errors in the citations to non-fabricated papers, and to evaluate adherence to APA citation format. Within this set of documents, 55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated. Likewise, 43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors. Although GPT-4 is a major improvement over GPT-3.5, problems remain.\"\n\nhttps://www.nature.com/articles/s41598-023-41032-5",
"sig": "5a0721acd1802d9de89a2fa5a2515046e88c2a70fff01cd98b795daa7293083be2ca59bee82e2ae78c2e0f1ffcb99dc72c5a7eebb3ec0c74f243bf9c544f6c34"
}