“…indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs)”
npub1gw4guamk89rqavrn0qf7nr7qsuj234dupqd00m782zymyhk6nwyqe4pu63 (npub1gw4…pu63)
#ai
https://www.nature.com/articles/s41586-024-07566-y
Gert :debian: :gnu: :linux: /
npub1c2…dvehu
2024-09-07 11:02:24
Author Public Key
npub1c2c6lgjxax5pnqp4t8fempdgupzdhvvvchnyae2xnyrf5vpa07zqjdvehuPublished at
2024-09-07 11:02:24Event JSON
{
"id": "61f0d2ba9a01d7d32f5c20e2278d1bca2f8c5a61e303a4f3f67a36443c4c67ca",
"pubkey": "c2b1afa246e9a819803559d39d85a8e044dbb18cc5e64ee54699069a303d7f84",
"created_at": 1725706944,
"kind": 1,
"tags": [
[
"p",
"43aa8e777639460eb0737813e98fc08724a8d5bc081af7efc75089b25eda9b88",
"wss://relay.mostr.pub"
],
[
"t",
"ai"
],
[
"proxy",
"https://qoto.org/users/Gert/statuses/113095930296606284",
"activitypub"
]
],
"content": "“…indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as ‘model collapse’ and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs)”\nnostr:npub1gw4guamk89rqavrn0qf7nr7qsuj234dupqd00m782zymyhk6nwyqe4pu63 \n#ai \nhttps://www.nature.com/articles/s41586-024-07566-y",
"sig": "6d28a7eff2d8198ba4e4aec9274b64a4ca9a382038fb5c7bbd9eb225ed8158fe5c4e059816db5bc9fa5551c34f129f93358af905dfa254e6a25a4eaeb423152d"
}