this post was submitted on 17 May 2025
303 points (94.4% liked)

Technology

70249 readers
3848 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
303
The Collapse of GPT: Will future artificial intelligence systems perform increasingly poorly due to AI-generated material in their training data? (cacm.acm.org)
submitted 6 days ago by Pro@programming.dev to c/technology@lemmy.world
58 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] Knock_Knock_Lemmy_In@lemmy.world 8 points 4 days ago (1 children)

https://en.wikipedia.org/wiki/Model_collapse

[–] toastmeister@lemmy.ca 2 points 4 days ago

To make this more precise, we say that original data follows a normal distribution {\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})}, and we possess {\displaystyle M\_{0}} samples {\displaystyle X\_{j}^{0}} for {\displaystyle j\in {\\{\\,1,\dots ,M\_{0}\\,{}\\}}}. Denoting a general sample {\displaystyle X\_{j}^{i}} as sample {\displaystyle j\in {\\{\\,1,\dots ,M\_{i}\\,{}\\}}} at generation {\displaystyle i}, then the next generation model is estimated using the sample mean and variance:

{\displaystyle \mu \_{i+1}={\frac {1}{M\_{i}}}\sum \_{j}X\_{j}^{i};\quad \sigma \_{i+1}^{2}={\frac {1}{M\_{i}-1}}\sum \_{j}(X\_{j}^{i}-\mu \_{i+1})^{2}.}

Leading to a conditionally normal next generation model {\displaystyle X\_{j}^{i+1}|\mu \_{i+1},\\;\sigma \_{i+1}\sim {\mathcal {N}}(\mu \_{i+1},\sigma \_{i+1}^{2})}. In theory, this is enough to calculate the full distribution of {\displaystyle X\_{j}^{i}}. However, even after the first generation, the full distribution is no longer normal: It follows a variance-gamma distribution.