you are viewing a single comment's thread
view the rest of the comments
[–] 117 points 1 day ago (5 children)

Copyright violation from American AI hoovering up all the data is okie dokie unless it's used by peasants to threaten technology corporations.

I'm not defending them it's just comical hypocrisy, and ans I doubt China AI will have the same kneecapped coding capabilities.

  • source
  • hideshow 5 child comments
  • [–] 9 points 11 hours ago (3 children)

    Chinese models are kneecapping Tiananmen Square and Winnie the Pooh, they'll happily do anything else.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 11 hours ago (2 children)

    But at least they are open (Most of them at least).

    So there's at least the possibility to lever these restrictions away.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 7 points 10 hours ago (1 child)

    While I agree that open-weight is better than nothing, it is not the same as open-source.
    Having a truly open-source model would be wonderful and it would also allow us to verify what data it is trained on and whether it has the right to use that data.

    The Open Source Initiative's definition of a fully open-source AI model includes the training data as a key element. Some developers wonder, however, how feasible it is to distribute the enormous datasets required.

    He says Stanford HAI has staked out a specific bar: alignment with the Linux Foundation’s Model Openness Framework at its top tier, the “Open Science” class – meaning the code, the training data (or a thoroughly documented, auditable account of it), and the tooling are all released, not just the weights, alongside a real way for outside researchers and communities to download, run, study, contribute to, and modify the work. Anything short of that, in his view, forecloses entire lines of research: You can’t, for example, study how a model’s training data shapes its cultural assumptions, audit it for how fairly it represents different populations, or test new architectures at the scale where genuinely new behavior shows up. You can only observe a finished system from the outside and guess at why it does what it does.

  • source
  • parent
  • hideshow 1 child comment
  • Right I agree, but this is basically the entire internet (and more, like scanned books), what it is trained on. They are using reinforcement learning now. And this is also one of the reasons why they have escaped the sandboxes in the American models for example.

    So you can probably safely assume that they are trained on all data that is available easily. and very likely also artificially generated data at this point. How long they are trained on, what kind of data is a different story. I think it gets increasingly more interesting how these models are trained.

    So while I agree it would be really nice to know exactly what it is trained on and how it is trained on that, I think that those that have the resources to get at least close to frontier models (i.e. AI companies) don't just open up on that, sadly.

  • source
  • parent