655
PSA Huggingface is scraping GitHub (europe.pub)
submitted 2 months ago by to c/programmer_humor@programming.dev
126 comments fedilink hide all child comments
 

https://huggingface.co/spaces/HuggingFaceCode/in-the-stack

you are viewing a single comment's thread
view the rest of the comments
[–] 27 points 2 months ago* (6 children)

Well… I’d rather the dataset be public and there, with an ostensible centralized opt-out, instead of every AI startup frantically rescraping the same things their predecessors did.

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] [S] 9 points 2 months ago

    True, I understand why this is better than all those companies scraping it individually (for both ability to opt out and site load), but the way they handled opt out is still quite silly.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 months ago (2 children)

    Second that. After all all my open repositories are all either licenced under GPL or MIT, so complaining would be a kind of hypocritical.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 12 points 2 months ago* (last edited 2 months ago) (1 child)

    GPL is dodgy to be included in training data for AIs that are then used to generate non-openaource code tbh. Even MIT loses the attribution it's supposed to have once laundered through AI...

    Speaking as someone that's been into FOSS for a long time, it does piss me off how quickly copyright got thrown under the bus the moment it became inconvenient for people with money.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 6 points 2 months ago*

    Yes, but what HuggingFace is doing here is the distribution of a data set. And so long the data set itself is open that doesn't conflict with GPL. HuggingFace is not responsible about how others use that data set.

    Whether LLMs themselves violate copyright for being trained on MIT or GPL code is an entirely different discussion.

  • permalink
  • fedilink
  • source
  • parent