I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

you are viewing a single comment's thread
view the rest of the comments
[–] 221 points 4 days ago (29 children)

There are two proposed solutions I have seen. I do not claim that this is exhaustive, just only limited to the proposals that I have stumbled across.

FASP - Fediverse Auxiliary Service Provider

The idea that mastodon has is to have a third-party service that does something like generate a link preview once and then share that to fediverse instances that need it. They have a whole spec they wrote up about these kinds of services that can be run to help out fediverse instances with different things like this, search, or discoverability. They call these FASPs - the repo with the spec is here.

I don't yet know of any FASP that is live and running, so it is kind of a paper spec at the moment. I don't do too much on the microblogging side of the fediverse though, so I very well could be wrong.

Federate link metadata and the post at the same time

This comes from an issue on PieFed's repo. It is basically just an idea at this point, but it is a pretty basic one; have the originating instance fetch the link preview information and simply federate it out alongside the post itself.

  • source
  • hideshow 29 child comments
  • [–] 132 points 4 days ago (12 children)

    That second solution is what I thought of as doing as I read this. It doesn’t rely on a third party service being up. And that instance has to do that work anyway. So why not just share the results with others?

  • source
  • parent
  • hideshow 12 child comments
  • [–] 23 points 4 days ago (5 children)

    how would you know the results havent been tampered with?

  • source
  • parent
  • hideshow 5 child comments
  • [–] 119 points 4 days ago (2 children)

    The same way you know the rest of the data hasn’t been tampered with - you don’t. And when you find untrustworthy instances, you defederate from them.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 16 points 4 days ago (1 child)

    That's not true, you know the data hasn't been tampered with because of HTTP signatures. The fediverse isn't just "yolo, just trust me bro". There are some parts that are a bit more towards that side (particularly with groups on Lemmy) but I don't think this should be.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 10 points 3 days ago*

    HTTP signatures are not a default feature, the baseline is ECDHE key exchange with forward secrecy and deniability. Headers are not typically signed.

    There isn't even an official standard for redistributable signed page metadata. There's just prototypes. I absolutely hope one of those prototypes becomes official, but we're not there yet.

    The activitypub fediverse is literally very extremely Yolo trust me bro, it works because you can see the originating server with everything and misbehaving ones gets banned.

  • source
  • parent
  • [–] 10 points 4 days ago (5 children)

    It was my first thought too but there are downsides to it. Trust is a small one given the nature of the system but compute and bandwidth costs could be an issue also, especially for larger instances.

    The first solution is more fragile but could be CDN'd and those are cheap.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 5 points 3 days ago (3 children)

    WebTorrent kind of solution like PeerTube is the best, because it's highly scalable yet not that resource-intensive, because everyone (even users) provides their bit; also, as it's entirely P2P, it acts sort like a distributed and decentralized CDN, so it's snappier for everyone because it's more likely that you have a peer near to you like that.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 3 days ago (2 children)

    Absolutely LOVE this idea but I feel like a solid implementation would be a pain in the fanny (and it also requires other peers to be online). I dip in and out over the day myself but I don't know what others use case is.

    Also it is far less trustworthy and harder to block bad actors (V's removing a known bad actor from a relatively small peer group).

  • source
  • parent
  • hideshow 2 child comments
  • Idk, the part of uptime of peers is solved precisely by the fact that not every peer will be connected at the same moments that every other peer. If I'm connected at 10pm, it's highly likely that there will be at least another single peer at that hour that has what I'm pulling.

    As for bad actors, that's something that at the end of the day is unavoidable. Though with some automated systems could be possible to make it more hard for them, but it's not that different from a centralised approach, and thus is better because is computationally more efficient.

  • source
  • parent
  • [–] 6 points 4 days ago

    I don't think compute resources should be an issue, if anything it saves other instances resources.

    As far as I understand, right now, all instances are fetching this data individually, so they all have the same compute load from this. Doing it on one and sending it along with federation should keep resource useage pretty much the same on the originating instance, and less for all others.

    This would still increase bandwidth of course, but at least with all VPS providers I have used, bandwith is pretty cheap.

  • source
  • parent
  • [–] 72 points 4 days ago* (6 children)

    I'm a fan of the second one. My thought is, if you trust a server enough to accept messages from it, then you should also trust it enough to let it scrape the page title, article image, etc. for you, too.

  • source
  • parent
  • hideshow 6 child comments
  • [+] 1 point 3 days ago* (last edited 2 days ago) (4 children)
  • [–] 5 points 3 days ago* (1 child)

    This is no different from how it is already, though. If a user posts a CSAM link, your server immediately visits that link to scrape it, and you suddenly have that content on your server.

    Of course, if you have a blocklist of forbidden sites, then your server will know not to scrape that site. And the same thing applies if we allow servers to send pre-scraped metadata. If the incoming message is for a blocked URL, just drop the message and don't save any part of it.

    The only new threat here is for a malicious server operator to modify outgoing messages to include CSAM metadata on a harmless link. However, that possibility already exists with any other content posted on the fediverse. If a malicious server is modifying its own outgoing messages, the only fix is to defederate from it.

    Allowing servers to forward pre-scraped metadata to other servers does not introduce any new threats that do not already exist.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 3 days ago (1 child)

    There's no reason to believe that this mechanism doesn't check for CSAM (I am also pretty stumped that this is actually a problem with Mastodon according to your comment?) . Lemmy had a phase where messed up stuff got federated, which led to @db0@lemmy.dbzer0.com's implementation of an anti-CSAM tool: https://github.com/db0/fedi-safety, which is based on the CSAM detection tool for the AI Horde: https://dbzer0.com/blog/ai-powered-anti-csam-filter-for-stable-diffusion/ - it's probably not so hard to adapt for Mastodon too.

  • source
  • parent
  • hideshow 1 child comment
  • [+] 5 points 3 days ago (2 children)
  • [–] 1 point 3 days ago (1 child)

    I've just had a look at the code to refresh my memory - it does use it :)

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 3 days ago

    Sorry, I think I'm just blind. I tried a non piefed.social post earlier today and didn't see it in the JSON, but checking a piefed.social post now, the property is there. So either I just didn't see it the first time around, or the instance I tried it with wasn't the best example to use.

    Anyway, deleted my comment. But then OP's post confuses me again, because there really shouldn't be a DDoS happening toward the small site. The instance hosting the thumbnail sure, but not the site that's being linked to.

  • source
  • parent
  • [–] 3 points 4 days ago

    Really the only metadata we are interested in is the thumbnail image url. Afaik PieFed already retrieves this from the og:image meta tag before sending the URL along with the other post content to the instance hosting the community.

    Another idea - those meta tags are also retrieved during post composition in an ajax call, in order to prepopulate the title and body fields so the image url could be stashed in a hidden field at that point...

  • source
  • parent
  • [–] 1 point 3 days ago (3 children)

    But what happens when a link target changes? For example, I've seen news articles change as the author gets new information. Might be an edge case, but a solution like this would always show the preview at the time of the initial post.

  • source
  • parent
  • hideshow 3 child comments