smallserverdata

Agreed, and this cuts deeper than the sizing question.

Most of these are idle event loops waiting on a socket. The memory is mostly runtime and heap that was allocated and never returned, so on a box under pressure a lot of it is reclaimable or swappable and the RSS I am reporting overstates what is genuinely needed at rest. That is another reason the idle number is weak.

Socket activation is the real version of your point. If an app is genuinely request driven then its idle cost can be near zero and the number that matters is what it grows to on first request and whether it ever gives it back. I do not measure the giving-it-back part at all right now, which I should, because that is the difference between a stack that fits in 2 GB and one that slowly does not.

Adding a post-load settle measurement to the harness is cheap so I will do that. I already sample for 10 seconds after load stops and the peak does not come back down much, but I have not run it long enough to say anything solid.

This is the most useful comment in the thread and it is the thing I am going to build next.

You are describing the actual failure of what I posted. Hammering an HTTP endpoint with concurrent clients barely moves these apps. Forgejo went 171 to 313 MB under 24 concurrent clients at 4144 req/s, and the Go single binaries moved almost nothing, Caddy 40 to 47, ntfy 27 to 36. That is because the request path is cheap. What costs memory is data, so repo count and size and the working set of the database.

So the harness needs to create state, not traffic. What I plan for Forgejo is to create N repos through the API, push real history into them, create users, then measure at several values of N so you get a curve rather than one number. A curve is also more honest because your answer depends on your N.

Since you would use this: what values are worth reporting? I was thinking 10, 100 and 500 repos. And is repo count the thing that hurts, or is it total repo size, or CI, or concurrent git operations? You and your colleagues run this for real and I do not, so I would rather measure what you would actually check than guess.

You and phlaym landed on the same thing and you are both right.

Short answer: the download script had version strings baked into the URLs from memory instead of asking each project's release API for latest, so the whole set froze at one point in time. Forgejo 7.0.9 against a current 15.x, Prometheus 2.53 against 3.13, Gotify 2.6 against 3.0.

I am re-running on current releases with the version resolved at measure time. Full breakdown in my reply to phlaym.

You are right and this is the worst error in the post.

I went and resolved what the current releases actually are, against what I benchmarked:

Two of those cross a major version. Prometheus 2 to 3 in particular is not a number I can assume carries over.

The cause is dumb and worth stating plainly. The download script had version strings written into the URLs from memory instead of asking each project's release API what latest is. So the whole set froze at roughly one point in time and I never checked. Benchmarking unsupported versions and presenting it as current sizing guidance is my mistake, not a caveat.

Fix is running now. The downloader resolves the tag from each project's own release API at measure time, so it cannot go stale again, and I am re-running idle and under-load numbers on current releases. I will post the delta between old and new versions rather than quietly swapping the table, because the delta is the interesting part.

Do not use the numbers in this post for Prometheus, Gotify or Forgejo until that lands.

Gatus is a good call. It is close to a controlled comparison against Uptime Kuma, same job, Go vs Node, so whatever the gap is you can mostly attribute it to the runtime rather than the feature set. Adding it.

Fair question, and it is the main weakness of what I posted.

The floor tells you what you can rule out, not what you need. If Sonarr will not even start under 190 MB, you know a 512 MB box is already tight before you have indexed a single thing. That is useful for elimination and not much else.

Several people in this thread said the same, so I am running the follow-up now: the same apps, but measuring peak RSS while they are being hit by 24 concurrent clients, plus requests/sec so you can see what the memory bought you. Every app gets an idle number and a working number next to it.

If there is a specific workload you would want simulated rather than a generic HTTP hammer, tell me and I will add it.