having the data inline

It's not as simple as that, depending on the architecture. Typically they would fetch 64-byte cache lines so your 128 bytes aren't going to be magically more cached than 128 bytes on the heap.

Avoiding allocations may help but also maybe not. This is definitely in "I don't believe it until I see benchmarks" realm. I would be really really surprised if the allocation cost was remotely bad enough to justify the "sorry your file is too long" errors.