Honestly though CPU processing is going to add almost no latency at all.

Almost all latency is sending to the RAM itself, so if you can compress it in a CPU cache before sending it, its nearly free RAM space.

Also systems are fucked fast these days you could probably make something responsive in pure python - other than millisecond timing scenarios like medical and audio stuff