NPUs are already a thing, specialized cpus for nerual networks. The problem is the models are just databases, and you need lots of fast memory in order to feed the NPUs data, and thats the current bottleneck.