Humans have used other things to act as reminders of things and, more importantly, as recorders for numbers, so I really doubt "decline in brain volume" happened because of writing. Not only that, literacy was low throughout most of human history and usually confined to small groups, which rarely included the nobles, by the way - they would just have a servant that could read/write for them.
It's deliciously ironic that we only know about those ancient greek snobs who despised writing thanks to their words being written.