I think the biggest pull (for me at least) is a uniform shopping experience. I go directly to the manufacturer when possible and bypass Amazon but I do notice the rigmarole of how products are laid out differently on each site, some with or without a search feature, most without reviews (ratings are crap but sometimes reviewers can clear up ambiguity), each having their own checkout system (PayPal makes things a bit easier), and each has it's own return policy which is rarely as good as Amazon.

I think setting up a standard API for finding products, getting their specs, reading their policies at a high level, and ordering with a single login would go a long way towards taking down Amazon. I see the solution more as a federated one where vendors either host their own instance or pay some percentage of sales or flat rate to list products on someone else's instance.