It could be decentralised and would actually be more secure if it was. I don't think Pete's wrong about this. Freenet did it (it failed, because it required a critical mass of users before it was usable (and never got them), but the tech was sound, I think a lot of it got rolled into what is now TOR).
Stuff is moving that way anyway. I mean all this 'cloud' stuff is virtualised on-demand servers with resources allocated on the fly. That's a step towards complete decentralisation.
There's a legal angle too. In the US people are starting to make home cloud servers - i.e. something very much like a NAS but with webmail, photo sharing etc. etc. - all the social stuff built in to the server. The reason for this is that if the authorities want access to all your online stuff they can get it pretty easily. If they want something from your home it's a lot tougher - they need more warrants and they need more justification.
As that kinda thing starts to become more common (and I think it will, eventually) decentralising gets a lot easier.
And as Pete says, all you'd need then is a chain of central hubs/database servers which glue all this together. You wouldn't even strictly /need/ that. It'd just be a good way to start it off.
I'm not sure Dan's trying to argue against that, more that he's got a fairly good idea of how much hardware costs and that Pete's rough budget isn't anywhere near enough. I'm inclined to agree with him.
And Pete's point is: What hardware? This idea could potentially be run on no-centralised-hardware which, last time I checked, cost no pounds. Anything you add is just to grease the wheels a little.
To put it another way: Dan's talking about creating a very robust 'enterprise level' 'server solution'. Which is nothing like what Pete's talking about, which is a decentralised, distributed (possibly p2p but that's pretty much beside the point) ad-hoc network.
Or to put it yet another way: Dan's suggesting a hardware solution to Pete's software problem.
You're assuming that everyone will want to set up a NAS or something similar in their home. But there are plenty of people (I'm thinking of the people in my office who panic when a new version of Internet Explorer gets installed) who don't/won't want that. Christ, I wouldn't want that (I would really, but just to make a point) - I pay enough for electricity as it is.
Even if you (as a company running this social network) does some coding magic to integrate things like Flickr, Picasa, ect and makes it look seamless, you're still going to have to pay those sites for access to their hardware. Especially if you're going to bring millions of new customers to them.
Basically, especially when you scale up to millions (tens of millions) of users, someone's going to have to pay for hardware somewhere along the line.