That said, I do enjoy climbing on a lighter bike, particularly one with lighter wheels--the perceived effort is far less though I don't think it makes me much faster...physics always wins...
Whether or not the mass of the rest of the bike matters is a matter of debate. It certainly matters in MTB and cyclocross where you pick up your bike. Lennard Zinn wrote a piece about this, although it's not based rigorous experiments it does seem that the extra weight has some affect.
It makes sense that your body would be adapted to carrying your own weight and then any extra weight in the bike, backpack or whatever you're carrying requires additional effort, so the less you laden yourself with the less additional effort you're required to expend.