you should make a choice... Use KDE instead of Gnome, beacese new Qt is fast and optimized. Linus himself also says that KDE is better. If you want a something between complete DE and total minimalism, use XFCE or even Enlightenment DR17 if you want your box to be more eye-candy. Remember that DR stands for Development Release, so don't expect everything to work flawlessly. My choice is XFCE. And if you are total minimalism phreak, use {Open|Flux|Black}Box. I would recommend OpenBox because of better configurability.
All in all, do not use something what is fastest to compile, but something you would like to use for a longer time. Don't waste time trying different stuff. You can do that with other non-sourced-based distro or LiveCds. Don't consider that as me disliking Gentoo. I run Gentoo and only Gentoo:)... Ok, ok, a have windows on dual boot:D
My personal choice is XFCE. I compiled KDE 3.5 and I must say that it is good and fast but I removed it because of personal religious reasons. :) Phuckin K is iritating me:) I run gentoo 2.6.18 @ 1.5 GHz Celeron M. Thinkpad Z60m with integrated Intel graphics