I spent ages today trying to figure out how to do a netinst of Xubuntu, the XFCE-based Ubuntu derivative. In the end I gave up and just did an Ubuntu netinst, using a rather convoluted method involving a Debian boot floppy and Ubuntu's netinst initrd. (the computer I was installing on can't do netboot nor read CD-Rs.)
It turns out that Xubuntu Desktop is an available option during the tasksel phase of the net installer (which is really just the Debian installer, it seems), or you can install a bare Ubuntu system with no desktop environment at all (don't select anything in tasksel) and then do
apt-get install xubuntu-desktop.