Joerg@ just committed meta-pkgs/modular-xorg-apps.
Thus, what you'd do is build the server, then
modular-xorg-{drivers,fonts,apps}. Of course, you don't _have_ to
build all of those, depending upon your needs, but you asked for
meta-pkgs. ;)
Regards,
--Blair