Joerg@ just committed meta-pkgs/modular-xorg-apps.
Thus, what you'd do is build the server, then
modular-xorg-{drivers,fonts,apps}.  Of course, you don't _have_ to
build all of those, depending upon your needs, but you asked for
meta-pkgs. ;)

Regards,

--Blair