Could it be the case that the X server expects some aspects of the video hardware to be initialized by the video console driver that are uninitialized in the serial console case? E.g., as you say outputs are shared between HDMI and VGA, the X video simply goes to the HDMI output?