> It would make sense with i386, but not with others.

Right.
Except you are going to emulate...

> Even on i386, VESA BIOS provides limited number of video
> modes, so display drivers have to know how to do mode switching. 

For mode switching, in general extended registers are needed.
So there are 2 possibilities:
a) you know the card, then the driver can do it
b) you don't, then the BIOS has to do it

Mixing a and b is dangerous at least.

btw, on my (not really fancy) grahics card, the VESA BIOS
knows about 44 modes.

best regards
Matthias