You're making sketchy assumptions.

You're assuming that arbitration priorities are uniform.  In many cases this 
is not true; in fact, there are devices (notably some Intel southbridge IDE 
controllers) which assert a hold on the bus and override anything else using 
it.  By using a smaller latency timer, and thus causing "goodput" to go down 
considerably, you can inadvertently hose everyone else.

I'm not sure you're hitting a case like that, but it's at least one scenario 
where reducing the latency timer will actually make performance of other 
devices worse.