Hi,
while looking at how bus_space_read/write can be reordered on various ports
I've dicovered the following:
1- not all drivers use bus_space_barrier() to ensure ordering of register
  access
2- because of 1, all bus_space implementation (on hardware where it's needed)
  have implicit barrier in the read and write methods
3- the implementation isn't in sync with the man page. The man page defines
  5 different barrier: BUS_SPACE_BARRIER_{READ,WRITE}_BEFORE_{READ,WRITE}
  and BUS_SPACE_BARRIER_SYNC, while the implementation has only
  BUS_SPACE_BARRIER_READ and BUS_SPACE_BARRIER_WRITE. The man page match
  more closely what's some hardware (e.g. sparc64) can do.
4- the man page mention a BUS_SPACE_MAP_PREFETCHABLE flag to bus_space_map(),
  which should "try to map the space so that accesses can be prefetched by the
  system, and writes can be buffered". This implies that if
  BUS_SPACE_MAP_PREFETCHABLE is not used, read and writes should stay
  ordered and bus_space_barrier() calls should not be needed.
5- for most hardware, a barrier is needed for *each* bus_space access.
  The only case where this may not be needed is hardware providing some
  ROM or buffer space directly mappable.

Because of this, I propose the following:
- change the implementation to be in sync with the man page (that is,
  remove BUS_SPACE_BARRIER_{READ,WRITE} and implement the 5 barriers
  defined in the man page)
- change the implementation so that if the space is mapped without
  BUS_SPACE_MAP_PREFETCHABLE, bus_space_barrier() calls are not needed.
  This may require using bus_space_barrier() in the read/write methods
  to enforce this, and in fact I'll probably be conservative and do it this
  way on most ports, leaving the portmaster change this to better suit
  the hardware.
- update drivers to match the new implementation (probably removing
  bus_space_barrier() calls for most of them, or converting them to
  BUS_SPACE_MAP_PREFETCHABLE)

Any comments ? Did I miss something ?

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--