On Wed, Nov 28, 2007 at 07:43:10PM +0100, Manuel Bouyer wrote:

> I see that the new atomic ops that have been added to kernel, are
> patched to behave like non-atomic ops on UP kernels. 
> 
> Even for UP systems there are cases where we want atomic memory operations:
> - read/write to in-memory structres shared with devices. Some device
>   controllers really behaves like a CPU.

The atomic ops defined in sys/atomic.h are specified to work only with RAM,
not device memory. Extending them to work reliably on device memory in a
portable way would be .. an interesting challenge.

> I know that reordered
>   load/store can cause issues with esiop, and I suspect the occasional timeouts
>   I've noticed can come from this. The lack of appropriate MI interface 
>   to fix this has always bothered be, and I think it's a real issue.

Is this on x86? bus_space_barrier (a no-op) should be enough on x86 if the
page is marked uncacheable, but I need to check that (i.e does PG_N imply
strongly ordered access). bus_dmamap_sync needs an lfence, which I can see
is missing. I will see about fixing that.

> - read-write to data shared with the hypervisor or another domain in Xen.
>   A guest may be UP, although the hardware it's running on is SMP.
>   Domain0 can eventually know the harware's number of CPUs but I'm
>   not sure this information is available to guests (and with migration, this
>   may change).

I feel this one is an x86/xen specific problem and should be handled there.
I have just realised that the atomic ops will be patched on xen/amd64, will
that cause problems for the pmap?

On a tangent, there is some provision for dynamic reconfiguration with Xen,
isn't there? That is: adding and removing virtual CPUs while the system is
running.

Thanks,
Andrew