Subject: Re: UP/SMP and atomic ops
To: Manuel Bouyer <firstname.lastname@example.org>
From: Andrew Doran <email@example.com>
Date: 11/28/2007 19:06:43
On Wed, Nov 28, 2007 at 07:43:10PM +0100, Manuel Bouyer wrote:
> I see that the new atomic ops that have been added to kernel, are
> patched to behave like non-atomic ops on UP kernels.
> Even for UP systems there are cases where we want atomic memory operations:
> - read/write to in-memory structres shared with devices. Some device
> controllers really behaves like a CPU.
The atomic ops defined in sys/atomic.h are specified to work only with RAM,
not device memory. Extending them to work reliably on device memory in a
portable way would be .. an interesting challenge.
> I know that reordered
> load/store can cause issues with esiop, and I suspect the occasional timeouts
> I've noticed can come from this. The lack of appropriate MI interface
> to fix this has always bothered be, and I think it's a real issue.
Is this on x86? bus_space_barrier (a no-op) should be enough on x86 if the
page is marked uncacheable, but I need to check that (i.e does PG_N imply
strongly ordered access). bus_dmamap_sync needs an lfence, which I can see
is missing. I will see about fixing that.
> - read-write to data shared with the hypervisor or another domain in Xen.
> A guest may be UP, although the hardware it's running on is SMP.
> Domain0 can eventually know the harware's number of CPUs but I'm
> not sure this information is available to guests (and with migration, this
> may change).
I feel this one is an x86/xen specific problem and should be handled there.
I have just realised that the atomic ops will be patched on xen/amd64, will
that cause problems for the pmap?
On a tangent, there is some provision for dynamic reconfiguration with Xen,
isn't there? That is: adding and removing virtual CPUs while the system is