On Tue, Feb 27, 2007 at 08:03:29AM +1100, Daniel Carosone wrote:
> On Sat, Feb 24, 2007 at 10:10:10PM +0100, Manuel Bouyer wrote:
> > OK, I found some time to look at this again. After several unsucessfull
> > experiements I ended up looking at the xen sources, and found that the
> > minimum allowed "address_bits" is hardcoded to 31 (2GB). So with a stock
> > Xen kernel it's not possible to get bce working reliably under Xen :(
> > (This value seems to come from a restriction on aac devices, from what I
> > found in comments).
> 
> huh.  wow.  Thanks for the effort.  I'm confused why it works with
> dom0_mem=256m, then - especially as (the way I understand it) dom0
> starts at the top of physical ram, out of the way of all these
> allocations.

I have no explaination for this.

> 
> > I you want to run Xen on this system I should be able to build a xen kernel
> > which allows "address_bits" to be 30 instead of 31. 
> 
> That would be great;

ftp://asim.lip6.fr/outgoing/bouyer/xen.gz

> if it's not too hacky is this something worth
> feeding back to the xen folks?  iiuc, bce(4) is not the only device
> with the 1gb dma limit; i think there's at least some wireless device
> with the same limit.

As a compile-time option, maybe. But it's realy a hugly hack. With this, I
suspect devices with the 2Gb limit will stop working because then
address_bits == 31 would not be honored. The story is that, in fact,
Xen doesn't really match its interface. Here's what it does:
 if (address_bits <= 30)
	return error;
 if (address_bits == 31)
	return "DMA memory";
 else
	return "normal memory" or "DMA memory";
with "DMA memory" and "normal memory" being 2 memory pools, one before 
2GB and one after (of course for boxes with less than 2GB the second is
always empty and all is allocated from the first one). What I did is to change
31 to 30, and change the limit between DMA and normal memory from 2G to 1G.
Handling address_bits properly (that is, returning appropriate memory for
different values of address_bits) would require a complete redesign of
the Xen memory allocator.

> 
> It also sounds like there's an error reporting problem, in that
> requests below this cap are silently treated as if they were for 31,
> rather than failing or generating some kind of warning.  Perhaps Xen
> ought to fix at least that.

They're not failing silently; the hypervisor returns an error and
the domain0 panic (because it doesn't expect it to fail at this point).
I didn't understand why the hypervisor returned an error before running
several different experiments, and finally going to look at the Xen sources.

-- 
Manuel Bouyer <bouyer@antioche.eu.org>
     NetBSD: 26 ans d'experience feront toujours la difference
--