Subject: Re: bce(4) and memory > 1GB problem
To: Manuel Bouyer <bouyer@antioche.eu.org>
From: Daniel Carosone <dan@geek.com.au>
List: tech-kern
Date: 02/28/2007 10:04:39
--DrWhICOqskFTAXiy
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Feb 27, 2007 at 11:54:13AM +0100, Manuel Bouyer wrote:
> > I'm confused why it works with dom0_mem=3D256m, then - especially as
> > (the way I understand it) dom0 starts at the top of physical ram,
> > out of the way of all these allocations.
>=20
> I have no explaination for this.

Maybe dom0 does actually get placed differently in this case, due to
some side-effect of the allocator.

> ftp://asim.lip6.fr/outgoing/bouyer/xen.gz

Excellent, thanks.  With this vmm, I can now boot dom0 with other
memory sizes, and it works as before (same dom0 kernel as I've been
using all along).

I noticed in dmesg I get a lot more _xen_alloc_contig messages, eg:

wpi0: interrupting at ioapic0 pin 17, event channel 5
_xen_alloc_contig low 0 high fffff000 address_bits 32
_xen_alloc_contig low 0 high fffff000 address_bits 32
_xen_alloc_contig low 0 high fffff000 address_bits 32
(about a dozen more)

=2E..
bce0 at pci5 dev 0 function 0: Broadcom BCM4401-B0 10/100 Ethernet
bce0: interrupting at ioapic0 pin 17, event channel 5
_xen_alloc_contig low 0 high 40000000 address_bits 30
_xen_alloc_contig low 0 high 40000000 address_bits 30
(about 160 more!)
=2E..

I also still get the occasional:
mmapbatch: remap error 14!

perhaps triggered by network traffic and/or bridging to domU's.

> As a compile-time option, maybe. But it's realy a hugly hack. With this, I
> suspect devices with the 2Gb limit will stop working because then
> address_bits =3D=3D 31 would not be honored. The story is that, in fact,
> Xen doesn't really match its interface. Here's what it does:
>  if (address_bits <=3D 30)
> 	return error;
>  if (address_bits =3D=3D 31)
> 	return "DMA memory";
>  else
> 	return "normal memory" or "DMA memory";
> with "DMA memory" and "normal memory" being 2 memory pools, one before=20
> 2GB and one after (of course for boxes with less than 2GB the second is
> always empty and all is allocated from the first one). What I did is to c=
hange
> 31 to 30, and change the limit between DMA and normal memory from 2G to 1=
G.

Hmm.  Well, it seems possible to do something like:

 if (address_bits < 30)
	return error;
 if (address_bits < 32)
	return "DMA memory";
 else
	return "normal memory" or "DMA memory";

ie, will the aac devices with a 2Gb limit break if they happen to get
all their allocations with a 31-bit limit satisfied from memory that
also happens to fit within a 30-bit limit? =20

Still a huge hack, but perhaps one that will work around both sets of
losing hardware, if you happen to have them in the same machine.

--
Dan.
--DrWhICOqskFTAXiy
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (NetBSD)

iD8DBQFF5LkHEAVxvV4N66cRAlfjAJ9SLmkTI9G+i4Rxz5N96xPyxigPEQCg8D/k
pCsTMxnSFXD/9FQSefYbFTk=
=I5zb
-----END PGP SIGNATURE-----

--DrWhICOqskFTAXiy--