Hi,

How is NetBSD dom0 supposed to work on hosts with 4 GB or more memory?

I installed NetBSD/i386 4.0_STABLE on Dell PowerEdge 850 (it was previously
running NetBSD/i386 3.1.1 as dom0) and configured it as dom0. I'm having
problems with this dom0, domUs simply will not start as they die during the
early stages of startup.

# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   256     1     r-----     16.8

# cat /usr/pkg/etc/xen/xxx
kernel = "/usr/pkg/etc/xen/kernels/NetBSD-XEN3_DOMU.gz"
disk = [ 'file:/xen/netbsd-4GB.tmpl,0x1,w' ]
memory = 512
name = "xxx"
vif = [ 'mac=00:16:3e:11:11:11, bridge=bridge0' ]

# xm create xxx
Using config file "/usr/pkg/etc/xen/xxx".
Started domain xxx

# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   256     1     r-----     19.3

In the logs I found these:

[2008-01-24 14:19:23 106] WARNING (XendDomainInfo:1154) Domain has crashed: 
name=xxx id=7.
[2008-01-24 14:19:23 106] ERROR (XendDomainInfo:1284) VM xxx restarting too fast 
(1.098851 seconds since the last restart).  Refusing to restart to avoid loops.

Next I found the solution in our other Xen 3 server, i.e. I modified
/grub/menu.lst to limit the memory to 3072MB. I tried different values
here and the largest working value is somewhere between 3340M and 3350M.

--- menu.lst    2008-01-23 16:22:31.270154902 +0200
+++ menu.lst    2008-01-24 14:13:09.603381192 +0200
@@ -3,7 +3,7 @@

  title          NetBSD/xen
  root           (hd0,0,a)
-kernel         (hd0,0,a)/xen.gz dom0_mem=262144
+kernel         (hd0,0,a)/xen.gz dom0_mem=262144 mem=3072M
  module         (hd0,0,a)/netbsd root=/dev/hda1 ro console=tty0

After this change to /grub/menu.lst everything works as expected

# xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   256     1     r-----     13.1
xxx                                          1   512     1     -b----      1.2

# xm info
host                   : xen2.mydomain.com
release                : 4.0_STABLE
version                : NetBSD 4.0_STABLE (XEN3_DOM0) #0: Thu Jan 24 10:49:06 
EET 2008  root@xen2.mydomain.com:/usr/src/sys/arch/i386/compile/XEN3_DOM0
machine                : i386
nr_cpus                : 2
nr_nodes               : 1
sockets_per_node       : 1
cores_per_socket       : 1
threads_per_core       : 2
cpu_mhz                : 2800
hw_caps                : bfebfbff:00000000:00000000:00000180:0000441d
total_memory           : 3071
free_memory            : 2782
xen_major              : 3
xen_minor              : 1
xen_extra              : .2
xen_caps               : xen-3.0-x86_32
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xfc000000
xen_changeset          : unavailable
cc_compiler            : gcc version 4.1.2 20061021 prerelease (NetBSD nb3 20061125)
cc_compile_by          : root
cc_compile_domain      : research
cc_compile_date        : Wed Jan 23 14:24:34 EET 2008
xend_config_format     : 4