Re: nfe problem on dom0 kernel 5.99.24 [need help]

[Adam Hamsik <haaaad%gmail.com@localhost>]

Hi Manual,
On Feb,Friday 26 2010, at 12:14 PM, Manuel Bouyer wrote:

> On Fri, Feb 26, 2010 at 04:44:21AM -0600, Sam Fourman Jr. wrote:
>>> If possible, boot with a xen-debug kernel, and post the xm dmesg. Failing to
>>> create DMA memory is likely to be due to an error returned by hypervisor.
>>> 
>>> --
>>> Jean-Yves Migeon
>>> jeanyves.migeon%free.fr@localhost
>>> 
>> here it is
>> 
>> http://www.puffybsd.com/xm-dmesg.txt
>> 
>> These lines look weird
>> 
>> (XEN) traps.c:2180:d0 Domain attempted WRMSR 00000000000001a0 from
>> 00000040:60970481 to 00000040:60931c09.
>> (XEN) io_apic.c:2192:
>> (XEN) ioapic_guest_write: apic=0, pin=2, old_irq=0, new_irq=-1
>> (XEN) ioapic_guest_write: old_entry=000009f0, new_entry=00010900
>> (XEN) ioapic_guest_write: Attempt to remove IO-APIC pin of in-use IRQ!
> 
> No, that's not a problem. This is because the dom0 kernel tries to
> reconfigure all interrupt lines, but some of them are already used
> by the hypervisor. 
> 
> I guess the next thing to trie would be to add some printfs in the
> bus_dma code, to see where it's failing exactly ...

I'm helping Sam with this issue. This is part of dmesg with dome debug printfs

nfe0 at pci0 dev 17 function 0: vendor 0x10de product 0x0373 (rev. 0xa2)
APCH: Picked IRQ 22 with weight 1
linkdev APCH returned ACPI global irq 22, line 22
nfe0: interrupting at ioapic0 pin 22, event channel 7
nfe0: Ethernet address 00:1a:92:c0:d4:b2
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
_xen_alloc_contig 0
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
_xen_alloc_contig 0
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
xen_alloc_contig: XENMEM_decrease_reservation failed!
_xen_alloc_contig 12
nfe0: could not create DMA map
nfe0: could not allocate Tx ring
nfe1 at pci0 dev 18 function 0: vendor 0x10de product 0x0373 (rev. 0xa2)
AMC1: Picked IRQ 23 with weight 1
linkdev AMC1 returned ACPI global irq 23, line 23
nfe1: interrupting at ioapic0 pin 23, event channel 8
nfe1: Ethernet address 00:1a:92:c0:df:70
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
_xen_alloc_contig 0
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
_xen_alloc_contig 0
_xen_bus_dmamem_alloc_range boundary check
_bus_dmamem_alloc_range == uvm_pglistalloc
_bus_dmamem_alloc_range == uvm_pglistalloc -> 0
_xen_alloc_contig
_xen_alloc_contig -> uvm_pglistalloc
_xen_alloc_contig -> uvm_pglistalloc == 0
xen_alloc_contig: XENMEM_decrease_reservation failed!
_xen_alloc_contig 12
nfe1: could not create DMA map
nfe1: could not allocate Tx ring

This output narrows problem to this code 

for (pg = mlistp->tqh_first; pg != NULL; pg = pg->pageq.queue.tqe_next) {
                pa = VM_PAGE_TO_PHYS(pg);
                mfn = xpmap_ptom(pa) >> PAGE_SHIFT;
                xpmap_phys_to_machine_mapping[
                    (pa - XPMAP_OFFSET) >> PAGE_SHIFT] = INVALID_P2M_ENTRY;
                xenguest_handle(res.extent_start) = &mfn;
                res.nr_extents = 1;
                res.extent_order = 0;
                res.domid = DOMID_SELF;
                if (HYPERVISOR_memory_op(XENMEM_decrease_reservation, &res) 
<-====- Problematic xen call
                    != 1) {
                        printf("xen_alloc_contig: XENMEM_decrease_reservation "
                            "failed!\n");
                        xpmap_phys_to_machine_mapping[
                            (pa - XPMAP_OFFSET) >> PAGE_SHIFT] = mfn;

                        error = ENOMEM;
                        goto failed;
                }
        }

Do you have any ideas how can it be debug more ?

Regards

Adam.