kern/46828: 6.0_BETA2 and 6.0_RC1 won't start on DL320/G5p

[makoto%ki.nu@localhost]

>Number:         46828
>Category:       kern
>Synopsis:       6.0_BETA2 and 6.0_RC1 won't start on DL320/G5p
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 23 14:35:10 +0000 2012
>Originator:     Makoto Fujiwara
>Release:        NetBSD 6.0_RC1
>Organization:
KINU Corporation
        
>Environment:
        
        
System: NetBSD DL320 6.0_RC1 NetBSD 6.0_RC1 (GENERIC) #1: Thu Aug 23 18:08:19 
JST 2012 root@modena:/export/cvs-work/src/sys/arch/amd64/compile/obj/GENERIC 
amd64
Architecture: x86_64
Machine: amd64
>Description:
DL320/G5p is a HP made, rack mount 1U server. The machine
booted fine on NetBSD 5.1, while with 6.0 (BETA, BETA2 and
RC_1) it hung when ehci proved. The point of hung is at
pci/ehci_pci.c
----------------------
391  pci_conf_write(pc, tag, addr + PCI_EHCI_USBLEGSUP,
392   legsup);
----------------------

I have two work around if USB device is NOT necessary. 
Either of them boots fine.

1. Set USB Legacy support disable at BIOS setup.
    ----------------------
    rbsu> SET CONFIG USB CONTROL 3
    USB Control
    1|USB Enabled
    2|USB Disabled
    3|Legacy USB Disabled <=
    4|External USB Ports Disabled
    ----------------------
    With this, dmesg is at (say after)
      http://www.ki.nu/~makoto/dmesg/6.0_RC1/DL320_G5p-USB-Legacy-disabled
    while USB Enabled (as a default, say before),
      http://www.ki.nu/~makoto/dmesg/6.0_RC1/DL320_G5p-USB-Enabled

    Major difference before and after is:
    ------------
     ehci0: interrupting at ioapic0 pin 21
    +ehci0: EHCI version 1.0
    +....
    ------------
But this work around has (naturally) two big problems.
  -  Won't let you to use USB keyboard
  -  Can not boot from USB device

2. Disable ehci on kernel configuration

The second work around won't probe USB device at all.

Some strange point (to me), as far as ehci is enabled (as
GENERIC kernel), even when BIOS setup as
'USB Legace support disable',
USB keyboard and USB mass storage can be used. The part of
dmessage for such situation is:
--------
ehci0 at pci0 dev 29 function 7: vendor 0x8086 product 0x293a (rev. 0x02)
ehci0: interrupting at ioapic0 pin 21
ehci0: EHCI version 1.0
ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2 uhci3
...
uhub4 at usb4: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub4: 8 ports with 8 removable, self powered
...
ehci0: handing over low speed device on port 5 to uhci2
...
umass0 at uhub4 port 6 configuration 1 interface 0
umass0: vendor 0x05e3 USB TO IDE, rev 2.00/0.33, addr 2
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, 1 lun per target
sd0 at scsibus0 target 0 lun 0: <HTS54108, 0G9AT00, 0811> disk fixed
sd0: fabricating a geometry
sd0: 76319 MB, 76319 cyl, 64 head, 32 sec, 512 bytes/sect x 156301488 sectors
sd0: fabricating a geometry
--------

If we need boot from USB mass storage, we can NOT take above
work around. For such case, I have really-ad-hoc change as
follows.

Index: sys/dev/pci/ehci_pci.c
===================================================================
RCS file: /cvs/cvsroot/src/sys/dev/pci/ehci_pci.c,v
retrieving revision 1.55
diff -u -r1.55 ehci_pci.c
--- sys/dev/pci/ehci_pci.c      10 Jun 2012 06:15:53 -0000      1.55
+++ sys/dev/pci/ehci_pci.c      26 Jun 2012 10:07:17 -0000
@@ -390,6 +390,11 @@
        addr = EHCI_HCC_EECP(cparams);
        while (addr != 0) {
                cap = pci_conf_read(pc, tag, addr);
+               aprint_normal( "%s: %s:%d _ %08x %08x\n", 
__FILE__,__func__,__LINE__,
+                              EHCI_CAP_GET_ID(cap),
+                              EHCI_CAP_ID_LEGACY);
+               ms = EHCI_MAX_BIOS_WAIT;
+               goto skip;
                if (EHCI_CAP_GET_ID(cap) != EHCI_CAP_ID_LEGACY)
                        goto next;
                legsup = pci_conf_read(pc, tag, addr + PCI_EHCI_USBLEGSUP);
@@ -406,6 +411,7 @@
                                        break;
                                delay(10000);
                        }
+               skip:
                        if (ms == EHCI_MAX_BIOS_WAIT) {
                                aprint_normal("%s: BIOS refuses to give up "
                                    "ownership, using force\n", devname);

This will boot as follows.
----------------------
usb3 at uhci3: USB revision 1.0
ehci0 at pci0 dev 29 function 7: vendor 0x8086 product 0x293a (rev. 0x02)
ehci0: interrupting at ioapic0 pin 21
/export/src-a.j.n.o/src/sys/dev/pci/ehci_pci.c:
   ehci_get_ownership:393 _ 00000001 00000001
ehci0: BIOS refuses to give up ownership, using force
ehci0: EHCI version 1.0
ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2 uhci3
usb4 at ehci0: USB revision 2.0
ppb8 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0x92)
----------------------

I've placed the full dmesg for four combinations.
  http://www.ki.nu/~makoto/dmesg/6.0_RC1/

  4487 Aug 23 12:12 DL320_G5p-USB-Enabled
  9312 Aug 23 12:25 DL320_G5p-USB-Legacy-disabled
  9750 Aug 23 19:01 patched-DL320_G5p-USB-Enabled
  9555 Aug 23 18:55 patched-DL320_G5p-USB-Legacy-disabled
>How-To-Repeat:
        (1) Have HP DL320/G5p machine
        (2) Setup network boot
        (3) Use netbsd-INSTALL.gz kernel from either one of 
            6.0_BETA, 6.0_BETA2, 6.0_RC1
>Fix:
        Not known

>Unformatted: