kern/56848: bge(4) chips with ASF/IPMI firmware don't work when NetBSD boots

[Brian Buhrow <buhrow%nfbcal.org@localhost>]

>Number:         56848
>Category:       kern
>Synopsis:       bge(4) chips with ASF/IPMI firmware don't work when NetBSD boots
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu May 19 15:40:01 +0000 2022
>Originator:     Brian Buhrow
>Release:        NetBSD 9.99.77
>Organization:
nfbcal.org
	
>Environment:
	
	
NetBSD ns2.nfbcal.org 9.99.77 NetBSD 9.99.77 (MIRKWOOD) #0: Wed May 18 15:15:56 PDT 2022
buhrow%loth-9.nfbcal.org@localhost:/usr/local/netbsd/src-current/sys/arch/amd64/compile/MIRKWOOD amd64
Architecture: x86_64
Machine: amd64
>Description:
	
When NetBSD configures a bge(4) device at boot time and that device contains an ASF/IPMI
firmware module, meaning the system network management port shares the same physical port with
the host system, the ASF/IPMI firmware loses connectivity with the network until NetBSD brings
up the bge(4) interface with ifconfig.  As a consequence, if the system console is running
through that ASF/IPMI interface, it is impossible to log in through the console if the system
comes up in single user mode or if the system doesn't configure that specific network interface
in its boot sequence.  Worse, if the system is also managed through that ASF/IPMI interface for
tasks such as power cycling the system, checking the BIOS or just monitoring the health of the
system, that management ability is lost.

Note that this behavior is a regression from NetBSD-5.2, which configured these ports properly,
meaning systems with this type of hardware could be fully managed remotely without a problem.

>How-To-Repeat:
	
Boot NetBSD on one of these systems and prepare to lose control of its network management interface.
>Fix:
	
Below is the log for the commit  that fixes this problem in -current.
Below that is a patch against the NetBSD-9 source which applies cleanly to the NetBSD-9 branch.



                                    CVS commit: src/sys/dev/pci
     ______________________________________________________________________________________

     * To: source-changes%NetBSD.org@localhost
     * Subject: CVS commit: src/sys/dev/pci
     * From: "Brian Buhrow" <buhrow%netbsd.org@localhost>
     * Date: Thu, 19 May 2022 04:43:43 +0000
     ______________________________________________________________________________________

Module Name:    src
Committed By:   buhrow
Date:           Thu May 19 04:43:43 UTC 2022

Modified Files:
        src/sys/dev/pci: if_bge.c

Log Message:
For chips which contain an ASF/IPMI firmware, instruct the chip to shut the host ASF firmware
down when attaching the device so the IPMI BMC can use the same physical port even when NetBSD
doesn't have a network configuration on the device.  By contrast, when the device gets a
network configuration assigned to it and bge_init() is called, the host ASF firmware is brought
up so both NetBSD and the IPMI BMc can use the same
physical port.

This now matches FreeBSD behavior, as well as  behavior from NetBSD-5.2.

Tested on a Sunfire X2200-M2 system with the following chip:

bge1 at pci7 dev 4 function 1: Broadcom BCM5715 Gigabit Ethernet
bge1: interrupting at ioapic0 pin 11
bge1: HW config 000000d4, 00000014, 00000000, 00000000 00000000
bge1: ASIC BCM5715 A3 (0x9003), Ethernet address 00:1e:68:XX:XX:XX
bge1: setting short Tx thresholds
brgphy1 at bge1 phy 1: BCM5714 1000BASE-T/X media interface, rev. 0
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto


To generate a diff of this commit:
cvs rdiff -u -r1.352 -r1.353 src/sys/dev/pci/if_bge.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.




Index: if_bge.c
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/if_bge.c,v
retrieving revision 1.335.2.1
diff -u -r1.335.2.1 if_bge.c
--- if_bge.c	27 Nov 2019 11:08:24 -0000	1.335.2.1
+++ if_bge.c	19 May 2022 15:26:17 -0000
@@ -1,4 +1,4 @@
-/*	$NetBSD$	*/
+/*	$NetBSD: if_bge.c,v 1.335.2.1 2019/11/27 11:08:24 martin Exp $	*/
 
 /*
  * Copyright (c) 2001 Wind River Systems
@@ -79,7 +79,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD$");
+__KERNEL_RCSID(0, "$NetBSD: if_bge.c,v 1.335.2.1 2019/11/27 11:08:24 martin Exp $");
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -3646,7 +3646,7 @@
 	BGE_SETBIT_FLUSH(sc, BGE_MISC_LOCAL_CTL, BGE_MLC_AUTO_EEPROM);
 
 	bge_stop_fw(sc);
-	bge_sig_pre_reset(sc, BGE_RESET_START);
+	bge_sig_pre_reset(sc, BGE_RESET_SHUTDOWN);
 	if (bge_reset(sc))
 		aprint_error_dev(sc->bge_dev, "chip reset failed\n");
 
@@ -3680,8 +3680,8 @@
 	    "HW config %08x, %08x, %08x, %08x %08x\n",
 	    hwcfg, hwcfg2, hwcfg3, hwcfg4, hwcfg5);
 
-	bge_sig_legacy(sc, BGE_RESET_START);
-	bge_sig_post_reset(sc, BGE_RESET_START);
+	bge_sig_legacy(sc, BGE_RESET_SHUTDOWN);
+	bge_sig_post_reset(sc, BGE_RESET_SHUTDOWN);
 
 	if (bge_chipinit(sc)) {
 		aprint_error_dev(sc->bge_dev, "chip initialization failed\n");

>Unformatted: