NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: port-amd64/60641: KX-7000 - page fault trap on boot when SMAP feature is enabled



The following reply was made to PR port-amd64/60641; it has been noted by GNATS.

From: Andrius V <vezhlys%gmail.com@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: port-amd64/60641: KX-7000 - page fault trap on boot when SMAP
 feature is enabled
Date: Thu, 17 Sep 2026 00:41:41 +0300

 On Wed, Sep 16, 2026 at 11:26=E2=80=AFAM Andrius V <vezhlys%gmail.com@localhost> wrot=
 e:
 >
 > On Tue, Aug 25, 2026 at 2:36=E2=80=AFPM vezhlys%gmail.com@localhost via gnats
 > <gnats-admin%netbsd.org@localhost> wrote:
 > >
 > > >Number:         60641
 > > >Category:       port-amd64
 > > >Synopsis:       KX-7000 - page fault trap on boot when SMAP feature is=
  enabled
 > > >Confidential:   no
 > > >Severity:       serious
 > > >Priority:       medium
 > > >Responsible:    port-amd64-maintainer
 > > >State:          open
 > > >Class:          sw-bug
 > > >Submitter-Id:   net
 > > >Arrival-Date:   Tue Aug 25 11:35:00 +0000 2026
 > > >Originator:     Andrius V
 > > >Release:        NetBSD 11/current
 > > >Organization:
 > > >Environment:
 > > >Description:
 > > Besides the cache associativity issue reported in the PR 60619, KX-7000=
  CPU fails to boot late in the boot process with a supervisor-mode page fau=
 lt (see the report below).
 > >
 > > After some digging, it appears that SMAP (Supervisor Mode Access Preven=
 tion) triggers the failure.
 > > Disabling SMAP allows the system to boot as a workaround.
 > >
 > > However, further analysis is needed to understand the actual cause.
 > > It may be a CPU issue, but I don't see CPU specific quirks in FreeBSD/L=
 inux, both of which boot successfully.
 > > It may instead be an edge case in the NetBSD code.
 > > AMD/Intel CPUs don't seem to suffer from this (at least I haven't found=
  any reports, and I use an AMD CPU supporting SMAP myself).
 > > OpenBSD fails similarly, although I didn't test if disabling SMAP helps=
 .
 > >
 > > bt:
 > > uvm_fault(0xffff80f576039040, 0xffff81409fdff000, 2) -> e
 > > fatal page fault in supervisor mode
 > > trap type 6 code 0x3 rip 0xffffffff805ccb21 cs 0x8 rflags 0x10202 cr2 0=
 xffff81409fdffff8 ilevel 0 rsp 0xffff9602473338e0
 > > curlwp 0xffff80f576441c00 pid 1.1 lowest kstack 0xffff96024732f2c0
 > > kernel: page fault trap, code=3D0
 > > Stopped in pid 1.1 (init) at    netbsd:pmap_enter_ma+0x6db:     movq   =
  %rax,0(%
 > > rdx)
 > > pmap_enter_ma() at netbsd:pmap_enter_ma+0x6db
 > > pmap_enter_default() at netbsd:pmap_enter_default+0x29
 > > uvm_fault_lower_enter() at netbsd:uvm_fault_lower_enter+0xba
 > > uvm_fault_internal() at netbsd:uvm_fault_internal+0xff8
 > > trap() at netbsd:trap+0x46b
 > > --- trap (number 6) ---
 > > copyout() at netbsd:copyout+0x33
 > > ds          1c00
 > > es          3968
 > > fs          3970
 > > gs          83b9
 > > rdi         ffff96000978f000
 > > rsi         ffff80f576cc2200
 > > rbp         ffff9602473339e0
 > > rbx         7f7fffffe000
 > > rdx         ffff81409fdffff8
 > > rcx         15
 > > rax         23d3a2007
 > > r8          ffffffff81a3cf00    cpu_info_primary
 > > r9          ffffffffffff
 > > r10         fffffffe
 > > r11         ffffffff81addf10    pmap_pvp_cache+0xd0
 > > r12         ffffffff81addc80    normal_pdes
 > > r13         1
 > > r14         3fbffff
 > > r15         2
 > > rip         ffffffff805ccb21    pmap_enter_ma+0x6db
 > > cs          8
 > > rflags      10202
 > > rsp         ffff9602473338e0
 > > ss          10
 > > netbsd:pmap_enter_ma+0x6db:     movq    %rax,0(%rdx)
 > >
 > > full dmesg until crash:
 > > WARNING: system needs entropy for security; see entropy(7)
 > > cpu_rng: rdrand/rdseed
 > > entropy: ready
 > > pool redzone disabled for 'pcache'
 > > pool redzone disabled for 'pdppl'
 > > pool redzone disabled for 'pvpage'
 > > pool redzone disabled for 'kmem-02048'
 > > pool redzone disabled for 'kmem-04096'
 > > Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
 > >     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
 > >     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
 > >     2024, 2025, 2026
 > >     The NetBSD Foundation, Inc.  All rights reserved.
 > > Copyright (c) 1982, 1986, 1989, 1991, 1993
 > >     The Regents of the University of California.  All rights reserved.
 > >
 > > NetBSD 11.99.7 (GENERIC) #3: Wed Aug 19 13:26:51 EEST 2026
 > >         andriusv@agraphic-pc:/home/andriusv/obj/sys/arch/amd64/compile/=
 GENERIC
 > > total memory =3D 7490 MB
 > > avail memory =3D 7216 MB
 > > pool redzone disabled for 'buf2k'
 > > pool redzone disabled for 'buf4k'
 > > pool redzone disabled for 'buf32k'
 > > pool redzone disabled for 'buf64k'
 > > entropy: no seed from bootloader
 > > pool redzone disabled for 'sigacts'
 > > pool redzone disabled for 'mclpl'
 > > pool redzone disabled for 'dirhashblk'
 > > pool redzone disabled for 'dirhashblk'
 > > timecounter: Timecounters tick every 10.000 msec
 > > Kernelized RAIDframe activated
 > > efi: systbl at pa 86b68018
 > > efi: systbl mapped at va ffff960236a92018
 > > efi: signature 5453595320494249 revision 20050 crc32 65108280
 > > efi: firmware revision 10000
 > > efi: runtime services at pa 0x86b68b98
 > > efi: boot services at pa 0x0
 > > efi: cfgtbl at pa 86b68c98
 > > efi: cfgtbl mapped at va ffff960236a94c98
 > > efi: 16 cfgtbl entries:
 > > efi: 0x86aae918 dba6a7e3-bb57-4be7-8af8-d578db7e5687
 > > efi: 0x86aae898 a31280ad-481e-41b6-95e8-127f4c984779
 > > efi: 0x8d5258c0 76b6bdfa-2acd-4462-9e3f-cb58c969d937
 > > efi: 0x86a71f98 ee4e5898-3914-4259-9d6e-dc7bd79403cf
 > > efi: 0x8d51ba60 05ad34ba-6f02-4214-952e-4da0398e2bb9
 > > efi: 0x86a83018 7739f24c-93d7-11d4-9a3a-0090273fc14d
 > > efi: 0x8d51c2d0 4c19049f-4137-4dd3-9c10-8b97a83ffdfa
 > > efi: 0x8d51ef68 49152e77-1ada-4764-b7a2-7afefed95e8b
 > > efi: 0x8d1c8f18 4e28ca50-d582-44ac-a11f-e3d56526db34
 > > efi: 0x86b65000 eb9d2d31-2d88-11d3-9a16-0090273fc14d SMBIOS
 > > efi: 0x86b63000 f2fd1544-9794-4a2c-992e-e5bbcf20e394 SMBIOS3
 > > efi: 0x8d1fd000 eb9d2d30-2d88-11d3-9a16-0090273fc14d ACPI 1.0
 > > efi: 0x8d1fd014 8868e871-e4f1-11d3-bc22-0080c73c8881 ACPI 2.0
 > > efi: 0x854de018 dcfa911d-26eb-469f-a220-38b7dc461220
 > > efi: 0x843de498 b122a263-3661-4f68-9929-78f8b0d62180
 > > efi: 0x86b1ab98 d719b2cb-3d3a-4596-a3bc-dad00e67656f
 > > efi_runtime_init: map 106 pages at 0 to 0x86aaf000 type 5 attrs 0x80000=
 0000000000f
 > > efi_runtime_init: map 80 pages at 0 to 0x86b19000 type 6 attrs 0x800000=
 000000000f
 > > efi_runtime_init: map 65536 pages at 0 to 0xe0000000 type 0 attrs 0x800=
 0000000000001
 > > efi_runtime_init: map 1 pages at 0 to 0xfe020000 type 0 attrs 0x8000000=
 000000001
 > > efi_runtime_init: map 128 pages at 0 to 0xfeb00000 type 0 attrs 0x80000=
 00000000001
 > > efi_runtime_init: map 1 pages at 0 to 0xfec00000 type 0 attrs 0x8000000=
 000000001
 > > efi_runtime_init: map 1 pages at 0 to 0xfecc0000 type 0 attrs 0x8000000=
 000000001
 > > efi_runtime_init: map 4352 pages at 0 to 0xfef00000 type 0 attrs 0x8000=
 000000000001
 > > SMBIOS rev. 3.6.0 @ 0x86b62000
 > > mainbus0 (root)
 > > ACPI: RSDP 0x000000008D1FD014 000024 (v02 _BYO_ )
 > > ACPI: XSDT 0x000000008D1FC0E8 000064 (v01 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: FACP 0x000000008D1FA000 00010C (v05 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: DSDT 0x000000008D1EA000 00DE00 (v02 _BYO_  ZX_PLAT  00000001 INTL=
  20190215)
 > > ACPI: FACS 0x000000008D1DF000 000040
 > > ACPI: APIC 0x000000008D1FB000 0000C6 (v03 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: HPET 0x000000008D1F9000 000038 (v01 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: MCFG 0x000000008D1F8000 00003C (v01 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: SSDM 0x000000008D1E9000 000055 (v01 _BYO_  BYOSOFT  00000002 BYO =
  00000001)
 > > ACPI: FPDT 0x000000008D1E8000 000044 (v01 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: SSDT 0x000000008D1E7000 000144 (v02 ZXPLTA ADebTabl 00001000 INTL=
  20190215)
 > > ACPI: BGRT 0x000000008D1E6000 000038 (v01 _BYO_  BYOSOFT  00000002 BYO =
  01000013)
 > > ACPI: 2 ACPI AML tables successfully acquired and loaded
 > > ioapic0 at mainbus0 apid 0: pa 0xfec00000, version 0x20, 24 pins
 > > ioapic1 at mainbus0 apid 1: pa 0xfec01000, version 0x20, 24 pins
 > > cpu0 at mainbus0 apid 0
 > > cpu0: 16 page colors
 > > cpu0: Use cpuid to serialize rdtsc
 > > cpu0: TSC freq from delay 2966316600 Hz
 > > cpu0: calibrating local timer
 > > cpu0: apic clock running at 99 MHz
 > > cpu0: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu0: node 0, package 0, core 0, smt 0
 > > cpu1 at mainbus0 apid 1
 > > cpu1: 2 page colors
 > > i82489_ipi_init: ESR 00000040
 > > cpu1: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu1: node 0, package 0, core 0, smt 0
 > > cpu2 at mainbus0 apid 2
 > > cpu2: 2 page colors
 > > cpu2: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu2: node 0, package 0, core 0, smt 0
 > > cpu3 at mainbus0 apid 3
 > > cpu3: 2 page colors
 > > cpu3: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu3: node 0, package 0, core 0, smt 0
 > > cpu4 at mainbus0 apid 4
 > > cpu4: 2 page colors
 > > cpu4: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu4: node 0, package 0, core 0, smt 0
 > > cpu5 at mainbus0 apid 5
 > > cpu5: 2 page colors
 > > cpu5: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu5: node 0, package 0, core 0, smt 0
 > > cpu6 at mainbus0 apid 6
 > > cpu6: 2 page colors
 > > cpu6: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu6: node 0, package 0, core 0, smt 0
 > > cpu7 at mainbus0 apid 7
 > > cpu7: 2 page colors
 > > cpu7: ZHAOXIN KaiXian KX-7000, id 0x607b1
 > > cpu7: node 0, package 0, core 0, smt 0
 > > acpi0 at mainbus0: Intel ACPICA 20260408
 > > acpi0: X/RSDT: OemId <_BYO_ ,BYOSOFT ,00000002>, AslId <BYO ,01000013>
 > > allocated pic ioapic0 type level pin 9 level 6 to cpu0 slot 0 idt entry=
  96
 > > acpi0: MCH0 dependencies: ND00
 > > acpi0: GPI0 dependencies: ND00
 > > acpi0: NBF1 dependencies: ND00
 > > acpi0: NBF2 dependencies: ND00
 > > acpi0: NBF3 dependencies: ND00
 > > acpi0: NBF4 dependencies: ND00
 > > acpi0: NBF5 dependencies: ND00
 > > acpi0: NBF6 dependencies: ND00
 > > acpi0: NPE0 dependencies: ND00
 > > acpi0: PE0S dependencies: NPE0
 > > acpi0: NPE1 dependencies: ND00
 > > acpi0: PE1S dependencies: NPE1
 > > acpi0: NPE2 dependencies: ND00
 > > acpi0: PE2S dependencies: NPE2
 > > acpi0: NPE3 dependencies: ND00
 > > acpi0: PE3S dependencies: NPE3
 > > acpi0: NPE5 dependencies: ND00
 > > acpi0: PE5S dependencies: NPE5
 > > acpi0: NPE6 dependencies: ND00
 > > acpi0: PE6S dependencies: NPE6
 > > acpi0: NPE7 dependencies: ND00
 > > acpi0: PE7S dependencies: NPE7
 > > acpi0: NPE8 dependencies: ND00
 > > acpi0: PE8S dependencies: NPE8
 > > acpi0: NPG0 dependencies: ND00
 > > acpi0: PG0S dependencies: NPG0
 > > acpi0: NPG1 dependencies: ND00
 > > acpi0: PG1S dependencies: NPG1
 > > acpi0: NPG2 dependencies: ND00
 > > acpi0: PG2S dependencies: NPG2
 > > acpi0: NPG3 dependencies: ND00
 > > acpi0: PG3S dependencies: NPG3
 > > acpi0: NPG4 dependencies: ND00
 > > acpi0: PG4S dependencies: NPG4
 > > acpi0: NPG5 dependencies: ND00
 > > acpi0: PG5S dependencies: NPG5
 > > acpi0: NPG6 dependencies: ND00
 > > acpi0: PG6S dependencies: NPG6
 > > acpi0: NPG7 dependencies: ND00
 > > acpi0: PG7S dependencies: NPG7
 > > acpi0: NPG8 dependencies: ND00
 > > acpi0: PG8S dependencies: NPG8
 > > acpi0: DRI0 dependencies: ND00
 > > acpi0: PEPD dependencies: ND00
 > > acpi0: EUMA dependencies: ND00
 > > acpi0: CRT dependencies: EUMA
 > > acpi0: LCD dependencies: EUMA
 > > acpi0: TV dependencies: EUMA
 > > acpi0: DVI dependencies: EUMA
 > > acpi0: HDMI dependencies: EUMA
 > > acpi0: HDTV dependencies: EUMA
 > > acpi0: DP dependencies: EUMA
 > > acpi0: DP2 dependencies: EUMA
 > > acpi0: EDP1 dependencies: EUMA
 > > acpi0: S3AC dependencies: ND00
 > > acpi0: ATA0 dependencies: ND00
 > > acpi0: CHN0 dependencies: ATA0
 > > acpi0: DRV0 dependencies: CHN0
 > > acpi0: DRV1 dependencies: CHN0
 > > acpi0: CHN1 dependencies: ATA0
 > > acpi0: DRV0 dependencies: CHN1
 > > acpi0: DRV1 dependencies: CHN1
 > > acpi0: NHI0 dependencies: ND00
 > > acpi0: USB0 dependencies: ND00
 > > acpi0: USB1 dependencies: ND00
 > > acpi0: USB2 dependencies: ND00
 > > acpi0: EHC1 dependencies: ND00
 > > acpi0: USB3 dependencies: ND00
 > > acpi0: USB4 dependencies: ND00
 > > acpi0: EHC2 dependencies: ND00
 > > acpi0: XHC0 dependencies: ND00
 > > acpi0: RHUB dependencies: XHC0
 > > acpi0: HS01 dependencies: RHUB
 > > acpi0: HS02 dependencies: RHUB
 > > acpi0: HS03 dependencies: RHUB
 > > acpi0: HS04 dependencies: RHUB
 > > acpi0: HS05 dependencies: RHUB
 > > acpi0: HS06 dependencies: RHUB
 > > acpi0: HS07 dependencies: RHUB
 > > acpi0: HS08 dependencies: RHUB
 > > acpi0: HS09 dependencies: RHUB
 > > acpi0: HS10 dependencies: RHUB
 > > acpi0: SS01 dependencies: RHUB
 > > acpi0: SS02 dependencies: RHUB
 > > acpi0: SS03 dependencies: RHUB
 > > acpi0: SS04 dependencies: RHUB
 > > acpi0: SS05 dependencies: RHUB
 > > acpi0: SS06 dependencies: RHUB
 > > acpi0: HDAC dependencies: ND00
 > > acpi0: SBRG dependencies: ND00
 > > acpi0: WDRT dependencies: SBRG
 > > acpi0: PICI dependencies: SBRG
 > > acpi0: DMAD dependencies: SBRG
 > > acpi0: TMR dependencies: SBRG
 > > acpi0: RTC0 dependencies: SBRG
 > > acpi0: SPKR dependencies: SBRG
 > > acpi0: HPET dependencies: SBRG
 > > acpi0: RMSC dependencies: SBRG
 > > acpi0: COPR dependencies: SBRG
 > > acpi0: I2C0 dependencies: SBRG
 > > acpi0: TPD0 dependencies: I2C0
 > > acpi0: TPD2 dependencies: I2C0
 > > acpi0: I2S0 dependencies: I2C0
 > > acpi0: WADT dependencies: SBRG
 > > acpi0: PS2M dependencies: SBRG
 > > acpi0: PS2K dependencies: SBRG
 > > acpi0: LPT dependencies: SBRG
 > > acpi0: ECP dependencies: SBRG
 > > acpi0: UAR1 dependencies: SBRG
 > > acpi0: UAR2 dependencies: SBRG
 > > acpi0: VTSB dependencies: ND00
 > > acpi0: SMBC dependencies: ND00
 > > acpi0: SVLK dependencies: ND00
 > > acpi0: MCFG: segment 0, bus 0-255, address 0x00000000e0000000
 > > MCFG: MEMMAP: 0x0000000000000000-0x000000000009bfff, size=3D0x000000000=
 009c000, type=3D1(Memory)
 > > MCFG: MEMMAP: 0x000000000009c000-0x000000000009ffff, size=3D0x000000000=
 0004000, type=3D2(Reserved)
 > > MCFG: MEMMAP: 0x0000000000100000-0x0000000086aaefff, size=3D0x000000008=
 69af000, type=3D1(Memory)
 > > MCFG: MEMMAP: 0x0000000086aaf000-0x000000008d1c8fff, size=3D0x000000000=
 671a000, type=3D2(Reserved)
 > > MCFG: MEMMAP: 0x000000008d1c9000-0x000000008d1dffff, size=3D0x000000000=
 0017000, type=3D4(NVS)
 > > MCFG: MEMMAP: 0x000000008d1e0000-0x000000008d1fdfff, size=3D0x000000000=
 001e000, type=3D3(ACPI)
 > > MCFG: MEMMAP: 0x000000008d1fe000-0x000000008f0e0fff, size=3D0x000000000=
 1ee3000, type=3D1(Memory)
 > > MCFG: MEMMAP: 0x000000008f0e1000-0x0000000094121fff, size=3D0x000000000=
 5041000, type=3D2(Reserved)
 > > MCFG: MEMMAP: 0x0000000094122000-0x00000000962abfff, size=3D0x000000000=
 218a000, type=3D1(Memory)
 > > MCFG: MEMMAP: 0x00000000962ac000-0x00000000962acfff, size=3D0x000000000=
 0001000, type=3D2(Reserved)
 > > MCFG: MEMMAP: 0x00000000962ad000-0x000000009f9fdfff, size=3D0x000000000=
 9751000, type=3D1(Memory)
 > > MCFG: MEMMAP: 0x000000009f9fe000-0x000000009fffffff, size=3D0x000000000=
 0602000, type=3D2(Reserved)
 > > MCFG: MEMMAP: 0x00000000e0000000-0x00000000efffffff, size=3D0x000000001=
 0000000, type=3D2(Reserved)
 > > acpi0: MCFG: segment 0, bus 0-255, address 0x00000000e0000000
 > > acpi0: MCFG: Bus=3D0, Seg=3D0xffff80f447a77cc0
 > > MCFG: range 0x0000 size 0x100 (16-bit bus)
 > > MCFG: range 0x0000 size 0x3b0 (16-bit io)
 > > MCFG: range 0x03e0 size 0x918 (16-bit io)
 > > MCFG: range 0x1000 size 0xf000 (16-bit io)
 > > MCFG: range 0x03b0 size 0x30 (16-bit io)
 > > MCFG: range 0x000a0000 size 0x20000 (32-bit mem)
 > > MCFG: range 0x000c0000 size 0x20000 (32-bit mem)
 > > MCFG: range 0xa0000000 size 0x40000000 (32-bit mem)
 > > MCFG: range 0x00000000 size 0 (32-bit mem)
 > > MCFG: range 0x00000000 size 0 (32-bit mem)
 > > MCFG: range 0x00000000 size 0 (32-bit mem)
 > > MCFG: range 0x00000000 size 0 (32-bit mem)
 > > MCFG: range 0x0000007800000000 size 0x800000000 (64-bit pmem)
 > > PCI: 0000:00:00.0 1d17:1007 30 0x060000 (device)
 > > PCI: 0000:00:00.1 1d17:31b0 30 0x060000 (device)
 > > PCI: 0000:00:00.2 1d17:31b1 30 0x060000 (device)
 > > PCI: 0000:00:00.3 1d17:31b2 30 0x060000 (device)
 > > PCI: 0000:00:00.4 1d17:31b3 30 0x060000 (device)
 > > PCI: 0000:00:00.5 1d17:31b4 30 0x060000 (device)
 > > PCI: 0000:00:00.6 1d17:31b5 30 0x060000 (device)
 > > PCI: 0000:00:00.7 1d17:3200 30 0x060000 (device)
 > > PCI: 0000:00:01.0 1d17:3a05 30 0x030000 (device)
 > > PCI: 0000:00:01.0 [device] resource BAR0: mem @ 0xc0000000 size 0x20000=
 0, 32-bit non-prefetchable
 > > PCI: 0000:00:01.0 [device] resource BAR1: mem @ 0xa0000000 size 0x20000=
 000, 64-bit prefetchable
 > > PCI: 0000:00:01.1 1d17:9146 30 0x040300 (device)
 > > PCI: 0000:00:01.1 [device] resource BAR0: mem @ 0xc0304000 size 0x4000,=
  32-bit non-prefetchable
 > > PCI: 0000:00:02.0 1d17:0717 30 0x060400 (bridge 0 -> 1-1)
 > > PCI: 0000:00:03.0 1d17:0719 30 0x060400 (bridge 0 -> 2-2)
 > > PCI: 0000:00:04.0 1d17:071b 30 0x060400 (bridge 0 -> 3-3)
 > > PCI: 0000:00:05.0 1d17:071d 30 0x060400 (bridge 0 -> 4-4)
 > > PCI: 0000:00:05.0 [bridge] window io  0x1000-0x1fff
 > > PCI: 0000:00:05.0 [bridge] window mem 0xc0200000-0xc02fffff (non-prefet=
 chable)
 > > PCI: 0000:04:00.0 10ec:8168 15 0x020000 (device)
 > > PCI: 0000:04:00.0 [device] resource BAR0: io  @ 0x1000 size 0x100
 > > PCI: 0000:04:00.0 [device] resource BAR2: mem @ 0xc0204000 size 0x1000,=
  64-bit non-prefetchable
 > > PCI: 0000:04:00.0 [device] resource BAR4: mem @ 0xc0200000 size 0x4000,=
  64-bit non-prefetchable
 > > PCI: 0000:00:05.1 1d17:071e 30 0x060400 (bridge 0 -> 5-5)
 > > PCI: 0000:00:05.2 1d17:0732 30 0x060400 (bridge 0 -> 6-6)
 > > PCI: 0000:00:05.3 1d17:073b 30 0x060400 (bridge 0 -> 7-7)
 > > PCI: 0000:00:0e.0 1d17:3038 30 0x0c0300 (device)
 > > PCI: 0000:00:0e.0 [device] resource BAR4: io  @ 0x20a0 size 0x20
 > > PCI: 0000:00:0e.1 1d17:3038 30 0x0c0300 (device)
 > > PCI: 0000:00:0e.1 [device] resource BAR4: io  @ 0x2080 size 0x20
 > > PCI: 0000:00:0e.2 1d17:3038 30 0x0c0300 (device)
 > > PCI: 0000:00:0e.2 [device] resource BAR4: io  @ 0x2060 size 0x20
 > > PCI: 0000:00:0e.7 1d17:3104 30 0x0c0320 (device)
 > > PCI: 0000:00:0e.7 [device] resource BAR0: mem @ 0xc030e000 size 0x100, =
 32-bit non-prefetchable
 > > PCI: 0000:00:10.0 1d17:3038 30 0x0c0300 (device)
 > > PCI: 0000:00:10.0 [device] resource BAR4: io  @ 0x2020 size 0x20
 > > PCI: 0000:00:10.1 1d17:3038 30 0x0c0300 (device)
 > > PCI: 0000:00:10.1 [device] resource BAR4: io  @ 0x2000 size 0x20
 > > PCI: 0000:00:10.7 1d17:3104 30 0x0c0320 (device)
 > > PCI: 0000:00:10.7 [device] resource BAR0: mem @ 0xc030d000 size 0x100, =
 32-bit non-prefetchable
 > > PCI: 0000:00:11.0 1d17:1001 30 0x060100 (device)
 > > PCI: 0000:00:11.7 1d17:31b7 30 0x060000 (device)
 > > PCI: 0000:00:12.0 1d17:9204 30 0x0c0330 (device)
 > > PCI: 0000:00:12.0 [device] resource BAR0: mem @ 0xc030a000 size 0x1000,=
  64-bit non-prefetchable
 > > PCI: 0000:00:12.0 [device] resource BAR2: mem @ 0xc030b000 size 0x1000,=
  64-bit non-prefetchable
 > > PCI: 0000:00:12.0 [device] resource BAR4: mem @ 0xc030c000 size 0x1000,=
  64-bit non-prefetchable
 > > PCI: 0000:00:14.0 1d17:3288 30 0x040300 (device)
 > > PCI: 0000:00:14.0 [device] resource BAR0: mem @ 0xc0300000 size 0x4000,=
  64-bit non-prefetchable
 > > PCI: 0000:00:1f.0 1d17:3c00 30 0x060000 (device)
 > > PCI: 0000:00:01.0 [device] mem (prefetchable) 0xa0000000-0xbfffffff inv=
 alid (12)
 > > PCI: 0000:00:01.1 [device] not configured by firmware
 > > PCI: 0000:00:01.1 BAR0 assigned range 0xc0304000-0xc0307fff
 > > acpi0: MCFG: Bus=3D1, Seg=3D0xffff80f447a77cc0
 > > acpi0: MCFG: Walk _CRS: 5
 > > MPACPI PCI bus number 1 configuration failed: 6
 > > acpi0: SCI interrupting at int 9
 > > acpi0: fixed power button present
 > > timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
 > > acpi0: ACPI-Fast 24-bit timer
 > > hpet0 at acpi0: high precision event timer (mem 0xfeb97000-0xfeb97400)
 > > timecounter: Timecounter "hpet0" frequency 14318180 Hz quality 2000
 > > MCH0 (PNP0C01) at acpi0 not configured
 > > acpivga0 at acpi0 (EUMA): ACPI Display Adapter
 > > acpivga0: capabilities: _DOS _DOD
 > > acpiout0 at acpivga0 (CRT, 0x0100): ACPI Display Output Device
 > > acpiout0: capabilities: _DCS _DGS _DSS
 > > acpiout1 at acpivga0 (LCD, 0x0110): ACPI Display Output Device
 > > acpiout1: capabilities: _BCL _BCM _DCS _DGS _DSS
 > > acpiout1: failed to test _BQC quirk
 > > acpiout1: brightness levels: [20,30,40,50,60,80,90,100]
 > > acpiout2 at acpivga0 (TV, 0x0200): ACPI Display Output Device
 > > acpiout2: capabilities: _DCS _DGS _DSS
 > > acpiout3 at acpivga0 (DVI, 0x0120): ACPI Display Output Device
 > > acpiout3: capabilities: _DCS _DGS _DSS
 > > acpiout4 at acpivga0 (HDMI, 0x0240): ACPI Display Output Device
 > > acpiout4: capabilities: _DCS _DGS _DSS
 > > acpiout5 at acpivga0 (HDTV, 0x0210): ACPI Display Output Device
 > > acpiout5: capabilities: _DCS _DGS _DSS
 > > acpiout6 at acpivga0 (DP, 0x0280): ACPI Display Output Device
 > > acpiout6: capabilities: _DCS _DGS _DSS
 > > acpiout7 at acpivga0 (DP2, 0x02a0): ACPI Display Output Device
 > > acpiout7: capabilities: _DCS _DGS _DSS
 > > acpiout8 at acpivga0 (EDP1, 0x0111): ACPI Display Output Device
 > > acpiout8: capabilities: _BCL _BCM _BQC _DCS _DGS _DSS
 > > acpiout8: brightness levels: [0,10,20,30,40,50,60,70,80,90,100]
 > > acpivga0: connected output devices:
 > > acpivga0:   0x0100 (acpiout0): Ext. Monitor, head 0, bios detect
 > > acpivga0:   0x0110 (acpiout1): LCD Panel, head 0, bios detect
 > > acpivga0:   0x0111 (acpiout8): Unknown Output Device, head 0, bios dete=
 ct
 > > acpivga0:   0x0120 (acpiout3): Unknown Output Device, head 0, bios dete=
 ct
 > > acpivga0:   0x0240 (acpiout4): Unknown Output Device, head 0, bios dete=
 ct
 > > acpivga0:   0x0200 (acpiout2): TV, head 0, bios detect
 > > acpivga0:   0x0210 (acpiout5): Unknown Output Device, head 0, bios dete=
 ct
 > > acpivga0:   0x0280 (acpiout6): Unknown Output Device, head 0, bios dete=
 ct
 > > acpivga0:   0x02a0 (acpiout7): Unknown Output Device, head 0, bios dete=
 ct
 > > attimer1 at acpi0 (TMR, PNP0100): io 0x40-0x43
 > > pcppi1 at acpi0 (SPKR, PNP0800): io 0x61
 > > spkr0 at pcppi1: PC Speaker
 > > wsbell at spkr0 not configured
 > > midi0 at pcppi1: PC speaker
 > > sysbeep0 at pcppi1
 > > TPD2 (HTIX5288) at acpi0 not configured
 > > I2S0 (ESSXB261) at acpi0 not configured
 > > lpt0 at acpi0 (LPT, PNP0400): io 0x378-0x37f irq 7
 > > allocated pic ioapic0 type edge pin 7 level 6 to cpu0 slot 1 idt entry =
 97
 > > com0 at acpi0 (UAR1, PNP0501-1): io 0x3f8-0x3ff irq 4
 > > com: ns16550a, 16-byte FIFO
 > > com0: console
 > > allocated pic ioapic0 type edge pin 4 level 8 to cpu0 slot 2 idt entry =
 129
 > > com1 at acpi0 (UAR2, PNP0501-2): io 0x2f8-0x2ff irq 3
 > > com1: ns16550a, 16-byte FIFO
 > > allocated pic ioapic0 type edge pin 3 level 8 to cpu0 slot 3 idt entry =
 130
 > > VTSB (PNP0C01) at acpi0 not configured
 > > acpibut0 at acpi0 (PWRB, PNP0C0C-170): ACPI Power Button
 > > attimer1: attached to pcppi1
 > > pci0 at mainbus0 bus 0: configuration mode 1
 > > acpi0: MCFG: 000:00:0: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:0: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:1: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:1: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:2: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:2: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:3: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:3: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:4: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:4: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:5: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:5: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:6: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:6: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:00:7: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:00:7: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:01:0: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:01:0: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:01:1: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:01:1: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: 000:02:0: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:03:0: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:04:0: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:05:0: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:05:1: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:05:2: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:05:3: Ok (cfg[0x100]=3D0x15010001 extconf=3DY)
 > > acpi0: MCFG: 000:14:0: invalid config space (cfg[0x100]=3D0x30381d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:14:0: Ok (cfg[0x100]=3D0x30381d17 extconf=3DN)
 > > acpi0: MCFG: 000:14:1: invalid config space (cfg[0x100]=3D0x30381d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:14:1: Ok (cfg[0x100]=3D0x30381d17 extconf=3DN)
 > > acpi0: MCFG: 000:14:2: invalid config space (cfg[0x100]=3D0x30381d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:14:2: Ok (cfg[0x100]=3D0x30381d17 extconf=3DN)
 > > acpi0: MCFG: 000:14:7: invalid config space (cfg[0x100]=3D0x31041d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:14:7: Ok (cfg[0x100]=3D0x31041d17 extconf=3DN)
 > > acpi0: MCFG: 000:16:0: invalid config space (cfg[0x100]=3D0x30381d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:16:0: Ok (cfg[0x100]=3D0x30381d17 extconf=3DN)
 > > acpi0: MCFG: 000:16:1: invalid config space (cfg[0x100]=3D0x30381d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:16:1: Ok (cfg[0x100]=3D0x30381d17 extconf=3DN)
 > > acpi0: MCFG: 000:16:7: invalid config space (cfg[0x100]=3D0x31041d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:16:7: Ok (cfg[0x100]=3D0x31041d17 extconf=3DN)
 > > acpi0: MCFG: 000:17:0: invalid config space (cfg[0x100]=3D0x10011d17, a=
 lias=3Dtrue)
 > > acpi0: MCFG: 000:17:0: Ok (cfg[0x100]=3D0x10011d17 extconf=3DN)
 > > acpi0: MCFG: 000:17:7: invalid config space (cfg[0x100]=3D0xffffffff, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:17:7: Ok (cfg[0x100]=3D0xffffffff extconf=3DN)
 > > acpi0: MCFG: 000:18:0: invalid config space (cfg[0x100]=3D0xffffffff, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:18:0: Ok (cfg[0x100]=3D0xffffffff extconf=3DN)
 > > acpi0: MCFG: 000:20:0: Ok (cfg[0x100]=3D0x13010002 extconf=3DY)
 > > acpi0: MCFG: 000:31:0: invalid config space (cfg[0x100]=3D0x00000000, a=
 lias=3Dfalse)
 > > acpi0: MCFG: 000:31:0: Ok (cfg[0x100]=3D0x00000000 extconf=3DN)
 > > acpi0: MCFG: bus 0: valid devices
 > > acpi0: MCFG: 000:00:0
 > > acpi0: MCFG: 000:00:1
 > > acpi0: MCFG: 000:00:2
 > > acpi0: MCFG: 000:00:3
 > > acpi0: MCFG: 000:00:4
 > > acpi0: MCFG: 000:00:5
 > > acpi0: MCFG: 000:00:6
 > > acpi0: MCFG: 000:00:7
 > > acpi0: MCFG: 000:01:0
 > > acpi0: MCFG: 000:01:1
 > > acpi0: MCFG: 000:02:0
 > > acpi0: MCFG: 000:03:0
 > > acpi0: MCFG: 000:04:0
 > > acpi0: MCFG: 000:05:0
 > > acpi0: MCFG: 000:05:1
 > > acpi0: MCFG: 000:05:2
 > > acpi0: MCFG: 000:05:3
 > > acpi0: MCFG: 000:14:0
 > > acpi0: MCFG: 000:14:1
 > > acpi0: MCFG: 000:14:2
 > > acpi0: MCFG: 000:14:7
 > > acpi0: MCFG: 000:16:0
 > > acpi0: MCFG: 000:16:1
 > > acpi0: MCFG: 000:16:7
 > > acpi0: MCFG: 000:17:0
 > > acpi0: MCFG: 000:17:7
 > > acpi0: MCFG: 000:18:0
 > > acpi0: MCFG: 000:20:0
 > > acpi0: MCFG: 000:31:0
 > >
 > > pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
 > > pchb0 at pci0 dev 0 function 0: Zhaoxin product 1007 (rev. 0x30)
 > > pchb1 at pci0 dev 0 function 1: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 Standard Host Bridge (rev. 0x30)
 > > pchb2 at pci0 dev 0 function 2: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 Standard Host Bridge (rev. 0x30)
 > > pchb3 at pci0 dev 0 function 3: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 DRAM Controller (rev. 0x30)
 > > pchb4 at pci0 dev 0 function 4: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 Power Management Controller (rev. 0x30)
 > > pchb5 at pci0 dev 0 function 5: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 I/O APIC (rev. 0x30)
 > > pchb6 at pci0 dev 0 function 6: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 /=
  KH-40000 Scratch Device (rev. 0x30)
 > > pchb7 at pci0 dev 0 function 7: Zhaoxin product 3200 (rev. 0x30)
 > > genfb0 at pci0 dev 1 function 0: Zhaoxin KX-7000 C-1190 Integrated Grap=
 hics (rev. 0x30)
 > > genfb0: framebuffer at 0xa0000000, size 1024x768, depth 32, stride 4096
 > > wsdisplay0 at genfb0 kbdmux 1
 > > wsmux1: connecting to wsdisplay0
 > > drm at genfb0 not configured
 > > hdaudio0 at pci0 dev 1 function 1: HD Audio Controller
 > > allocated pic msi0 type edge pin 0 level 7 to cpu0 slot 16 idt entry 11=
 2
 > > hdaudio0: interrupting at msi0 vec 0
 > > hdaudio0: HDA ver. 1.0, OSS 3, ISS 0, BSS 0, SDO 1
 > > hdafg0 at hdaudio0 vendor 0x1D17 product 0x9F8E nid 0x01: Zhaoxin produ=
 ct 9f8e
 > > hdafg0: HDMI00 8ch: Digital Out [Jack]
 > >     nid=3D03 [pin: Digital Out (Red Jack)]
 > >            nid=3D02 [audio output] [source: dac]
 > > hdafg0: 8ch/0ch 32000Hz 44100Hz 48000Hz 88200Hz 96000Hz 192000Hz PCM16 =
 PCM20 PCM24 PCM32 AC3
 > > audio0 at hdafg0: playback
 > > audio0: slinear_le:16 2ch 48000Hz, blk 1920 bytes (10ms) for playback
 > > spkr1 at audio0: PC Speaker (synthesized)
 > > wsbell at spkr1 not configured
 > > hdafg1 at hdaudio0 vendor 0x1D17 product 0x9F8F nid 0x01: Zhaoxin produ=
 ct 9f8f
 > > hdafg1: HDMI00 8ch: Digital Out [Jack]
 > >     nid=3D03 [pin: Digital Out (Red Jack)]
 > >            nid=3D02 [audio output] [source: dac]
 > > hdafg1: 8ch/0ch 32000Hz 44100Hz 48000Hz 88200Hz 96000Hz 192000Hz PCM16 =
 PCM20 PCM24 PCM32 AC3
 > > audio1 at hdafg1: playback
 > > audio1: slinear_le:16 2ch 48000Hz, blk 1920 bytes (10ms) for playback
 > > spkr2 at audio1: PC Speaker (synthesized)
 > > wsbell at spkr2 not configured
 > > hdafg2 at hdaudio0 vendor 0x1D17 product 0x9F90 nid 0x01: Zhaoxin produ=
 ct 9f90
 > > hdafg2: HDMI00 8ch: Digital Out [Jack]
 > >     nid=3D03 [pin: Digital Out (Red Jack)]
 > >            nid=3D02 [audio output] [source: dac]
 > > hdafg2: 8ch/0ch 32000Hz 44100Hz 48000Hz 88200Hz 96000Hz 192000Hz PCM16 =
 PCM20 PCM24 PCM32 AC3
 > > audio2 at hdafg2: playback
 > > audio2: slinear_le:16 2ch 48000Hz, blk 1920 bytes (10ms) for playback
 > > spkr3 at audio2: PC Speaker (synthesized)
 > > wsbell at spkr3 not configured
 > > ppb0 at pci0 dev 2 function 0: Zhaoxin KX-5000|6000(G)|7000 / KH-40000 =
 PCI Express Root Port (rev. 0x30)
 > > ppb0: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x8 @ 16.0GT/s
 > > pci1 at ppb0 bus 1
 > > acpi0: MCFG: bus 1: no valid devices.
 > >
 > > pci1: i/o space, memory space enabled, rd/line, wr/inv ok
 > > ppb1 at pci0 dev 3 function 0: Zhaoxin KX-5000|6000(G)|7000 / KH-40000 =
 PCI Express Root Port (rev. 0x30)
 > > ppb1: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x1 @ 16.0GT/s
 > > pci2 at ppb1 bus 2
 > > acpi0: MCFG: bus 2: no valid devices.
 > >
 > > pci2: i/o space, memory space enabled, rd/line, wr/inv ok
 > > ppb2 at pci0 dev 4 function 0: Zhaoxin KX-5000|6000(G)|7000 / KH-40000 =
 PCI Express Root Port (rev. 0x30)
 > > ppb2: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x4 @ 16.0GT/s
 > > pci3 at ppb2 bus 3
 > > acpi0: MCFG: bus 3: no valid devices.
 > >
 > > pci3: i/o space, memory space enabled, rd/line, wr/inv ok
 > > ppb3 at pci0 dev 5 function 0: Zhaoxin KX-5000|6000(G) / KH-40000 PCI E=
 xpress Root Port (rev. 0x30)
 > > ppb3: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x1 @ 16.0GT/s
 > > ppb3: link is x1 @ 2.5GT/s
 > > pci4 at ppb3 bus 4
 > > acpi0: MCFG: 004:00:0: Ok (cfg[0x100]=3D0x14020001 extconf=3DY)
 > > acpi0: MCFG: bus 4: valid devices
 > > acpi0: MCFG: 004:00:0
 > >
 > > pci4: i/o space, memory space enabled, rd/line, wr/inv ok
 > > re0 at pci4 dev 0 function 0pci_mem_find: void region
 > > : RealTek 8168/8111 PCIe Gigabit Ethernet (rev. 0x15)
 > > allocated pic msix1 type edge pin 0 level 6 to cpu0 slot 17 idt entry 9=
 8
 > > re0: interrupting at msix1 vec 0
 > > re0: RTL8168H/8111H (0x5400)
 > > re0: Ethernet address 90:e2:fc:b7:ca:43
 > > re0: using 1024 tx descriptors
 > > rgephy0 at re0 phy 7: RTL8251 1000BASE-T media interface, rev. 0
 > > rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX,=
  auto
 > > ppb4 at pci0 dev 5 function 1: Zhaoxin KX-5000|6000(G)|7000 / KH-40000 =
 PCI Express Root Port (rev. 0x30)
 > > ppb4: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x1 @ 16.0GT/s
 > > pci5 at ppb4 bus 5
 > > acpi0: MCFG: bus 5: no valid devices.
 > >
 > > pci5: i/o space, memory space enabled, rd/line, wr/inv ok
 > > ppb5 at pci0 dev 5 function 2: Zhaoxin product 0732 (rev. 0x30)
 > > ppb5: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x1 @ 16.0GT/s
 > > pci6 at ppb5 bus 6
 > > acpi0: MCFG: bus 6: no valid devices.
 > >
 > > pci6: i/o space, memory space enabled, rd/line, wr/inv ok
 > > ppb6 at pci0 dev 5 function 3: Zhaoxin product 073b (rev. 0x30)
 > > ppb6: PCI Express capability version 2 <Root Port of PCI-E Root Complex=
 > x1 @ 16.0GT/s
 > > pci7 at ppb6 bus 7
 > > acpi0: MCFG: bus 7: no valid devices.
 > >
 > > pci7: i/o space, memory space enabled, rd/line, wr/inv ok
 > > uhci0 at pci0 dev 14 function 0: Zhaoxin USB UHCI Controller (rev. 0x30=
 )
 > > allocated pic ioapic0 type level pin 23 level 6 to cpu0 slot 4 idt entr=
 y 99
 > > uhci0: interrupting at ioapic0 pin 23
 > > usb0 at uhci0: USB revision 1.0
 > > uhci1 at pci0 dev 14 function 1: Zhaoxin USB UHCI Controller (rev. 0x30=
 )
 > > allocated pic ioapic0 type level pin 22 level 6 to cpu0 slot 5 idt entr=
 y 100
 > > uhci1: interrupting at ioapic0 pin 22
 > > usb1 at uhci1: USB revision 1.0
 > > uhci2 at pci0 dev 14 function 2: Zhaoxin USB UHCI Controller (rev. 0x30=
 )
 > > allocated pic ioapic0 type level pin 21 level 6 to cpu0 slot 6 idt entr=
 y 101
 > > uhci2: interrupting at ioapic0 pin 21
 > > usb2 at uhci2: USB revision 1.0
 > > ehci0 at pci0 dev 14 function 7: Zhaoxin USB EHCI Controller (rev. 0x30=
 )
 > > ehci0: 32-bit DMA
 > > allocated pic ioapic0 type level pin 20 level 6 to cpu0 slot 7 idt entr=
 y 102
 > > ehci0: interrupting at ioapic0 pin 20
 > > ehci0: EHCI version 1.0
 > > ehci0: 3 companion controllers, 2 ports each: uhci0 uhci1 uhci2
 > > ehci0: Using DMA subregion for control data structures
 > > usb3 at ehci0: USB revision 2.0
 > > uhci3 at pci0 dev 16 function 0: Zhaoxin USB UHCI Controller (rev. 0x30=
 )
 > > allocated pic ioapic0 type level pin 20 level 6 to cpu0 slot 7 idt entr=
 y 102
 > > uhci3: interrupting at ioapic0 pin 20
 > > usb4 at uhci3: USB revision 1.0
 > > uhci4 at pci0 dev 16 function 1: Zhaoxin USB UHCI Controller (rev. 0x30=
 )
 > > allocated pic ioapic0 type level pin 22 level 6 to cpu0 slot 5 idt entr=
 y 100
 > > uhci4: interrupting at ioapic0 pin 22
 > > usb5 at uhci4: USB revision 1.0
 > > ehci1 at pci0 dev 16 function 7: Zhaoxin USB EHCI Controller (rev. 0x30=
 )
 > > ehci1: 32-bit DMA
 > > allocated pic ioapic0 type level pin 21 level 6 to cpu0 slot 6 idt entr=
 y 101
 > > ehci1: interrupting at ioapic0 pin 21
 > > ehci1: EHCI version 1.0
 > > ehci1: 2 companion controllers, 2 ports each: uhci3 uhci4
 > > ehci1: Using DMA subregion for control data structures
 > > usb6 at ehci1: USB revision 2.0
 > > pcib0 at pci0 dev 17 function 0: Zhaoxin ZX-D/ZX-E/KH-40000/KX-7000 Mis=
 cellaneous Bus (rev. 0x30)
 > > pchb8 at pci0 dev 17 function 7: Zhaoxin ZX-100 / KX-5000|6000(G)|7000 =
 / KH-40000 Standard Host Bridge (rev. 0x30)
 > > xhci0 at pci0 dev 18 function 0: Zhaoxin KX-6000(G)|7000 USB xHCI Contr=
 oller (rev. 0x30)
 > > xhci0: 64-bit DMA
 > > allocated pic msix2 type edge pin 0 level 6 to cpu0 slot 18 idt entry 1=
 03
 > > xhci0: interrupting at msix2 vec 0
 > > xhci0: xHCI version 1.10
 > > xhci0: hcs1=3D10000420 hcs2=3Dfc000031 hcs3=3De70004
 > > xhci0: hcc=3D0x205fe3<XECP=3D0x20,MAXPSA=3D0x5,CFC,SEC,SPC,PAE,NSS,LTC,=
 LHRC,BNC,AC64>
 > > xhci0: xECP 80
 > > xhci0: hcc2=3D0x7f<ETC,CIC,LEC,CTC,FSC,CMC,U3C>
 > > xhci0: ECR: 0x01000801
 > > xhci0: ECR: 0x02000802
 > > xhci0:  SP: 0x02000802 0x20425355 0x30180a01 0x00000000
 > > xhci0:  hs ports 1 - 10
 > > xhci0: ECR: 0x03109002
 > > xhci0:  SP: 0x03109002 0x20425355 0x2000060b 0x00000000
 > > xhci0:  ss ports 11 - 16
 > > xhci0: ECR: 0x0003000a
 > > xhci0: PAGESIZE 0x00000001
 > > xhci0: sc_pgsz 0x00001000
 > > xhci0: sc_maxslots 0x00000020
 > > xhci0: sc_maxports 16
 > > xhci0: sc_maxspbuf 31
 > > xhci0: sc_isthresh 1
 > > xhci0: eventst: 0x000000010f129fc0 0xffff96023715ffc0 1000
 > > xhci0: dcbaa: 0x000000010f12a000 0xffff960237160000 1000
 > > xhci0: current IMOD 0
 > > xhci0: USBCMD 0x00000005
 > > usb7 at xhci0: USB revision 3.1
 > > usb8 at xhci0: USB revision 2.0
 > > hdaudio1 at pci0 dev 20 function 0: HD Audio Controller
 > > allocated pic msi3 type edge pin 0 level 7 to cpu0 slot 19 idt entry 11=
 3
 > > hdaudio1: interrupting at msi3 vec 0
 > > hdaudio1: HDA ver. 1.0, OSS 4, ISS 4, BSS 0, SDO 1, 64-bit
 > > hdafg3 at hdaudio1 vendor 0x10EC product 0x0897 nid 0x01: Realtek ALC89=
 7
 > > hdafg3: duplicate pin in association
 > > hdafg3: DAC00 2ch: Speaker [Jack]
 > >     nid=3D14 [pin: Speaker (Green Jack)]
 > >            nid=3D0C [audio mixer] [source: dac, imonitor]
 > >                   nid=3D02 [audio output] [source: dac]
 > >                   nid=3D0B [audio mixer] [source: imonitor]
 > > hdafg3: ADC02 2ch: Mic In [Jack]
 > >     nid=3D18 [pin: Mic In (Pink Jack)]
 > > hdafg3: ADC03 2ch: Mic In [Jack]
 > >     nid=3D19 [pin: Mic In (Pink Jack)]
 > > hdafg3: 2ch/2ch 32000Hz 44100Hz 48000Hz 88200Hz 96000Hz 192000Hz PCM16 =
 PCM20 PCM24 AC3
 > > audio3 at hdafg3: playback, capture, full duplex, independent
 > > audio3: slinear_le:16 2ch 48000Hz, blk 1920 bytes (10ms) for playback
 > > audio3: slinear_le:16 2ch 48000Hz, blk 1920 bytes (10ms) for recording
 > > spkr4 at audio3: PC Speaker (synthesized)
 > > wsbell at spkr4 not configured
 > > pchb9 at pci0 dev 31 function 0: Zhaoxin product 3c00 (rev. 0x30)
 > > isa0 at pcib0
 > > pckbc0 at isa0 port 0x60-0x64
 > > pckbdprobe: reset error 5
 > > check_keyboard_by_id: getid failed with 5
 > > pmsprobe: reset error 5
 > > acpicpu0 at cpu0: ACPI CPU
 > > acpicpu0: C1: HLT, lat   0 us, pow     0 mW
 > > acpicpu0: P0: FFH, lat  10 us, pow  6800 mW, 3400 MHz
 > > acpicpu0: P1: FFH, lat  10 us, pow  6000 mW, 3000 MHz
 > > acpicpu0: P2: FFH, lat  10 us, pow  5800 mW, 2900 MHz
 > > acpicpu0: P3: FFH, lat  10 us, pow  5600 mW, 2800 MHz
 > > acpicpu0: P4: FFH, lat  10 us, pow  5400 mW, 2700 MHz
 > > acpicpu0: P5: FFH, lat  10 us, pow  5200 mW, 2600 MHz
 > > acpicpu0: P6: FFH, lat  10 us, pow  5000 mW, 2500 MHz
 > > acpicpu0: P7: FFH, lat  10 us, pow  4800 mW, 2400 MHz
 > > acpicpu0: P8: FFH, lat  10 us, pow  4600 mW, 2300 MHz
 > > acpicpu0: P9: FFH, lat  10 us, pow  4400 mW, 2200 MHz
 > > acpicpu0: P10: FFH, lat  10 us, pow  4200 mW, 2100 MHz
 > > acpicpu0: P11: FFH, lat  10 us, pow  4000 mW, 2000 MHz
 > > acpicpu0: P12: FFH, lat  10 us, pow  3800 mW, 1900 MHz
 > > acpicpu0: P13: FFH, lat  10 us, pow  3600 mW, 1800 MHz
 > > acpicpu0: P14: FFH, lat  10 us, pow  3000 mW, 1500 MHz
 > > acpicpu0: P15: FFH, lat  10 us, pow  2800 mW, 1400 MHz
 > > acpicpu0: P16: FFH, lat  10 us, pow  2600 mW, 1300 MHz
 > > acpicpu0: P17: FFH, lat  10 us, pow  2400 mW, 1200 MHz
 > > acpicpu0: P18: FFH, lat  10 us, pow  2200 mW, 1100 MHz
 > > acpicpu0: P19: FFH, lat  10 us, pow  2000 mW, 1000 MHz
 > > acpicpu0: P20: FFH, lat  10 us, pow  1800 mW,  900 MHz
 > > acpicpu0: P21: FFH, lat  10 us, pow  1600 mW,  800 MHz
 > > acpicpu0: T0: I/O, lat   1 us, pow     0 mW, 100 %
 > > acpicpu0: T1: I/O, lat   1 us, pow     0 mW,  94 %
 > > acpicpu0: T2: I/O, lat   1 us, pow     0 mW,  88 %
 > > acpicpu0: T3: I/O, lat   1 us, pow     0 mW,  82 %
 > > acpicpu0: T4: I/O, lat   1 us, pow     0 mW,  76 %
 > > acpicpu0: T5: I/O, lat   1 us, pow     0 mW,  70 %
 > > acpicpu0: T6: I/O, lat   1 us, pow     0 mW,  64 %
 > > acpicpu0: T7: I/O, lat   1 us, pow     0 mW,  58 %
 > > acpicpu0: T8: I/O, lat   1 us, pow     0 mW,  52 %
 > > acpicpu0: T9: I/O, lat   1 us, pow     0 mW,  46 %
 > > acpicpu0: T10: I/O, lat   1 us, pow     0 mW,  40 %
 > > acpicpu0: T11: I/O, lat   1 us, pow     0 mW,  34 %
 > > acpicpu0: T12: I/O, lat   1 us, pow     0 mW,  28 %
 > > acpicpu0: T13: I/O, lat   1 us, pow     0 mW,  22 %
 > > acpicpu0: T14: I/O, lat   1 us, pow     0 mW,  16 %
 > > acpicpu0: T15: I/O, lat   1 us, pow     0 mW,  10 %
 > > acpicpu0: id 0, lapic id 0, cap 0x08fd, flags 0x00186e27
 > > acpicpu0: P-state coordination: 1 CPUs, domain 0, type HW_ALL
 > > viac7temp0 at cpu0: VIA C7/Nano temperature sensor
 > > acpicpu1 at cpu1: ACPI CPU
 > > acpicpu1: id 1, lapic id 1, cap 0x08fd, flags 0x00186e27
 > > acpicpu1: P-state coordination: 1 CPUs, domain 1, type HW_ALL
 > > viac7temp1 at cpu1: VIA C7/Nano temperature sensor
 > > acpicpu2 at cpu2: ACPI CPU
 > > acpicpu2: id 2, lapic id 2, cap 0x08fd, flags 0x00186e27
 > > acpicpu2: P-state coordination: 1 CPUs, domain 2, type HW_ALL
 > > viac7temp2 at cpu2: VIA C7/Nano temperature sensor
 > > acpicpu3 at cpu3: ACPI CPU
 > > acpicpu3: id 3, lapic id 3, cap 0x08fd, flags 0x00186e27
 > > acpicpu3: P-state coordination: 1 CPUs, domain 3, type HW_ALL
 > > viac7temp3 at cpu3: VIA C7/Nano temperature sensor
 > > acpicpu4 at cpu4: ACPI CPU
 > > acpicpu4: id 4, lapic id 4, cap 0x08fd, flags 0x00186e27
 > > acpicpu4: P-state coordination: 1 CPUs, domain 4, type HW_ALL
 > > viac7temp4 at cpu4: VIA C7/Nano temperature sensor
 > > acpicpu5 at cpu5: ACPI CPU
 > > acpicpu5: id 5, lapic id 5, cap 0x08fd, flags 0x00186e27
 > > acpicpu5: P-state coordination: 1 CPUs, domain 5, type HW_ALL
 > > viac7temp5 at cpu5: VIA C7/Nano temperature sensor
 > > acpicpu6 at cpu6: ACPI CPU
 > > acpicpu6: id 6, lapic id 6, cap 0x08fd, flags 0x00186e27
 > > acpicpu6: P-state coordination: 1 CPUs, domain 6, type HW_ALL
 > > viac7temp6 at cpu6: VIA C7/Nano temperature sensor
 > > acpicpu7 at cpu7: ACPI CPU
 > > acpicpu7: id 7, lapic id 7, cap 0x08fd, flags 0x00186e27
 > > acpicpu7: P-state coordination: 1 CPUs, domain 7, type HW_ALL
 > > viac7temp7 at cpu7: VIA C7/Nano temperature sensor
 > > SRAT: 0 NUMA nodes
 > > Initializing SSP: 223f7ee6000eb8b9 7ec8fc1474fcb723 e0540b9c9a05a4f8 27=
 abe3fdbcdaaa88 6412b418b0616068 d55f757c751a5163 ec53644ecf3736d4 e5064529c=
 4606d9
 > > cpu0 has 1 core siblings: cpu0
 > > cpu0 has 1 pkg siblings: cpu0
 > > cpu0 has 8 1st siblings: cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 cpu0
 > > cpu0 first in package: cpu0
 > > cpu1 has 1 core siblings: cpu1
 > > cpu1 has 1 pkg siblings: cpu1
 > > cpu1 has 8 1st siblings: cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 cpu0 cpu1
 > > cpu1 first in package: cpu1
 > > cpu2 has 1 core siblings: cpu2
 > > cpu2 has 1 pkg siblings: cpu2
 > > cpu2 has 8 1st siblings: cpu3 cpu4 cpu5 cpu6 cpu7 cpu0 cpu1 cpu2
 > > cpu2 first in package: cpu2
 > > cpu3 has 1 core siblings: cpu3
 > > cpu3 has 1 pkg siblings: cpu3
 > > cpu3 has 8 1st siblings: cpu4 cpu5 cpu6 cpu7 cpu0 cpu1 cpu2 cpu3
 > > cpu3 first in package: cpu3
 > > cpu4 has 1 core siblings: cpu4
 > > cpu4 has 1 pkg siblings: cpu4
 > > cpu4 has 8 1st siblings: cpu5 cpu6 cpu7 cpu0 cpu1 cpu2 cpu3 cpu4
 > > cpu4 first in package: cpu4
 > > cpu5 has 1 core siblings: cpu5
 > > cpu5 has 1 pkg siblings: cpu5
 > > cpu5 has 8 1st siblings: cpu6 cpu7 cpu0 cpu1 cpu2 cpu3 cpu4 cpu5
 > > cpu5 first in package: cpu5
 > > cpu6 has 1 core siblings: cpu6
 > > cpu6 has 1 pkg siblings: cpu6
 > > cpu6 has 8 1st siblings: cpu7 cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6
 > > cpu6 first in package: cpu6
 > > cpu7 has 1 core siblings: cpu7
 > > cpu7 has 1 pkg siblings: cpu7
 > > cpu7 has 8 1st siblings: cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7
 > > cpu7 first in package: cpu7
 > > cpu0: TSC freq from HPET 2994060000 Hz
 > > cpu0: recalibrating local timer
 > > cpu0: apic clock running at 99 MHz
 > > timecounter: Timecounter "lapic" frequency 99802000 Hz quality -100
 > > timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
 > > UVM: using package allocation scheme, 1 package(s) per bucket
 > > cpu1: TSC skew=3D60 drift=3D-60
 > > cpu1: running
 > > cpu2: TSC skew=3D0 drift=3D0
 > > cpu2: running
 > > cpu3: running
 > > cpu3: TSC skew=3D0 drift=3D0
 > > cpu4: TSC skew=3D0 drift=3D60
 > > cpu4: running
 > > cpu5: TSC skew=3D60 drift=3D-60
 > > cpu5: running
 > > cpu6: running
 > > cpu6: TSC skew=3D0 drift=3D-60
 > > cpu7: TSC skew=3D0 drift=3D60
 > > cpu7: running
 > > TSC not known invariant on this CPU
 > > timecounter: Timecounter "TSC" frequency 2994060000 Hz quality -100
 > > uhub0 at usb7: NetBSD (0x0000) xHCI root hub (0x0000), class 9/0, rev 3=
 .00/1.00, addr 0
 > > uhub0: 6 ports with 6 removable, self powered
 > > uhub1 at usb8: NetBSD (0x0000) xHCI root hub (0x0000), class 9/0, rev 2=
 .00/1.00, addr 0
 > > uhub1: 10 ports with 10 removable, self powered
 > > acpicpu7: ACPI CPUs started
 > > uhub2 at usb1: NetBSD (0x0000) UHCI root hub (0x0000), class 9/0, rev 1=
 .00/1.00, addr 1
 > > uhub2: 2 ports with 2 removable, self powered
 > > uhub3 at usb3: NetBSD (0x0000) EHCI root hub (0x0000), class 9/0, rev 2=
 .00/1.00, addr 1
 > > uhub3: 6 ports with 6 removable, self powered
 > > uhub4 at usb5: NetBSD (0x0000) UHCI root hub (0x0000), class 9/0, rev 1=
 .00/1.00, addr 1
 > > uhub4: 2 ports with 2 removable, self powered
 > > uhub5 at usb2: NetBSD (0x0000) UHCI root hub (0x0000), class 9/0, rev 1=
 .00/1.00, addr 1
 > > uhub5: 2 ports with 2 removable, self powered
 > > uhub6 at usb6: NetBSD (0x0000) EHCI root hub (0x0000), class 9/0, rev 2=
 .00/1.00, addr 1
 > > uhub6: 4 ports with 4 removable, self powered
 > > uhub7 at usb4: NetBSD (0x0000) UHCI root hub (0x0000), class 9/0, rev 1=
 .00/1.00, addr 1
 > > uhub7: 2 ports with 2 removable, self powered
 > > uhub8 at usb0: NetBSD (0x0000) UHCI root hub (0x0000), class 9/0, rev 1=
 .00/1.00, addr 1
 > > uhub8: 2 ports with 2 removable, self powered
 > > IPsec: Initialized Security Association Processing.
 > > aes: Intel AES-NI
 > > chacha: x86 SSE2 ChaCha
 > > adiantum: self-test passed
 > > ehci1: handing over low speed device on port 2 to uhci3
 > > aes_ccm: self-test passed
 > > blake2s: self-test passed
 > > uhub6: port 2, device disappeared after reset
 > > umass0 at uhub0 port 2 configuration 1 interface 0
 > > umass0: USB (0x0781) SanDisk 3.2Gen1 (0x55a3), rev 3.00/1.00, addr 1
 > > umass0: using SCSI over Bulk-Only
 > > scsibus0 at umass0: 2 targets, 1 lun per target
 > > sd0 at scsibus0 target 0 lun 0: <USB, SanDisk 3.2Gen1, 1.00> disk remov=
 able
 > > sd0: 14664 MB, 29793 cyl, 16 head, 63 sec, 512 bytes/sect x 30031872 se=
 ctors
 > > sd0: GPT GUID: 52eca000-6e0f-11f1-8000-000000000000
 > > dk0 at sd0: "EFI system", 262144 blocks at 2048, type: msdos
 > > dk1 at sd0: "NBImgRoot", 4853760 blocks at 264192, type: ffs
 > > uhub9 at uhub3 port 4: vendor 05e3 (0x05e3) USB2.0 Hub (0x0608), class =
 9/0, rev 2.00/85.36, addr 2
 > > uhub9: single transaction translator
 > > uhub9: 4 ports with 4 removable, self powered
 > > waiting for devices: usb3 usb4 uhub3 uhub7 uhub9
 > > waiting for devices: usb3 usb4 uhub3 uhub7
 > > waiting for devices: usb4 uhub7
 > > uhidev0 at uhub7 port 2 configuration 1 interface 0
 > > uhidev0: CASUE (0x2a7a) CASUE USB Keyboard (0x9a18), rev 1.10/0.01, add=
 r 2, iclass 3/1
 > > ukbd0 at uhidev0
 > > wskbd0 at ukbd0 mux 1
 > > wskbd0: connecting to wsdisplay0
 > > waiting for devices: usb4 uhub7 uhidev0
 > > waiting for devices:uhidev1 at uhub7 usb4 port 2 uhub7 configuration 1
 > >  interface 1
 > > uhidev1: CASUE (0x2a7a) CASUE USB Keyboard (0x9a18), rev 1.10/0.01, add=
 r 2, iclass 3/0
 > > uhidev1: 2 report ids
 > > uhid0 at uhidev1 reportid 1: input=3D1, output=3D0, feature=3D0
 > > waiting for devices:uhid1 at uhidev1 usb4 reportid 2 uhub7 uhidev1
 > > : input=3D3, output=3D0, feature=3D0
 > > waiting for devices: usb4 uhub7
 > > waiting for devices: usb4
 > > crypto: assign driver 0, flags 2
 > > crypto: driver 0 registers alg 1 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 2 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 3 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 4 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 5 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 26 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 27 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 29 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 33 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 17 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 6 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 19 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 7 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 20 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 15 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 24 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 25 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 8 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 21 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 16 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 9 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 10 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 13 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 14 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 28 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 30 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 31 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 32 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 11 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 18 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 23 flags 0 maxoplen 0
 > > crypto: driver 0 registers alg 22 flags 0 maxoplen 0
 > > cgd: self-test aes-xts-256
 > > cgd: self-test aes-xts-512
 > > cgd: self-test aes-cbc-128
 > > cgd: self-test aes-cbc-256
 > > cgd: self-test 3des-cbc-192
 > > cgd: self-test blowfish-cbc-448
 > > cgd: self-test aes-cbc-128 (encblkno8)
 > > cgd: self-tests passed
 > > swwdog0: software watchdog initialized
 > > Searching for RAID components...
 > > WARNING: 2 errors while detecting hardware; check system log.
 > > boot device: sd0
 > > root on dk1
 > > root file system type: ffs
 > > kern.module.path=3D/stand/amd64/11.99.7/modules
 > > WARNING: no TOD clock present
 > > WARNING: using filesystem time
 > > WARNING: CHECK AND RESET THE DATE!
 > >
 > > >How-To-Repeat:
 > > Boot on KX-7000 CPU with cache associativity fix and observe the crash.
 > > >Fix:
 > > Disable SMAP as a workaround at the end of cpu_probe_c3():
 > >
 > >
 > > if (cpuid_level < 7)
 > >         return;
 > >
 > > /* XXX: Disable SMAP for now; it causes a page fault. */
 > > cpu_feature[5]  &=3D ~CPUID_SEF_SMAP;
 > > ci->ci_feat_val[5] &=3D ~CPUID_SEF_SMAP;
 > >
 > >
 >
 > Hi,
 >
 > So far, I've found two places where TLB entry invalidation (invlpg)
 > lets the system boot, after which it appears stable.
 > The first place is in pmap_install_ptp(), which helps with the
 > originally reported crash.
 >
 > The second one is in pmap_enter_ma(), where it helps avoid the crash at:
 > pmap_pte_cas(ptep, opte, npte) !=3D opte
 > (https://nxr.netbsd.org/xref/src/sys/arch/x86/x86/pmap.c?r=3D1.429#5183).
 >
 > In theory, for the second case, invalidating the TLB entry for some
 > unrelated address also seems to help. For example, adding:
 > pmap_update_pg((vaddr_t)pmap_enter_ma); at
 > https://nxr.netbsd.org/xref/src/sys/arch/x86/x86/pmap.c?r=3D1.429#5130
 > outside the loop also seems to work. However, this is harder to explain.
 > It probably invalidates more memory addresses or has some other side effe=
 ct.
 >
 > At present, I can't tell what is causing this. It may be related to
 > the recursive nature of memory access in the NetBSD implementation.
 > Potentially, FreeBSD and Linux use direct addresses instead.
 > I also don't know whether this is a CPU erratum, some implementation
 > quirk, or simply an existing issue that is more visible on this CPU.
 >
 > I am reluctant to apply the patch without fully answering the
 > questions above, but perhaps this could be treated as an erratum and
 > guarded by CPUID.
 > Alternatively, it could be guarded by SMAP support.
 > So far, AMD/Intel users don't seem to have reported or observed this
 > issue, or at least I haven't been able to find a similar PR.
 >
 > I also haven't tested this under heavier memory pressure yet.
 >
 > I would appreciate comments on the patch, as well as any ideas or
 > suggestions on how to proceed.
 >
 > Thanks!
 >
 > --- a/sys/arch/x86/x86/pmap.c
 > +++ b/sys/arch/x86/x86/pmap.c
 > @@ -2635,6 +2635,7 @@ pmap_install_ptp(struct pmap *pmap, struct
 > pmap_ptparray *pt, vaddr_t va,
 >                 ptp->wire_count =3D 1;
 >                 pmap->pm_ptphint[i - 2] =3D ptp;
 >                 pa =3D VM_PAGE_TO_PHYS(ptp);
 > +               pmap_update_pg((vaddr_t)&pva[index]);
 >                 pmap_pte_set(&pva[index], (pd_entry_t)
 >                     (pmap_pa2pte(pa) | PTE_U | PTE_W | PTE_P));
 >
 > @@ -5143,6 +5144,9 @@ pmap_enter_ma(struct pmap *pmap, vaddr_t va,
 > paddr_t ma, paddr_t pa,
 >                 if (((opte ^ npte) & (PTE_FRAME | PTE_P)) =3D=3D 0) {
 >                         npte |=3D opte & (PTE_A | PTE_D);
 >                 }
 > +
 > +               if ((opte ^ npte) & (PTE_FRAME | PTE_P))
 > +                       pmap_update_pg((vaddr_t)ptep);
 >  #if defined(XENPV)
 >                 if (domid !=3D DOMID_SELF) {
 >                         /* pmap_pte_cas with error handling */
 >
 > Regards,
 > Andrius V
 
 I have managed to narrow it down to the newly allocated PTPs only, so
 the current patch is more targeted
 and performs the extra TLB invalidations only when a new PTP is being insta=
 lled.
 
 It seems KX-7000 either requires additional invalidation when new PTEs
 are installed,
 or some previously cached state isn't properly invalidated.
 
 The patch should have a smaller performance impact, but it's not free eithe=
 r.
 
 diff --git a/sys/arch/x86/x86/pmap.c b/sys/arch/x86/x86/pmap.c
 index 780732256df..14deba21377 100644
 --- a/sys/arch/x86/x86/pmap.c
 +++ b/sys/arch/x86/x86/pmap.c
 @@ -2630,6 +2630,9 @@ pmap_install_ptp(struct pmap *pmap, struct
 pmap_ptparray *pt, vaddr_t va,
                         continue;
                 }
 
 +               if (i =3D=3D 2)
 +                       pmap_update_pg((vaddr_t)&pva[index]);
 +
                 ptp =3D pt->pg[i];
                 ptp->flags &=3D ~PG_BUSY; /* never busy */
                 ptp->wire_count =3D 1;
 @@ -5137,6 +5140,9 @@ pmap_enter_ma(struct pmap *pmap, vaddr_t va,
 paddr_t ma, paddr_t pa,
         do {
                 opte =3D *ptep;
 
 +               if (getptp)
 +                       pmap_update_pg((vaddr_t)ptep);
 +
                 /*
                  * if the same page, inherit PTE_A and PTE_D.
                  */
 



Home | Main Index | Thread Index | Old Index