bin/55892: npf cannot handle large tables

[technet%netdog.org@localhost]

>Number:         55892
>Category:       bin
>Synopsis:       npf cannot handle large tables
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 22 13:40:00 +0000 2020
>Originator:     Hector
>Release:        NetBSD 9.1
>Organization:
>Environment:
NetBSD apu4ed.home.lan 9.1 NetBSD 9.1 (GENERIC) #0: Sun Oct 18 19:24:30 UTC 2020  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
On attempting to load a npf(7) ruleset which references a table with tens of thousands of entries, npfctl(8) silently runs for a very long time, and then emits some garbage output. At this point, the npf is left in an operable state.

The failure to properly load the ruleset is one bad behaviour.

That loading a ruleset takes minutes is another bad behavior.
>How-To-Repeat:
Here you can download a minimal npf.conf which tries to load a table of about 52,000 subnets.

http://lab.netdog.org/npf.conf

http://lab.netdog.org/ip-blacklist-52k.gz

On a 4-core machine with 4GB of memory, this command:

 # npfctl reload

chewed in silence for about 7 minutes, and then produced this output:

  npfctl: �8

With a larger table, the run time is longer, and the garbage output is different, being longer.
>Fix: