misc/50216: USB, PCI ID data grow quadratically in the repository

[gson%gson.org@localhost (Andreas Gustafsson)]

>Number:         50216
>Category:       misc
>Synopsis:       USB, PCI ID data grow quadratically in the repository
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Sep 07 13:30:00 +0000 2015
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current, source date 2015.01.26.10.53.21
>Organization:

>Environment:
System: NetBSD
Architecture: x86_64
Machine: amd64
>Description:

A while ago, I added a new device to src/sys/dev/usb/usbdevs
and regenerated usbdevs.h and usbdevs_data.h.

When I did this, I noticed that the addition of a single line to
usbdevs caused a large number of lines to change in usbdevs_data.h.

It looks that the number of lines in usb_data.h that change when
adding a new device is typically on the order of a few thousand, and I
suspect it is actually proportional to the number of devices that
already exist, on average.  This is a problem, because as new USB
devices are added one by one, the space used by usbdevs_data.h in the
repository will grow proportionally to the *square* of the total
number of devices.  The PCI device data also appears to have the same
problem.

As of today, the size of usbdevs_data.h,v in the repository is 3.2
megabytes, and pcidevs_data.h,v is 57 megabytes.  But it's not the
current size that worries me as much as what will happen over time if
we keep adding new devices and this results in the the repository
files growing not just at a constant rate, but at an *accelerating*
rate.

>How-To-Repeat:

  cd src/sys/dev/pci
  cvs log pcidevs_data.h | grep 'lines:' 

Notice how thousands of lines change in each commit.

>Fix:

Revert the compression done in devlist2h.awk, or replace it with a
different compression algorithm where a small change in the input only
results in a small change in the output.