Re: misc/50216: USB, PCI ID data grow quadratically in the repository

[christos%zoulas.com@localhost (Christos Zoulas)]

On Oct 27, 11:55am, kre%munnari.OZ.AU@localhost (Robert Elz) wrote:
-- Subject: Re: misc/50216: USB, PCI ID data grow quadratically in the reposi

|  and I understand that.   I see two ways that could be resolved.  First,
|  the files (as well as being used in the build) could be installed
|  somewhere (not in an include directory, somewhere like /usr/share/misc)
|  so they'd be available ... but that means only after the system has been
|  built and installed, so is probably not the solution that will satisfy.
|  
|  Or second, there are two files generated (for each *devs file).  That is,
|  for pcidevs we get pcidevs.h and pcidevs_data.h
|  
|  The first one I don't think is too much of a problem (and a little tweaking
|  might be able to reduce the churn in that one even more.)
|  
|  The _data.h file is the one that really causes the problem, and frankly
|  I can't see anyone wanting to grep that (or even keep it visible more than
|  a few seconds so it can be included in whatever #includes it.)  If
|  for some reason you do, you can always generate it - obviously a method
|  to do that is going to be needed.
|  
|  So, for now the solution I am going to suggest, at least as a first
|  quick fix, is that we change the procedure to only generate and check in
|  the namedevs.h files when namedevs is altered, and we fix the Makefiles
|  to generate namedevs_data.h on the fly during compilation (in the obj
|  directories).
|  
|  Then we cvs delete the namedevs_data.h files, and arrange with the admins to
|  make them all read only in the repo, so there can be no more commits to them,
|  ever, from any release - any of the older releases that wants to change the
|  device lists will need to adopt the new procedure first. Then after NetBSD-8
|  has faded from memory (years from now) we simply nuke them.
|  
|  I mean rm namedevs_data.h,v so they are simply gone!  Anyone wanting to
|  checkout an older source tree after that will need to regenerate the files
|  (we could make a script for that to make life easier) before building, using
|  the appropriate devlist2h.awk scripts that match the distribution in question.
|  
|  I think this is a much better solution than looking for a more stable
|  compression algorithm for the _data.h files (and then having to debug
|  the code to extract data from it) where what we have now seems to work
|  well enough ... but doing this would also make it much easier for someone
|  with the desire to create better algorithms, and experiment.
|  
|  Any objections to this strategy?   [The nuking the files from the repo
|  part is kind of an optional extra, just to reduce the repo size, it is
|  not crucial to the rest of the plan.]
|  
|  If not, I'll start the process making all of this happen.   I'll also look
|  at why we seem to need 15 devlist2h.awk files in the tree.  That seems
|  a little excessive, I find it hard to believe that they're all so
|  different from each other that every single one of them is needed.

I am happy with the plan to move the foo_data.h file to be
auto-generated during the build, and cvs delete it from head.
Perhaps it should build both files (foo.h foo_data.h) int the obj
directory and check that the committed foo.h matches the newly
generated one.

I would not do anything at the repo level until that stabilizes;
any read-only decision etc, can be deferred.

As for merging them further, by all means. I tried and merged some
of them a while ago, the problem is that they are not all the same
:-)

christos