Re: Running makemandb on every boot

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Mon, 19 May 2014 20:26:37 +0200
    From:        Joerg Sonnenberger <joerg%britannica.bec.de@localhost>
    Message-ID:  <20140519182637.GA12611%britannica.bec.de@localhost>

  | But this is going to stat(2) every file, so it doesn't really change
  | anything fundamentally.

It is the difference between "Doesn't work," and "Works, but could be even
more efficient", which is kind of fundamental.

Further, the change (as measured and reported) was a speedup of a factor of
70 in elapsed time, and 350 in user time ... that kind of thing is worth
having, regardless of whether it could be better, and that there is probably
some other problem somewhere contributing to much of the saved time.

  | As I said, the check for existing entries is way
  | too slow right now, so that should be fixed.

Agreed.   It should.

  | Everything else is just distraction until that happened.

No, it's not.   Improvements are improvements, whether they achieve
perfection or not, and they're all worth having.  Making this change would
not (in any way) prevent anyone from doing something better - but while
waiting for that to happen, things are improved.

Anything notably different that this (aside from fixing whatever is
apparently making sparc sqlite startup slow - I haven't touched a sparc
in over a decade, so I can't comment on that one) is likely to involve
changes in places that don't necessarily have a clear cut one right way.

Eg: one way to vastly speed this would be to have man page installation
(from the makefiles that do it) touch a flag file - if that file is missing
(as it would be after a new install, or if something happens to it) or if
it is newer than the database, then rebuild, otherwise exit - all with
three stats in a

        if [ ! -e ${FLAG_FILE} ] || [ ${FLAG_FILE} -nt ${MAN_DB} ]
        then
                touch ${FLAG_FILE}-new
                rebuild... && mv ${FLAG_FILE}-new ${FLAG_FILE}
        fi

kind of thing.  Even better would be to list the changed man pages in the file,
and rebuild the database entries for just those man pages (again, rebuilding
all if the file is missing).

There's lots that could be done - but until someone does one of them,
just avoiding the sqlite init until it is determined that somethig needs
to be updated seems like a very sane idea to me.

kre

ps: while playing in this area, it would be nice to have the daily
rebuild (for as much as it remains happening) stop cluttering the daily
report with noise like ...

Updating man page index:
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
XTestQueryExtension.3: No such file or directory
afGetFrameCount.3: No such file or directory
afGetFrameCount.3: No such file or directory
afInitAESChannelDataTo.3: No such file or directory
afInitSampleFormat.3: No such file or directory
[.... more lines elided ...]

I get that noise mailed at me every day, and really, I just don't care!