Re: repo missing package

[Greg Troxel <gdt%lexort.com@localhost>]

Patrick Welche <prlw1%cam.ac.uk@localhost> writes:

> On Wed, Jan 08, 2020 at 06:38:16PM -0500, Greg Troxel wrote:
>> I was able to build rust and firefox on 2019Q4.
>
> Did it just work, or did you keep having to restart it? (PR pkg/54795)
> (and gpgme always hangs for me in pbulk, but always works with cd gpgme && make)

I am pretty sure it just worked, but I was not paying attention.  What I
am doing is:

    hardware is a 2008 MacBookPro3,1 (4G RAM, 232G spinning disk)
    system is netbsd-8 amd64 (up to date, not 8.0)
    pkgsrc is checked out and symlinked into /usr/pkgsrc
    I use ccache
    MAKE_JOBS=1 (because I'd rather wait than have heat)

  loop:

    I update pkgsrc (was head, and this was during freeze, and at
    branching I have flipped the box to Q4 with -r).

    I rm all the work directories.

    I run "pkg_rolling-replace -uvk", and on later runs when I get
    serious about fixing things (vs getting a handle on the state of the
    branch), I omit the -k.

    I look at what failed, try to fix them, often committing something,
    and often just rm'ing some packages and rebuilding because there is
    some problem that causes make replace trouble but not a pbulk build
    (scons version conflict, files moving between packages, etc.)

    goto loop

I have a function to list package installation times (which for pkg_rr
correspond to build times):

  pkgwhen () 
  { 
      ( cd /var/db/pkg && lt */+CONTENTS | sed -e 's/\/+CONTENTS//' )
  }
  lt () 
  { 
      ls -ltr "$@"
  }

Here is a fragment of the output of pkgwhen.  You can see that rust took an
enormously long time to build, ~17h.  firefox took ~10h.

I am 99.9% sure I didn't touch
the box during this time, other than perhaps to tail the pkg_rr output
or run pkgwhen  to see how it was doing.


-rw-r--r--  1 root  wheel    93776 Dec 18 18:32 x11-links-1.31
-rw-r--r--  1 root  wheel      390 Dec 18 18:33 ccache-3.7.6
-rw-r--r--  1 root  wheel      905 Dec 18 18:36 sqlite3-3.30.1
[snip]
-rw-r--r--  1 root  wheel      857 Dec 20 12:02 erlang-iconv-1.0.8
-rw-r--r--  1 root  wheel      955 Dec 20 12:03 erlang-fast_yaml-1.0.15
-rw-r--r--  1 root  wheel     1086 Dec 20 12:03 erlang-fast_tls-1.0.23
-rw-r--r--  1 root  wheel     1071 Dec 20 12:04 erlang-eimp-1.0.6
-rw-r--r--  1 root  wheel  2134552 Dec 21 05:14 rust-1.39.0nb2
-rw-r--r--  1 root  wheel      918 Dec 21 06:23 erlang-stringprep-1.0.12
-rw-r--r--  1 root  wheel     1814 Dec 21 06:23 erlang-fast_xml-1.1.32
-rw-r--r--  1 root  wheel    10324 Dec 21 06:24 erlang-xmpp-1.2.2
-rw-r--r--  1 root  wheel     8396 Dec 21 09:26 py27-qwt-qt4-5.2.0nb4
-rw-r--r--  1 root  wheel   275794 Dec 21 09:28 py27-twisted-18.9.0nb1
-rw-r--r--  1 root  wheel    40064 Dec 21 09:29 py27-nevow-0.14.4
-rw-r--r--  1 root  wheel      481 Dec 21 09:30 tex-pdfcrop-1.37nb4
-rw-r--r--  1 root  wheel    33067 Dec 21 09:44 py27-foolscap-0.13.1
-rw-r--r--  1 root  wheel     1224 Dec 21 09:46 py37-libxml2-2.9.10
-rw-r--r--  1 root  wheel      946 Dec 21 09:46 itstool-2.0.6nb1
-rw-r--r--  1 root  wheel    10461 Dec 21 09:47 mate-polkit-1.22.0nb2
-rw-r--r--  1 root  wheel   381495 Dec 21 09:59 gnome-themes-standard-3.20.2nb9
-rw-r--r--  1 root  wheel     8004 Dec 21 10:02 xfce4-xarchiver-0.5.4.14
-rw-r--r--  1 root  wheel   462230 Dec 21 10:04 xfce4-wm-themes-4.10.0nb12
-rw-r--r--  1 root  wheel      375 Dec 21 10:54 cbindgen-0.10.1
-rw-r--r--  1 root  wheel   700456 Dec 21 21:45 firefox-71.0
-rw-r--r--  1 root  wheel     5357 Dec 21 21:49 libcanberra-0.30nb1
[snip]
-rw-r--r--  1 root  wheel     8106 Dec 22 01:40 xfce4-desktop-4.14.1
-rw-r--r--  1 root  wheel    11073 Dec 22 01:44 py37-anytree-2.7.2
-rw-r--r--  1 root  wheel    13438 Dec 22 01:48 py37-lxml-4.4.2
-rw-r--r--  1 root  wheel    77281 Dec 22 01:49 py37-pygments-2.4.2
-rw-r--r--  1 root  wheel    23177 Dec 22 01:54 tex-oberdiek-2019

and then the timestamps jump making it clear there were later runs.


I did an update (on the branch) last night, and am doing a fresh pkg_rr
run.  After that's done I can do a 'make package' in rust and then in
firefox.

On a similar machine (but a 2006 non-pro macbook, on which I am running
netbsd-8 i386), rust and firefox also built ok.

So maybe I've been lucky.  Maybe it's MAKE_JOBS.  Maybe it's something
else we don't understand.