Re: bin/54730: postinstall(8) fix doesn't remove obsolete /etc/rc.d files

[Christos Zoulas <christos%zoulas.com@localhost>]

I don't think that's there anything wrong with this change. The problem is
that there is no information about obsolete files in /etc/mtree so you need
to use source sets. 

$ cat x
#!/bin/sh
# does not work
#etcsets=/etc/mtree/set.etc
etcsets=/usr/src/distrib/sets/lists/etc/mi
SED=sed

select_obsolete_files()
{
        ${SED} -n -e "/obsolete/s@\.$1$2[[:space:]].*@\1@p" $3
}

# obsolete rc file in /etc/rc.d
rc_obsolete_files="$(select_obsolete_files /etc/rc.d/ \
    "\([^[:space:]]*\)" ${etcsets})"

echo ${rc_obsolete_files}
$./x 
NETWORK btattach btconfig btcontrol btdevctl bthcid btuartd dhclient fixsb fsck.sh gated isdnd kerberos lkm1 lkm2 lkm3 poffd rtsold sdpd systemfs xntpd ypset


christos
> On Dec 1, 2019, at 11:55 AM, Izumi Tsutsui <tsutsui%ceres.dti.ne.jp@localhost> wrote:
> 
>> Number:         54730
>> Category:       bin
>> Synopsis:       postinstall(8) fix doesn't remove obsolete /etc/rc.d files
>> Confidential:   no
>> Severity:       non-critical
>> Priority:       high
>> Responsible:    bin-bug-people
>> State:          open
>> Class:          sw-bug
>> Submitter-Id:   net
>> Arrival-Date:   Sun Dec 01 16:55:00 +0000 2019
>> Originator:     Izumi Tsutsui
>> Release:        NetBSD 9.0_RC1
>> Organization:
>> Environment:
> System: NetBSD evbarm 9.0_RC1 (HPT5325)
> Architecture: earm (but all ports, or all arm ports?)
> Machine: evbarm (but all?)
> 
>> Description:
> It looks postinstall(8) fix command on 9.0_RC1 with the etc.tgz set
> doesn't remove at least obsolete /etc/rc.d/isdnd, which it's marked
> as obsolete in src/distrib/sets/lists/etc/mi and /var/db/obsolete/etc.
> 
> As a result, /etc/rc complains on boot:
> ---
> /etc/rc: WARNING: $dhclient is not set properly - see rc.conf(5).
> :
> /etc/rc: WARNING: $isdnd is not set properly - see rc.conf(5).
> ---
> even after postinstall-fix.
> 
>> How-To-Repeat:
> 
> # ftp -o /tmp/etc.tgz http://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0_RC1/evbarm-earm/binary/sets/etc.tgz
> Trying [2a04:4e42:15::262]:80 ...
> ftp: Can't connect to `2a04:4e42:15::262:80': No route to host
> Trying 151.101.89.6:80 ...
> Requesting http://cdn.netbsd.org/pub/NetBSD/NetBSD-9.0_RC1/evbarm-earm/binary/sets/etc.tgz
> 100% |***********************************|   642 KiB    3.49 MiB/s    00:00 ETA
> 657924 bytes retrieved in 00:00 (3.48 MiB/s)
> # postinstall -s /tmp/etc.tgz fix
> Note: Creating temporary directory /tmp/_postinstall.2798.0/etc.tgz
> Note: Extracting files from /tmp/etc.tgz
> Source directory: /tmp/_postinstall.2798.0/etc.tgz
> (extracted from: /tmp/etc.tgz)
> Target directory: /
> bluetooth fix:
> ddbonpanic fix:
> defaults fix:
> 	(Checking for pf.boot.conf from /tmp/_postinstall.2798.0/etc.tgz/etc/defaults instead of /tmp/_postinstall.2798.0/etc.tgz/usr.sbin/pf/etc/defaults)
> dhcpcd fix:
> 	(Checking for dhcpcd.conf from /tmp/_postinstall.2798.0/etc.tgz/etc instead of /tmp/_postinstall.2798.0/etc.tgz/external/bsd/dhcpcd/dist/src)
> dhcpcdrundir fix:
> envsys fix:
> fontconfig fix:
> 	/tmp/_postinstall.2798.0/etc.tgz/etc/fonts/conf.avail is not a directory; skipping check
> gid fix:
> 	Error groups (FIX MANUALLY): _rtadvd (missing) _gpio (missing) _nsd (missing) _unbound (missing) nvmm (missing)
> 	Use the following as a template:
> _rtadvd:*:30:
> _gpio:*:29:
> _nsd:*:33:
> _unbound:*:32:
> nvmm:*:34:root
> 	and adjust if necessary.
> gpio fix:
> hosts fix:
> iscsi fix:
> makedev fix:
> 	(Checking for MAKEDEV from /tmp/_postinstall.2798.0/etc.tgz/dev instead of /tmp/_postinstall.2798.0)
> 	(Checking for MAKEDEV.local from /tmp/_postinstall.2798.0/etc.tgz/dev instead of /tmp/_postinstall.2798.0/etc.tgz/etc)
> motd fix:
> mtree fix:
> named fix:
> pam fix:
> periodic fix:
> pf fix:
> 	(Checking for pf.os from /tmp/_postinstall.2798.0/etc.tgz/etc instead of /tmp/_postinstall.2798.0/etc.tgz/dist/pf/etc)
> pwd_mkdb fix:
> rc fix:
> 	(Checking for blacklistd from /tmp/_postinstall.2798.0/etc.tgz/etc/rc.d instead of /tmp/_postinstall.2798.0/etc.tgz/external/bsd/blacklist/etc/rc.d)
> ssh fix:
> 	(Checking for moduli from /tmp/_postinstall.2798.0/etc.tgz/etc instead of /tmp/_postinstall.2798.0/etc.tgz/crypto/external/bsd/openssh/dist)
> wscons fix:
> x11 fix:
> xkb fix:
> uid fix:
> 	Error users (FIX MANUALLY): _rtadvd (missing) _nsd (missing) _unbound (missing)
> 	Use the following as a template:
> _rtadvd:*:30:30::0:0:& pseudo-user:/var/chroot/rtadvd:/sbin/nologin
> _nsd:*:33:33::0:0:& pseudo-user:/var/chroot/nsd:/sbin/nologin
> _unbound:*:32:32::0:0:& pseudo-user:/var/chroot/unbound:/sbin/nologin
> 	and adjust if necessary.
> varrwho fix:
> tcpdumpchroot fix:
> atf fix:
> 	(Checking for NetBSD.conf from /tmp/_postinstall.2798.0/etc.tgz/etc/atf instead of /tmp/_postinstall.2798.0/etc.tgz/external/bsd/atf/etc/atf)
> 	(Checking for atf-run.hooks from /tmp/_postinstall.2798.0/etc.tgz/etc/atf instead of /tmp/_postinstall.2798.0/etc.tgz/external/bsd/atf/dist/tools/sample)
> catpages fix:
> manconf fix:
> ptyfsoldnodes fix:
> varshm fix:
> obsolete fix:
> eabi
> eabihf
> oabi
> postinstall fixes passed: bluetooth ddbonpanic defaults dhcpcd dhcpcdrundir envsys fontconfig gpio hosts iscsi makedev motd mtree named pam periodic pf pwd_mkdb rc ssh wscons x11 xkb varrwho tcpdumpchroot atf catpages manconf ptyfsoldnodes varshm obsolete
> postinstall fixes failed: gid uid
> # ls -l /etc/rc.d/isdnd
> -r-xr-xr-x  1 root  wheel  472 Apr 13  2014 /etc/rc.d/isdnd
> # cd /tmp
> # tar zxf etc.tgz
> # grep -R isdnd var
> var/db/obsolete/etc:./etc/rc.d/isdnd
> # 
> 
>> Fix:
> Something wrong in this change?
> http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/postinstall/postinstall.in.diff?r1=1.3&r2=1.4
> 
> --- src/usr.sbin/postinstall/postinstall.in	2019/06/14 14:17:45	1.3
> +++ src/usr.sbin/postinstall/postinstall.in	2019/06/14 14:39:49	1.4
> @@ -1422,6 +1428,15 @@ select_set_files()
> 	    -e "/^\.${qdir}/s@^.$2[[:space:]].*@\1@p" $3
> }
> 
> +# select obsolete files in a sets file
> +# $1: directory pattern
> +# $2: file pattern
> +# $3: filename
> +select_obsolete_files()
> +{
> +	${SED} -n -e "/obsolete/s@\.$1$2[[:space:]].*@\1@p" $3
> +}
> +
> exclude()
> {
> 	if [ -z "$*" ]; then
> @@ -1465,9 +1480,8 @@ do_rc()
> 	    exclude ${rc_external_files})"
> 
> 	# obsolete rc file in /etc/rc.d
> -	local rc_obsolete_files="$(${SED} -n \
> -	    -e '/obsolete/s@./etc/rc.d/\([^[:space:]]*\)[[:space:]].*@\1@p' \
> -	    ${etcsets})"
> +	local rc_obsolete_files="$(select_obsolete_files /etc/rc.d/ \
> +	    "\([^[:space:]]*\)" ${etcsets})"
> 
> 	compare_dir "${op}" "${SRC_DIR}/etc" "${DEST_DIR}/etc" 644 \
> 		${rc_644_files}