pkg/30929: bin-install foiled by cached package tgz's

[nbgnats%anastigmatix.net@localhost]

>Number:         30929
>Category:       pkg
>Synopsis:       bin-install foiled by cached package tgz's
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 06 22:36:00 +0000 2005
>Originator:     Chapman Flack
>Release:        2.0.0 (irrelevant to issue)
>Organization:
>Environment:
NetBSD lundestad.anastigmatix.net 2.0 NetBSD 2.0 (lundestad) #11: Sat Mar  5 
14:01:49 EST 2005  xxx@xxx:/usr/src/sys/arch/i386/compile/lundestad i386 
(irrelevant to issue: relevant code is the same in CVS head)
>Description:
When bin-install tries to add a binary package, it uses pkg_add to do it.
pkg_add will naturally add, if it can, any other binary packages the package
depends on. pkg_add has the ability to search for such packages on a path
of repository sites.

However, real-su-bin-install thwarts that capacity. It does its own search
along a path constructed from BINPKG_SITES, and in each step of the search
it calls pkg_add with a PKG_PATH of only the single site currently being
searched. This guarantees failure if the desired package is at that site
but some of its depends are at a different site on the path. The most
annoying case arises when a package tgz is cached in packages/All on the current
machine, usually because it was built from source at some prior expense of
time and cycles, but its depends are not there because they were available
in binary so they were merely installed, and you want to reinstall. What you
would want is for the local cached binary package to be installed while fetching
and installing again the available binary packages it depends on, but instead
the pkg_add fails, leading to an unwanted rebuild of the locally-built package.

Solution: simply have real-su-bin-install construct a PKG_PATH from the local
repository and all elements of BINPKG_SITES, and use that path in a single
request of pkg_add to install the package. Patch attached.

Note: this issue was also dicussed in
http://mail-index.netbsd.org/tech-pkg/2005/03/20/0023.html and
http://mail-index.netbsd.org/tech-pkg/2005/03/20/0025.html, with a patch that
combined a few issues and had been tested at that time. The patch attached here
was derived by isolating this issue from the original patch and applying it to
the current CVS head, but without thorough retesting. Nothing I noticed in the
CVS head /looks/ like it should break the patch....

>How-To-Repeat:
make bin-install for a package foo where foo has been built previously and 
foo.tgz is in PKGREPOSITORY on the local machine, but foo requires some package 
bar that is not installed, not in PKGREPOSITORY, but is available as a binary 
package from BINPKG_SITES.
>Fix:
--- bsd.pkg.mk  2005-08-06 16:45:08.000000000 -0500
+++ bsd.pkg.mk  2005-08-06 17:06:37.000000000 -0500
@@ -3474,6 +3474,9 @@
 _BIN_INSTALL_FLAGS=    ${BIN_INSTALL_FLAGS}
 _BIN_INSTALL_FLAGS+=   ${PKG_ARGS_ADD}
 
+_SHORT_UNAME_R!=       uname -r
+_SHORT_UNAME_R:=       ${_SHORT_UNAME_R:C@\.([0-9])*[_.].*@.\1@} # 
n.n[_.]anything => n.n
+
 # Install binary pkg, without strict uptodate-check first
 .PHONY: real-su-bin-install
 real-su-bin-install:
@@ -3485,27 +3488,20 @@
                ${SHCOMMENT} ${ECHO_MSG} "*** or use \`\`${MAKE} bin-update'' 
to upgrade it and all of its dependencies."; \
                exit 1;                                                 \
        fi
-       @if [ -f ${PKGFILE} ] ; then                                    \
-               ${ECHO_MSG} "Installing from binary pkg ${PKGFILE}" ;   \
-               ${PKG_ADD} ${_BIN_INSTALL_FLAGS} ${PKGFILE} ;           \
-       else                                                            \
-               rel=`${UNAME} -r | ${SED} 's@\.\([0-9]*\)[\._].*@\.\1@'`; \
-               arch=${MACHINE_ARCH};                                   \
-               for site in ${BINPKG_SITES} ; do                        \
-                       ${ECHO} Trying `eval ${ECHO} $$site`/All ;      \
-                       ${SHCOMMENT} ${ECHO} ${SETENV} PKG_PATH="`eval ${ECHO} 
$$site`/All" ${PKG_ADD} ${_BIN_INSTALL_FLAGS} ${PKGNAME}${PKG_SUFX} ; \
-                       if ${SETENV} PKG_PATH="`eval ${ECHO} $$site`/All" 
${PKG_ADD} ${BIN_INSTALL_FLAGS} ${PKGNAME}${PKG_SUFX} ; then \
-                               ${ECHO} "${PKGNAME} successfully installed."; \
-                               break ;                                 \
-                       fi ;                                            \
-               done ;                                                  \
-               if ! ${PKG_INFO} -qe "${PKGNAME}" ; then                \
-                       ${SHCOMMENT} Cycle through some FTP server here ;\
-                       ${ECHO_MSG} "Installing from source" ;          \
-                       ${MAKE} ${MAKEFLAGS} package                    \
-                               DEPENDS_TARGET=${DEPENDS_TARGET:Q} &&   \
-                       ${MAKE} ${MAKEFLAGS} clean ;                    \
-               fi ; \
+       @rel=${_SHORT_UNAME_R:Q} ; \
+       arch=${MACHINE_ARCH:Q} ; \
+       pkgpath=${PKGREPOSITORY:Q} ; \
+       for i in ${BINPKG_SITES} ; do pkgpath="$$pkgpath;$$i/All" ; done ; \
+       ${ECHO} "Trying $$pkgpath" ;    \
+       if ${SETENV} PKG_PATH="$$pkgpath" ${PKG_ADD} ${BIN_INSTALL_FLAGS} 
${PKGNAME}${PKG_SUFX} ; then \
+               ${ECHO} ${PKGNAME} successfully installed.; \
+               break ;                                 \
+       else                                                            \
+               ${SHCOMMENT} Cycle through some FTP server here ;\
+               ${ECHO_MSG} "Installing from source" ;          \
+               ${MAKE} ${MAKEFLAGS} package                    \
+                       DEPENDS_TARGET=${DEPENDS_TARGET:Q} &&   \
+               ${MAKE} ${MAKEFLAGS} clean ;                    \
        fi
 
 .PHONY: bin-install