Subject: Re: mergemaster
To: Martti Kuparinen <martti.kuparinen@iki.fi>
From: Greg A. Woods <woods@weird.com>
List: current-users
Date: 11/14/2001 13:42:15
--iY4F4jJ/dQ
Content-Type: text/plain; charset=us-ascii
Content-Description: message body and .signature
Content-Transfer-Encoding: 7bit

[ On Wednesday, November 14, 2001 at 11:52:57 (+0200), Martti Kuparinen wrote: ]
> Subject: mergemaster
>
> I've written a FreeBSD mergemaster-like utility to compare, merge and
> install files into /etc (from /usr/src/etc). Yes, I've read the previous
> "wars" on this subject but I needed a tool to do the config file update...
> 
> This script does NOT check for any CVS tags but runs cmp to find out
> if the file has been updated in /usr/src/etc or not.

I've been using the attached script, which was derived from the original
FreeBSD script that came before mergemeaster, for quite some time now.

It does use the CVS tags, but then again it is really only useful for
upgrades from source anyway.

To really properly help automate upgrades in general one needs to use a
technique similar to the CVS vendor-branch management -- i.e. where at
least the ancestor revision of each "vendor" release is stored and used
either to find the set of changes to the vendor file since the last
release.  The only safe way to best ensure that vendor changes are not
accidentally undone by a merge is to always only merge new changes from
the vendor files into the live files (and then allow the system manager
to modify them, of course).  One could use CVS to do this, but I think
it's easier and safer to use either plain RCS, or even just the
/var/backups mechanism with a ".orig" file to compliment the .backup and
.current files, especially in this case where the files are mostly
usualy owned by root, and also because this way the system manager isn't
forced to use a version control tool for all changes -- only for
upgrades.  Even permissions changes can be tracked sufficiently when you
use just plain files (unlike with CVS, or even RCS).

In fact I've already modified my own binary release installation
procedure to immediately copy all of the files listed in /etc/changelist
to their releative locations under /var/backups, but without any
extension on the filename (i.e. instead of using ".orig").  I've not yet
done an upgrade since using this new procedure though, so I've not yet
written the equivalent of a "mergemaster" script to assist (though emacs
ediff mode would do the trick mostly by itself!  :-).

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>     <woods@robohack.ca>
Planix, Inc. <woods@planix.com>;   Secrets of the Weird <woods@weird.com>


--iY4F4jJ/dQ
Content-Type: text/plain
Content-Description: Simple script to assist in merging /etc files during OS upgrades from /usr/src
Content-Disposition: inline;
	filename="etcupd.sh"
Content-Transfer-Encoding: 7bit

#! /bin/sh
:
#ident	"$Name:  $:$Id: etcupd.sh,v 1.2 1999/01/12 22:41:06 woods Exp $"
#
# etcupd - Compare /etc with /usr/src/etc to find files needing updates
# 
# This script compares all files found in the etc directory of the
# source distribution (/usr/src/etc by default) with those in /etc to
# alert you when /etc files need updating.
#
# The ouput of 'ident' is used to determine if the source version has a
# different revision ID, and if so the file is reported with the phrase
# "NEEDS MERGE", otherwise the file is simply noted as "DIFFERENT".
# 
# Inclusion and exclusion patterns must be an egrep pattern which is a
# list of file names separated by the pipe character.  The filenames
# must be relative to the etc directory of the source distribution
# (/usr/src/etc by default).
# 
# By default the script will use 'egrep -x' which means the patterns
# must exactly match for the files to be included or excluded.  This
# is generally what you want as you probably want to be able to type
# "etcud -e hosts" to exclude the file /etc/hosts but not the file
# /etc/hosts.lpd.  Power users can use -n to disable the use of -x
# with egrep.  This can be useful when dealing with the ppp directory
# for example.
#
# NOTES:
#       
#    -  You can use EITHER an exclusion OR an inclusion file pattern,
#       not both.  Subsequent uses will be ignored with a warning.
#
#    -  Use of the typical option (-t) will run etcud with the default
#       exclusion pattern below.  This mode overrides any previously
#       specified inclusion or exclusion patterns with a warning.
#       This mode also silently ignores -n.
#
# Original Author:  Matthew Thyer <thyerm@camtech.net.au>  1997, 1998
#
# Adapted from etcud-1.9 (FreeBSD PR#5147) by Greg A. Woods <woods@planix.com>

# The default directory for the installed files
DESTDIR=""

# The default directory for the source distribution
SRC_ROOT="/usr/src"

# The typical exclusion list
DEF_EXCLUSION_PATTERN='etc\..*/fstab\..*|group|hosts|kerberosIV/krb\.conf|kerberosIV/krb\.realms|motd|namedb/named\.boot|shells|rc\.local|master\.passwd|MAKEDEV.local'

PLATFORM=$(uname -m)

opt_all=0
opt_diffs=0
opt_non_exact=0
opt_typical=0
opt_inc=0
opt_exc=0

egrep_flags="-x"
diff_flags=""

pattern=""

argv0=$(basename $0)

OPTIONS="acde:hi:np:r:s:t"

USAGE="Usage: $argv0 [-a] [-d [-c]] [-s <dir>] [-r <dir> [-p <arch>]] [-t | [[-n] [-i <patt> | -e <patt>]]]"

HELP="$USAGE
   -a         Mention all files
   -d         Display diffs between the files (use '-c' to see context diffs)
   -n         Non-exact mode - i.e. dont use '-x' with egrep
   -p <arch>  Specify alternate platform architectrue (use with '-r')
   -t         Typical usage.  Equivalent to \"$argv0 -e '$DEF_EXCLUSION_PATTERN'\"
   -r <dir>   Directory where the target distribution is found (${DESTDIR:-/})
   -s <dir>   Directory where the source is found ($SRC_ROOT)
   -i <patt>  Inclusion filename pattern (those files to check)
   -e <patt>  Exclusion filename pattern (those file to ignore)"

usage ()
{
	echo $USAGE 1>&2
	exit 2
}

help ()
{
	echo "$HELP"
	exit 0
}

do_check ()
{
	src_file=$SRC_ROOT/etc/$x

	# First determine if we should check this file
	# (and what its installed location is)

	case $x in
	etc.${PLATFORM}/MAKEDEV)
		the_file=/dev/MAKEDEV
		;;
	etc.*/Makefile|etc.*/Makefile.inc|*Makefile|*Makefile.inc|etc.*/*.orig|*.orig|*-OLD|*CVS*|*.#*)
		the_file=SKIP
		;;
	etc.${PLATFORM}/*)
		the_file=/etc/$(echo $x | awk -F/ '{print $NF}')
		;;
	etc.*/*)
		the_file=SKIP
		;;
	root/*)
		the_file=${DESTDIR}/$(echo $x | sed 's;/dot\.;/.;')
		;;
	MAKEDEV.local)
		the_file=/dev/MAKEDEV.local
		;;
	minfree)
		the_file=/var/crash/minfree
		;;
	COPYRIGHT)
# XXX		the_file=${DESTDIR}/usr/share/man/COPYRIGHT
		the_file=SKIP
		;;
	rc.d/*)			# avoid "rc.d/*.sh" confusion
		the_file=${DESTDIR}/etc/$x
		;;
	*.sh)
		the_file=${DESTDIR}/etc/$(basename $x .sh)
		;;
	*)
		the_file=${DESTDIR}/etc/$x
		;;
	esac
	if [ $the_file != SKIP ] ; then
		if [ -r $the_file ] ; then
			if cmp $the_file $src_file > /dev/null 2>&1 ; then
				if [ $opt_all -eq 1 ] ; then
					echo "SAME: $src_file $the_file"
				fi
			else
				src_ver=$(ident -q $src_file | sed 1d)
				root_ver=$(ident -q $the_file | sed 1d)
				if [ $opt_diffs -eq 1 ] ; then
					echo "Index: $the_file"
					echo "==================================================================="
					diff $diff_flags $the_file $src_file
					echo ""
				else
					if [ X"$src_ver" != X"$root_ver" ] ; then
						echo "NEEDS MERGE: $src_file $the_file"
					else
						echo "DIFFERENT: $src_file $the_file"
					fi
				fi
			fi
		else # the file is not readable.... why ? perhaps it doesn't exist
			if [ ! -f $the_file ] ; then
				echo "MISSING: cp $src_file $the_file"
			else
				echo "Warning: '$the_file' is not readable...." 1>&2
			fi
		fi
	fi
}

# The main program begins.....

# First get the options
# 
while getopts $OPTIONS OPTCHAR ; do
	case $OPTCHAR in
	a)
		opt_all=1
		;;
	d)
		opt_diffs=1
		;;
	c)
		diff_flags="-c"
		;;
	n)
		opt_non_exact=1
		;;
	p)
		PLATFORM=$OPTARG
		;;
	t)
		opt_typical=1
		if [ $opt_inc -eq 1 -o $opt_exc -eq 1 ] ; then
			echo "Warning: Typical usage overriding prior inclusion or exclusion pattern" 1>&2
			opt_inc=0
		fi
		opt_exc=1
		pattern=$DEF_EXCLUSION_PATTERN
		;;
	r)
		if [ -d $OPTARG -a -d $OPTARG/etc ] ; then
			DESTDIR=$OPTARG
		else
			echo "Error: Root directory \"$OPTARG\" does not exist" 1>&2
			usage
		fi
		;;
	s)
		if [ -d $OPTARG -a -d $OPTARG/etc ] ; then
			SRC_ROOT=$(echo $OPTARG | sed 's;/$;;')
		else
			echo "Error: Source directory \"$OPTARG\" does not exist" 1>&2
			usage
		fi
		;;
	i)
		if [ $opt_inc -eq 1 -o $opt_exc -eq 1 ] ; then
			echo "Warning: subsequent inclusion pattern ignored" 1>&2
		else
			pattern=$OPTARG
			opt_inc=1
		fi
		;;
	e)
		if [ $opt_inc -eq 1 -o $opt_exc -eq 1 ] ; then
			echo "Warning: subsequent exclusion pattern ignored" 1>&2
		else
			pattern=$OPTARG
			opt_exc=1
		fi
		;;
	h)
		help
		;;
	\?)
		usage
		;;
	esac
done
shift $(expr $OPTIND - 1)

if [ $# -ne 0 ] ; then
	echo "Error: too many parameters" 1>&2
	usage
fi

if [ ! -d $SRC_ROOT/etc/etc.$PLATFORM ] ; then
	echo "Error: invalid platform: '$SRC_ROOT/etc/etc.$PLATFORM' does not exist." 1>&2
	usage
fi
if [ $PLATFORM != $(uname -m) -a -z "$DESTDIR" ] ; then
	echo "Error: must specify alternate root (with '-r <dir>') when using '-p $PLATFORM'." 1>&2
	usage
fi

if [ X"$diff_flags" != X -a $opt_diffs -eq 0 ] ; then
	echo "Warning: '-c' ignored without '-d'." 1>&2
fi

# Can only do non_exact mode if we are not doing '-t'
if [ $opt_non_exact -eq 1 -a $opt_typical -eq 0 ] ; then
	egrep_flags=""
fi

if [ $opt_exc -eq 1 ] ; then
	egrep_flags=$egrep_flags" -v "
fi

cd $SRC_ROOT/etc

if [ $opt_inc -eq 1 -o $opt_exc -eq 1 ] ; then
	find . -type f -print | sed 's;^\./;;' | egrep $egrep_flags $pattern | sort | while read x ; do
		do_check
	done
else
	find . -type f -print | sed 's;^\./;;' | sort | while read x ; do
		do_check
	done
fi

exit 0

--iY4F4jJ/dQ--