Re: FreeBSD/NetBSD makefs sync

[Ed Maste <emaste%freebsd.org@localhost>]

On 20 October 2015 at 16:54, Christos Zoulas <christos%astron.com@localhost> wrote:
>>
>>FreeBSD's has an option that is currently not taken in NetBSD's makefs:
>>-D    Treat duplicate paths in an mtree manifest as warnings not error.
>
> This functionality is give with the -r flag (r for replace). NetBSD's
> makefs by default will accept duplicate directory definitions and
> output the union of the files found in all the common directory
> definitions. If there are conflicts (same filenames in a directory)
> and -r is not specified, we get an error; if -r is specified the
> node is replaced silently with the last one found. This is useful
> to override passwd and group files for example with ones provided
> for the custom image.

Ok. We added -D in r247041 to treat duplicates as warnings instead of
errors for mtree specs. It keeps the first, not last, entry. We use it
to work around issues in our install infrastructure where the same
file gets installed more than once, so it doesn't matter for this use
which entry is kept. I can see how -r is useful, but -D without -r has
a distinct use case so I imagine we'll keep it even after bringing -r
over.

We also have two different ways to use an mtree spec file -- either
passing -F <specfile> and a directory, or specifying the mtree
manifest directly.  We're generally using the latter one in non-root
image builds.

>>We recently introduced an incompatible -r option (to round up the
>>image size). I've noted that in FreeBSD PR 203707 [2] which provided
>>the patch, and it's been updated to use -R instead.

The switch to -R and man page clean up is in review now at
https://reviews.freebsd.org/D3959 if you'd like to take a look and/or
bring into NetBSD.

>>[1] https://lists.freebsd.org/pipermail/freebsd-arch/2015-October/017376.html
>>[2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203707
>
> These are difficult questions. I think that the best way is for
> you to send us your fixes so we can create version that has all
> the fixes and features.  I will apply them and then you can decide
> how to import/ apply the changes on your side. We can also try in
> the process make the makefs program more header neutral, but we'll
> both need to make filesystem header changes for that to work I
> guess.

It's a bit tricky because at least one other developer is working
through outstanding PRs and Coverity reports against makefs and fixing
issues now, and it will be some time still before anyone will have a
chance to go through our history to find changes that aren't in
NetBSD.  Perhaps we can at least send you the new changes as we work
through them so there's only a gap in the middle to investigate later.

What's the most convenient way to get these changes to you? You could
subscribe to the PRs or subscribe (in Phabricator) to commits to
usr.sbin/makefs in our tree. Or we could send a list of the commits
once this current round of work is done. I think there's somewhere
around 10 issues reported in PRs from Coverity.

One example committed yesterday:
> Free buffer before returning from cd9660_write_path_table to avoid
> leaking it after returning from the function
https://svnweb.freebsd.org/base?view=revision&revision=289687
https://svnweb.freebsd.org/base?view=revision&revision=289693

I haven't checked if this one is already fixed in NetBSD, but a number
of open PRs have confirmation that the issue exists in both places.

Best,
Ed