Re: Patch name changes

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Fri, Jun 11, 2010 at 9:06 AM, Thomas Klausner <wiz%netbsd.org@localhost> 
wrote:
> On Thu, Jun 10, 2010 at 07:07:32PM +0000, David Holland wrote:
>> I don't think this is a good idea. A few packages already follow this
>> model (or some approximation of it) and IME it doesn't really make
>> updating any easier; it just makes it somewhat harder to work with the
>> patch directory by e.g. making it harder to write shell globs that
>> exclude editor backup files.
>
> What kind of globs do you use on your patch-directory?
> What exactly is the problem you're seeing? How would it change with
> this naming change?
>
>> In any case, I think it's moving in the wrong direction.
>>
>> Currently we have one patch per source file, which is necessary
>> because of the technical limitations of e.g. mkpatches. However, for
>> most purposes it's much more desirable to have one patch per topic
>> (DESTDIR support, pkgsrc config, 64-bit cleanliness, etc.)
>>
>> Such an organization groups all related changes together for easy
>> inspection and submission upstream. It also makes it much easier to
>> split related changes out into a distfile patch.
>
> I agree it would be nice.
>
>> I've tried three times now to wrangle the 234 patches in xview, the
>> intent being to move most of the changes to one or more distfile
>> patches. In order for that to be useful in the long run the changes
>> need to be sorted by topic; getting there without accidentally losing
>> any bits turns out to be a nontrivial problem.
>
> And I think that's exactly the problem we'd be seeing with
> patches-per-topic. It's perhaps not so hard to create a new patch
> that's per topic, but for package updates, I don't see how it can be
> automated in any meaningful way -- and I think package updates are the
> most common reason for changes in pkgsrc.

That's something the tools have to deal with, not the model.  The
underlying model for the patches should be correct in the first place.
 I'm not completely sure about if any other packaging system manages
patches by topic, but it'd be a very nice advantage for us.  (I know
ports don't, and I know debian package do not either, to say a
couple.)

I'm not advocating using directly devel/quilt, but here is how it
works.  We may want to "fix" it to not rely on gnu-specific tools (to
keep the dependencies very simple), or write our own custom version
(*shrug* why would we do that?).  Note that quilt is what was used
before git to prepare patchsets against the linux kernel tree, so it
is widely tested and it is proven to work for maintaining custom patch
sets.  It also has some features to automatically send patchsets to
upstream (quilt mail).

Let's say you have a patch set for package foo-0.1.  The patch set is
stored in foo/patches (as usual), and the ordering is recorded in
foo/patches/series (which is just a text file with the names of the
patches, in order).  When it is time to upgrade a package, you:

1) Update the Makefile to point to foo-0.2
2) make patch; if it works, nothing to do!
3) If make patch fails, clean the work dir (to make things simple), do
a make extract and go into the workdir.
4) quilt push -a -- this would attempt to apply all patches in the
series and fail (cleanly) on the first that doesn't work.
5) quilt push -f -- force application of first failing patch; it
merges conflicts similar to cvs, so you can manually inspect and fix
failures.
6) quilt refresh -- to update the failed patch with the new versions
7) repeat from 4 until everything is applied

There you go: all the patches have been updated.

And I hear, how do I create a new patch for a missing feature in feature 0.2?

1) cd into workdir
2) quilt push -a -- ensure all patches are applied
3) quilt new patch-fix-destdir.diff -- create a new patch at the top
of the series
4) quilt add Makefile.in src/Makefile.in -- register intent to modify
these files in the new patch
5) hack, hack, hack
6) quilt refresh -- to update the diff file
7) quilt header -e -- to edit the patch header description interactively

Done!  Note that you can add patches anywhere in the series, not only
at the top.  quilt refresh -a takes care of refreshing all patches, in
order, so any chunk offsets in "upper" patches would be automatically
corrected (and conflicts detected!).

Some other notes:

quilt lets you interactively modify the patch headers and maintains
them at all times.  These are used as email subject/body contents when
sending the patches upstream, for example.

quilt also allows you to push/pop patches in the series, and every
time you do that, it will ensure the patches are up-to-date and that
they apply cleanly.  If they do not, the tool aborts gracefully and
asks you to refresh/fix conflicts.

I do think this would work very well for pkgsrc.  It would involve
some training on the developers side... but for simple patchsets,
quilt would not be necessary (but it surely makes things much
simpler).

Cheers,

PS: All the above commands are from vague memories; they may be inaccurate.

PS2: I successfully used quilt back in SoC 2007 to maintain a pretty
big patchset against the NetBSD source tree to integrate ATF into it
and convert existing regression tests to the new framework.  It was a
poor-man's dvcs replacement.  I kept those patches in cvs, which is
exactly what we'd be doing here, and things were pretty nice. -- much
nicer than maintaining a heavily modified local netbsd src tree.

-- 
Julio Merino