Re: Proposed conversion strategy

["Eric S. Raymond" <esr%thyrsus.com@localhost>]

Jeff Rizzo <riz%tastylime.net@localhost>:
> I would love to see a better representation of the state of some of
> these tools which present different "views" of a repo - whether this
> is using hg tools to access a git repo, git to access hg, or cvs, or
> whatever - my experience with this sort of thing is generally that
> they work OK for a limited set of operations, but not real-world
> usage.  I'd love to be proved wrong.

Alas, I can't prove you wrong, as I have never used them seriously
myself.  The good news is that these front ends seem to be evovling
rapidly - so what isn't really production-ready now might well be in
three months.
 
> Likewise, some of the assertions about "git fast-export" format
> being a lingua franca don't seem to be panning out (at least in my
> googling) either - I've been having a heckuva time finding a useful
> way to (for example) go from the output of cvs-fast-export directly
> into Mercurial.  There are several different repos kicking around
> purporting to provide "hg fastimport", but none of them appear to
> work out of the box, at least not with the version of hg I have
> (looks like "3.1.2").

That is interesting. It suggests that something may have bitrotted.
 
I tested hg import on a real repo about 2.5 years ago and it worked fine.
I also know of a significant project - the Roundup issue tracker -
that moved from git to hg that way after I did their SVN-to-git
conversion.  That was maybe two years back.

That you're seeing flakiness now suggests the tools are badly
maintained.  Which is unfortunare news, but not irretrievable.  I have
infrastructure inside reposurgeon to build exporters
(fast-import-stream -> target-system) based only on access to the
target system's CLI.  If I have to write one for Mercurial I will.

> Thus far, the fastest and most reliable tool we have is for
> conversion from CVS to fossil (written by Jörg Sonnenberger, who
> maintains the github NetBSD mirror) - unfortunately, fossil itself
> has issues working with a dataset the size of the NetBSD repo, and
> is not currently a good end-choice.

Agreed.  I understand why exactly - mentioned it in previous email
here.  I hope the Fossil fans in NetBSD-land are realistic about this.

>                                 The conversion from fossil to
> git (which the most-used git mirror of NetBSD uses) takes nearly
> twice as long as from CVS->fossil.  cvs-fast-export (which thus far
> I've only gotten to src.fi stage - I have not yet actually imported
> into git or hg) alone takes slightly longer than the
> CVS->fossil->git dance that we're currently doing, at least on my
> hardware.*

Yeah, we need to figure out why.  I have that down to 3.5 hours
here, and I think it will go lower.

> I am of the opinion (which is not as widely shared within NetBSD as
> I'd like) that it does make sense to begin actively working on a CVS
> replacement within NetBSD, but any replacement is going to require
> months if not years of active developers working with both the
> existing setup and proposed replacement to find the gotchas.  This,
> in turn, is going to require the conversion tools to continue to
> improve, because we're not going to be able to evaluate things
> properly until we can get incremental conversions working properly -
> or get the full-run time (from CVS to system X) down to a few hours.
> We're much, much closer than we used to be, but we're not there yet.
> Any help getting there is appreciated - but this is going to take a
> long time.

I'm going to keep working on improving the conversion tools whatever
NetBSD does.

I'll keep working on making NetBSD's mirroring and conversion process
easier as long as there's an interlocutor on the BSD side actively
interested.  Which right now seems to be you.
-- 
		<a href="http://www.catb.org/~esr/";>Eric S. Raymond</a>