Updates on Mercurial

[Joerg Sonnenberger <joerg%bec.de@localhost>]

Hi all,
after a busy time this summer, I finally got a chance to come back to
working on the TODO list for Mercurial during fall. Quite a few things
have changed or have pending patches under review:

(1) Memory use during large operations and especially the initial clone
has much reduced. On AMD64, the unbundle of src's clonebundle as
baseline clocks in at just a bit over 1GB and the "real" history set
(e.g. after removing all conversion artifacts) just a bit over 650MB.
This is the peak RSS as measurement. The 5.6 release has about half of
the changes, the rest will arrive with 5.7.

(2) The phase update computation is much faster. Since that computation
was part of the transaction before all of the incoming data was
processed, it was the likely source of many of the timeouts seen in the
past. The changes are part of the 5.5 release.

(3) Updates for the branchmap (e.g. what are named branches and what are
their heads) are computed much faster. This and a related change for the
topic extension are still under review. The current performance impact
is reasons why the non-overlay repositories (src, pkgsrc, xsrc) still
exists as hgweb triggers the update somewhat badly.

(4) The work on the SHA1 migration front is going steady with a working
proof of concept and cutting things into smaller reviewable patches
underway. This is not necessarily a show stopper.

(5) Optional multi-threaded compression for zstd is under review. This
is useful for larger bundle creation like the src clonebundle. It
allows cutting done the time to 1/4 of the baseline.

--

I've decided that I'm done for now with analyzing the clone performance
for now. There are a few delays still in the process due to a
combination of the unusual history and somewhat high IO latency, but
they should no longer break anything and would disappear with a new
clonebundle.

Joerg