[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: git on small systems
Sorry to revive this old thread but after reading some news about
Microsoft and other big players moving *GIANT* repos to git (always
with special considerations) I thought it would be worth testing our
own larger repo.
Without any of the special tuning (well I still use threads = 1) and
the latest git (2.11.1) I am able to clone full src with a peak
between 350 - 400MB memory usage over ssh, which is higher memory than
This does appear to be a moderate improvement over our initial testing
two years ago.
On Wed, Jan 7, 2015 at 8:58 PM, matthew sporleder <msporleder%gmail.com@localhost> wrote:
> On Tue, Jan 6, 2015 at 10:13 PM, matthew sporleder <msporleder%gmail.com@localhost> wrote:
>> On Tue, Jan 6, 2015 at 8:22 PM, matthew sporleder <msporleder%gmail.com@localhost> wrote:
>>> On Tue, Jan 6, 2015 at 11:09 AM, matthew sporleder <msporleder%gmail.com@localhost> wrote:
>>>> On Tue, Jan 6, 2015 at 11:08 AM, Justin Cormack
>>>> <justin%specialbusservice.com@localhost> wrote:
>>>>> On Tue, Jan 6, 2015 at 3:55 PM, matthew sporleder <msporleder%gmail.com@localhost> wrote:
>>>>>> I did some testing over the last little bit for git on small systems.
>>>>>> Please correct anything weird you see; I'm not a git master.
>>>>>> I am testing on my VPC that looks like this:
>>>>>> NetBSD vc136-15.vc.panix.com 6.0.2 NetBSD 6.0.2 (PANIX-VC) #0: Sun May
>>>>>> 19 03:41:56 EDT 2013
>>>>> I am guessing that a 32 bit machine would use a noticeable amount
>>>>> less, but the shallow clone looks excellent anyway. Since recent git
>>>>> you can push from shallow clones which helps.
>>>> I wouldn't hold your breath, but if I get some time I'll try to get my
>>>> rpi booting and see what I can accomplish.
>>> So someone just gave me a login to an rpi (128 MB config)
>>> the "shallow" clone worked without any trouble. I'm actually pretty
>>> excited to try this on pkgsrc.
>>> Full clone, even over http, was killed by the UVM killer (not the
>>> weird signal I was getting on my 512 box).
>>> I am currently trying some more styles of clones on my 512 amd64 and
>>> the rpi reconfigured for 256 but my predictions are this:
>>> Even the smallest systems can easily track necessary branches for
>>> security fixes, developing patches, etc using the shallow clone
>>> methods described earlier.
>>> A small-ish system like mine: 512MB + some swap + some git tuning can
>>> get a full history clone. I think this is probably the lowest
>>> reasonable config for any kind of "normal" development (with the full
>>> 1GB of memory is comfortable and probably doesn't require much tuning,
>>> but I wouldn't mind seeing some testing.
>>> NetBSD should offer git over http/https* because it appears to be more
>>> memory-friendly. I am also testing an ssh-based checkout (git:// was
>>> not finishing).
>>> * NetBSD does not ship with mozilla rootcerts so https requires some
>>> extra stuff.
>>> Anyway I hope this helped give some facts around the "small systems"
>>> concerns of git.
>>> I also think there are some server-side options that can help the
>>> clients use less memory (smaller pack sizes or something like that) at
>>> the expense of disk space. I'd be interested to hear any ideas on
>>> that for when we are testing netbsd.org hosted git mirrors.
>> for the record: the ssh clone also failed on my 512 system.
>> I would suggest we do a weekly git bundle or git archive to handle
>> this clone overhead issue, which is similar to offering rsync/sup/etc
> After some even more aggressive tuning* I am able to use ssh or http
> clone of full history on my 512 + 256 amd64 system.
> I am also able to do http full clone on a 256 ram + 256 swap rpi, but
> not ssh/git://
> windowMemory = 1m
> packSizeLimit = 1m
> deltaCacheSize = 1m
> deltaCacheLimit = 10
> packSizeLimit = 1m
> threads = 1
> packedGitWindowSize = 1m
> packedGitLimit = 1m
> deltaBaseCacheLimit = 1m
> compression = 0
> loosecompression = 0
> bigFileThreshold = 10m
> sslVerify = false
> unpackLimit = 10
> So if you can come up with 512MB somehow you should be able to clone
> and work with the full git history over http.
> Again, you can do a --depth 1 clone on pretty much any system. This
> is also much faster.
> I am interested to try an rsync and test pull performance, etc, but
> that will have to wait until we have our own servers setup.
> Any more suggestions? I'm pretty much finished with this line of testing.
Main Index |
Thread Index |