tech-net archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Odd TCP snd_cwnd behaviour, resulting in pathetic throughput?

Paul Ripke <> writes:

> I've recently been trying to get duplicity running for backups of my
> main home server, out to the magical cloud (aws s3 to
> I have discovered that bandwidth oscillates in a sawtooth fashion,
> ramping up from near nothing, to saturate my uplink (somewhere around
> 20-30Mbit/s), then I get a dropped packet, and it drops to near nothing
> and repeats, each cycle taking around a minute. Looking at netstat -P
> for the PCB in question, I see the snd_cwnd following the pattern,
> which makes sense. I've flipped between reno, newreno & cubic, and
> while subtly different, they all have the snd_cwnd dropping to near
> nothing after a single dropped packet. I didn't think this was
> expected behaviour, especially with SACKs enabled.

Long ago I almost found a bug in how our TCP retransmits, and didn't
manage, for reasons not really about the bug, to identify and land a fix.

> Reading tcpdump, the only odd thing I see is that the duplicate ack
> triggering the fast retransmit is repeated 70+ times. But tracing
> other flows, this doesn't seem abnormal.

That is normal.   However on the third dupack it should cause "fast
recovery", clocking out one new (or retransmitted) packet every dupack.

> It's worth noting that running their "speedtest" thru firefox running
> on the same machine is fine - and bandwidth is as I'd expect.


> Is there anyone willing to take a look at a pcap and tell me what
> I'm missing? ie. cluebat, please?

Sure, send it to me, or put it up for download.

The right tool for this is in pkgsrc as xplot-devel, and it may be that
you need a not-packaged modified tcpdump2xplot to keep up with tcpdump

> fwiw, I do have npf and altq configured, but disabling altq doesn't
> appear to change the behaviour.
> fwiw#2, I briefly toyed with the idea of bringing BBR from FreeBSD,
> but I think we'd need more infrastructure for doing pacing? And while
> it might "fix" this, I think we're better off fixing whatever is
> actually broken.

What you should be seeing is a sawtooth in speed, but from half to full
and then dropping to half, very very roughly.

Attachment: signature.asc
Description: PGP signature

Home | Main Index | Thread Index | Old Index