Port-xen archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
sluggish network performance in *one* domU
Hello,
I'm experiencing a rather strange network problem within a Xen guest
system. The problem is bad network performance, it affects all tcp
connections and only appears in one domU.
The hardware is running Xen 3.1.2, NetBSD 4.0 dom0 on i386. There are 3
domUs running: two nearly identical NetBSD copies and one Debian system.
The `problematic' domU is one of those two NetBSD domUs. I checked lots
of stuff: the cpu is idle all the time, there is no disk or network I/O
and I couldn't find anything in the logs.
The layout is simple. Here is a clumsy drawing:
Linux Laptop
+------------------+
|.--------------. |
|| host-eth | | AMD Athlon System
||(host.foo.net)| | +-----------------------+
|`-----.--------' | |.-------. |
+------|-----------+ ,---.|dom0-re| |
| ,-----. / |`---+---' |
\ ,' `. | | \ |
`. / \ / | | |
`-( Internet )-' |.----+----. |
\ / ||dom0-xvif| |
`. ,' |`----`.---' Virtual |
'-----' | \ Machine |
| +----`.------------+|
| | \ ||
| | .-----`.-------. ||
| | | domU-xennet | ||
| | |(domU.bar.net)| ||
| | `--------------' ||
| +------------------+|
+-----------------------+
Based on that I tcpdumped and tcptraced the following ssh session from
my Linux Laptop at home (host.foo.net):
ssh domU.bar.com dd if=/dev/urandom bs=1m count=5 >/dev/null
from four different interfaces:
eth @ host.foo.net
re @ dom0
xvif @ dom0
xennet @ domU.bar.net
And it seems that I'm losing lots of data, since xennet, xvif and re
are all reporting the following:
unique bytes sent: 5254577
actual data pkts: 9346
actual data bytes: 5512613
rexmt data pkts: 369
rexmt data bytes: 258036
(the complete tcptraces are attached)
The xennet interface in domU owns a public IPv4 address which gets
forwarded and filtered by dom0 (+ipfilter).
Now, it would be easy to blame the ISP but it seems strange that the
other domUs are not affected at all. I tested the second NetBSD domU
and there were not one single `rexmt'.
Is there maybe something else I could try?
Thanks,
Petar Bogdanovic
host a: host.foo.net:45866
host b: domU.bar.net:22
complete conn: yes
elapsed time: 0:02:14.672303
total packets: 15436
a->b: b->a:
total packets: 6450 total packets: 8986
ack pkts sent: 6449 ack pkts sent: 8986
pure acks sent: 6394 pure acks sent: 5
sack pkts sent: 1664 sack pkts sent: 2
dsack pkts sent: 1 dsack pkts sent: 2
max sack blks/ack: 3 max sack blks/ack: 1
unique bytes sent: 4981 unique bytes sent: 5254577
actual data pkts: 54 actual data pkts: 8979
actual data bytes: 5821 actual data bytes: 5254701
rexmt data pkts: 2 rexmt data pkts: 1
rexmt data bytes: 840 rexmt data bytes: 124
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 0 outoforder pkts: 342
pushed data pkts: 54 pushed data pkts: 14
SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 6 adv wind scale: 0
req sack: Y req sack: Y
sacks sent: 1664 sacks sent: 2
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 1152 bytes max segm size: 1448 bytes
min segm size: 16 bytes min segm size: 32 bytes
avg segm size: 107 bytes avg segm size: 585 bytes
max win adv: 92672 bytes max win adv: 33580 bytes
min win adv: 5888 bytes min win adv: 33532 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 91439 bytes avg win adv: 33579 bytes
initial window: 21 bytes initial window: 49 bytes
initial window: 1 pkts initial window: 1 pkts
ttl stream length: 4981 bytes ttl stream length: 5254577 bytes
missed data: 0 bytes missed data: 0 bytes
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 134.555 secs data xmit time: 134.553 secs
idletime max: 7532.9 ms idletime max: 7532.9 ms
throughput: 37 Bps throughput: 39018 Bps
host a: host.foo.net:53062
host b: domU.bar.net:22
complete conn: yes
elapsed time: 0:02:14.603709
total packets: 13656
a->b: b->a:
total packets: 4303 total packets: 9353
ack pkts sent: 4302 ack pkts sent: 9353
pure acks sent: 4247 pure acks sent: 5
sack pkts sent: 1658 sack pkts sent: 2
dsack pkts sent: 1 dsack pkts sent: 2
max sack blks/ack: 3 max sack blks/ack: 1
unique bytes sent: 4981 unique bytes sent: 5254577
actual data pkts: 54 actual data pkts: 9346
actual data bytes: 5821 actual data bytes: 5512613
rexmt data pkts: 2 rexmt data pkts: 369
rexmt data bytes: 840 rexmt data bytes: 258036
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 0 outoforder pkts: 0
pushed data pkts: 54 pushed data pkts: 14
SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 6 adv wind scale: 0
req sack: Y req sack: Y
sacks sent: 1658 sacks sent: 2
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 1152 bytes max segm size: 1448 bytes
min segm size: 16 bytes min segm size: 32 bytes
avg segm size: 107 bytes avg segm size: 589 bytes
max win adv: 92672 bytes max win adv: 33580 bytes
min win adv: 5888 bytes min win adv: 33532 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 91369 bytes avg win adv: 33579 bytes
initial window: 21 bytes initial window: 49 bytes
initial window: 1 pkts initial window: 1 pkts
ttl stream length: 4981 bytes ttl stream length: 5254577 bytes
missed data: 0 bytes missed data: 0 bytes
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 134.497 secs data xmit time: 134.487 secs
idletime max: 7520.3 ms idletime max: 3997.8 ms
throughput: 37 Bps throughput: 39037 Bps
host a: host.foo.net:53062
host b: domU.bar.net:22
complete conn: yes
elapsed time: 0:02:14.671055
total packets: 13656
a->b: b->a:
total packets: 4303 total packets: 9353
ack pkts sent: 4302 ack pkts sent: 9353
pure acks sent: 4247 pure acks sent: 5
sack pkts sent: 1658 sack pkts sent: 2
dsack pkts sent: 1 dsack pkts sent: 2
max sack blks/ack: 3 max sack blks/ack: 1
unique bytes sent: 4981 unique bytes sent: 5254577
actual data pkts: 54 actual data pkts: 9346
actual data bytes: 5821 actual data bytes: 5512613
rexmt data pkts: 2 rexmt data pkts: 369
rexmt data bytes: 840 rexmt data bytes: 258036
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 0 outoforder pkts: 0
pushed data pkts: 54 pushed data pkts: 14
SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 6 adv wind scale: 0
req sack: Y req sack: Y
sacks sent: 1658 sacks sent: 2
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 1152 bytes max segm size: 1448 bytes
min segm size: 16 bytes min segm size: 32 bytes
avg segm size: 107 bytes avg segm size: 589 bytes
max win adv: 92672 bytes max win adv: 33580 bytes
min win adv: 5888 bytes min win adv: 33532 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 91369 bytes avg win adv: 33579 bytes
initial window: 21 bytes initial window: 49 bytes
initial window: 1 pkts initial window: 1 pkts
ttl stream length: 4981 bytes ttl stream length: 5254577 bytes
missed data: 0 bytes missed data: 0 bytes
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 134.564 secs data xmit time: 134.554 secs
idletime max: 7524.1 ms idletime max: 3999.8 ms
throughput: 37 Bps throughput: 39018 Bps
host a: host.foo.net:53062
host b: domU.bar.net:22
complete conn: yes
elapsed time: 0:02:14.671068
total packets: 13656
a->b: b->a:
total packets: 4303 total packets: 9353
ack pkts sent: 4302 ack pkts sent: 9353
pure acks sent: 4247 pure acks sent: 5
sack pkts sent: 1658 sack pkts sent: 2
dsack pkts sent: 1 dsack pkts sent: 2
max sack blks/ack: 3 max sack blks/ack: 1
unique bytes sent: 4981 unique bytes sent: 5254577
actual data pkts: 54 actual data pkts: 9346
actual data bytes: 5821 actual data bytes: 5512613
rexmt data pkts: 2 rexmt data pkts: 369
rexmt data bytes: 840 rexmt data bytes: 258036
zwnd probe pkts: 0 zwnd probe pkts: 0
zwnd probe bytes: 0 zwnd probe bytes: 0
outoforder pkts: 0 outoforder pkts: 0
pushed data pkts: 54 pushed data pkts: 14
SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1
req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y
adv wind scale: 6 adv wind scale: 0
req sack: Y req sack: Y
sacks sent: 1658 sacks sent: 2
urgent data pkts: 0 pkts urgent data pkts: 0 pkts
urgent data bytes: 0 bytes urgent data bytes: 0 bytes
mss requested: 1460 bytes mss requested: 1460 bytes
max segm size: 1152 bytes max segm size: 1448 bytes
min segm size: 16 bytes min segm size: 32 bytes
avg segm size: 107 bytes avg segm size: 589 bytes
max win adv: 92672 bytes max win adv: 33580 bytes
min win adv: 5888 bytes min win adv: 33532 bytes
zero win adv: 0 times zero win adv: 0 times
avg win adv: 91369 bytes avg win adv: 33579 bytes
initial window: 21 bytes initial window: 49 bytes
initial window: 1 pkts initial window: 1 pkts
ttl stream length: 4981 bytes ttl stream length: 5254577 bytes
missed data: 0 bytes missed data: 0 bytes
truncated data: 0 bytes truncated data: 0 bytes
truncated packets: 0 pkts truncated packets: 0 pkts
data xmit time: 134.564 secs data xmit time: 134.554 secs
idletime max: 7524.1 ms idletime max: 3999.8 ms
throughput: 37 Bps throughput: 39018 Bps
Home |
Main Index |
Thread Index |
Old Index