Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Weaver]

There has to be something odd about the UDP implementation.

[XGS_Is_On]

Ah. I don't use iPerf via Windows.

[Weaver]

Is it safe / legal / allowed /sensible to do iperf/UDP over the internet ? I tried it over IPv6 UDP between my hosted Raspberry Pi out on the internet (hosted by Mythic Beasts) and my iPad here, and all hell broke loose. It just shows that I didn’t know what I was doing.

[Alex Atkin UK]

Yeah, sending from your end to the Internet is probably okay but from the Internet to you could be rather problematic unless you specify the bandwidth correctly.  Although by default it only pushes 1Mbit, presumably to minimise the chances of someone accidentally killing the network on their first try.

That said I can't get it to work from either of my VPS over 3Mbit, one at IONOS and one at Mythic Beasts.  It also throws an error if I try to change the socket buffer size.

[Weaver]

Before I try again in future, I’ll ask the community to sanity check my command line parameters.

[Alex Atkin UK]

Yeah I can't get my head around it.  I tried changing the buffer length but it rejects the values even though they are in range according to the documentation.

Code: [Select]

-b, --bandwidth n[KM] Set target bandwidth to n bits/sec (default 1 Mbit/sec for UDP, unlimited for TCP).
If there are multiple streams (-P flag), the bandwidth limit is applied separately to
each stream. You can also add a ’/’ and a number to the bandwidth specifier. This
is called "burst mode". It will send the given number of packets without pausing,
even if that temporarily exceeds the specified bandwidth limit.

-l, --length n[KM] The length of buffers to read or write. iPerf works by writing an array of len bytes
a number of times. Default is 128 KB for TCP, 8 KB for UDP.

-w, --window n[KM] Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP
window size. (this gets sent to the server and used on that side too)

Code: [Select]

iperf3 -ub 4M -c <IP>iperf3: error - unable to read from stream socket: Resource temporarily unavailable

Code: [Select]

iperf3 -ub 3M -c <IP>
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   330 KBytes  2.70 Mbits/sec  240 
[  4]   1.00-2.00   sec   367 KBytes  3.01 Mbits/sec  267 
[  4]   2.00-3.00   sec   366 KBytes  3.00 Mbits/sec  266 
[  4]   3.00-4.00   sec   366 KBytes  3.00 Mbits/sec  266 
[  4]   4.00-5.00   sec   367 KBytes  3.01 Mbits/sec  267 
[  4]   5.00-6.00   sec   366 KBytes  3.00 Mbits/sec  266 
[  4]   6.00-7.00   sec   366 KBytes  3.00 Mbits/sec  266 
[  4]   7.00-8.00   sec   367 KBytes  3.01 Mbits/sec  267 
[  4]   8.00-9.00   sec   366 KBytes  3.00 Mbits/sec  266 
[  4]   9.00-10.00  sec   366 KBytes  3.00 Mbits/sec  266 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  3.54 MBytes  2.97 Mbits/sec  0.018 ms  0/2637 (0%) 
[  4] Sent 2637 datagrams

Ah, it seems the block size has to be lower than the MSS?

Code: [Select]

iperf3 -u -l 16K -c server
warning: Warning:  UDP block size 16384 exceeds TCP MSS 1408, may result in fragmentation / drops

Code: [Select]

iperf3 -u -l 1408 -b 900M -c server
Connecting to host server, port 5201

[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   102 MBytes   859 Mbits/sec  76304 
[  4]   1.00-2.00   sec   103 MBytes   865 Mbits/sec  76798 
[  4]   2.00-3.00   sec   105 MBytes   878 Mbits/sec  77965 
[  4]   3.00-4.00   sec   102 MBytes   852 Mbits/sec  75641 
[  4]   4.00-5.00   sec  82.5 MBytes   692 Mbits/sec  61472 
[  4]   5.00-6.00   sec  88.1 MBytes   739 Mbits/sec  65593 
[  4]   6.00-7.00   sec  97.2 MBytes   815 Mbits/sec  72365 
[  4]   7.00-8.00   sec   103 MBytes   865 Mbits/sec  76825 
[  4]   8.00-9.00   sec  97.5 MBytes   818 Mbits/sec  72579 
[  4]   9.00-10.00  sec  83.0 MBytes   696 Mbits/sec  61815 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec   963 MBytes   808 Mbits/sec  0.021 ms  325/717357 (0.045%) 
[  4] Sent 717357 datagrams
The packet loss is probably contention on the VPS host at Mythic Beasts rather than the broadband.

[Weaver]

In IPv6 as you know, there is no fragmentation in middleboxes, ie intermediate nodes in the path. Fragmentation only happens at the sending machine, and it is all cool. Indeed some good hardware can handle ‘large send’ offload, maybe even for IPv6 and maybe even for UDP now. You just give it a huge block of data and tell the hardware to get on with shifting it out. I would hope that with UDP the combination of hardware and software in your own sending machine doesn’t overdrive the max egress rate of the interface and first link. I would like to try this large block thing with UDP and compare it with small blocks of say 1280 bytes and 1500 bytes. I feel a lot happier about IPv6 fragment handling than IPv4, because the software involved is likely to be cleaner having been redesigned for IPv6. Evil receiver-end firewalls may remain a problem though.

[Alex Atkin UK]

IPv6 is interesting, it claims the MSS is higher - how?

Code: [Select]

iperf3 -u -l 1408 -b 900M -6 -c ping6

[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec  90.0 MBytes   755 Mbits/sec  67054 
[  4]   1.00-2.00   sec  97.4 MBytes   817 Mbits/sec  72500 
[  4]   2.00-3.00   sec   100 MBytes   841 Mbits/sec  74706 
[  4]   3.00-4.00   sec   104 MBytes   872 Mbits/sec  77418 
[  4]   4.00-5.00   sec  94.4 MBytes   792 Mbits/sec  70307 
[  4]   5.00-6.00   sec   106 MBytes   889 Mbits/sec  78907 
[  4]   6.00-7.00   sec   107 MBytes   894 Mbits/sec  79405 
[  4]   7.00-8.00   sec  95.1 MBytes   798 Mbits/sec  70827 
[  4]   8.00-9.00   sec   104 MBytes   869 Mbits/sec  77187 
[  4]   9.00-10.00  sec   106 MBytes   888 Mbits/sec  78799 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1003 MBytes   842 Mbits/sec  0.022 ms  307/747100 (0.041%) 
[  4] Sent 747100 datagrams
I have absolutely no idea what is going on here:

Code: [Select]

iperf3 -u -l 3000 -b 900M -6 -c ping6
warning: Warning:  UDP block size 3000 exceeds TCP MSS 1428, may result in fragmentation / drops

[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec  98.4 MBytes   825 Mbits/sec  34378 
[  4]   1.00-2.00   sec   108 MBytes   910 Mbits/sec  37910 
[  4]   2.00-3.00   sec   110 MBytes   920 Mbits/sec  38321 
[  4]   3.00-4.00   sec   111 MBytes   931 Mbits/sec  38788 
[  4]   4.00-5.00   sec   106 MBytes   892 Mbits/sec  37154 
[  4]   5.00-6.00   sec   109 MBytes   914 Mbits/sec  38065 
[  4]   6.00-7.00   sec   105 MBytes   878 Mbits/sec  36581 
[  4]   7.00-8.00   sec   110 MBytes   922 Mbits/sec  38435 
[  4]   8.00-9.00   sec   108 MBytes   907 Mbits/sec  37792 
[  4]   9.00-10.00  sec   107 MBytes   901 Mbits/sec  37521 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.05 GBytes   900 Mbits/sec  1.205 ms  225480/225491 (1e+02%) 
[  4] Sent 225491 datagrams

[Weaver]

1428 implies a header which is 40+20+12 = 72 if your IP MTU is 1500, which is IPv6 + TCP + TCP_timestamp, and that should be 20 bytes  longer than IPv4. So your IPv4 MSS should be 20 bytes less than IPv6 of course.

[XGS_Is_On]

Think you've that the wrong way around, Weaver. The headers will be 20 bytes larger than IPv4 so the v4 MSS will be 20 bytes larger than v6. Lower headers, more room for payload, higher MSS.

[Alex Atkin UK]

Think you've that the wrong way around, Weaver. The headers will be 20 bytes larger than IPv4 so the v4 MSS will be 20 bytes larger than v6. Lower headers, more room for payload, higher MSS.

That's what I thought which is why its so confusing.

[Weaver]

Now I reread it, what I wrote was ambiguous. I was referring to the headers to begin with, not the MSS. So bad writing.

[XGS_Is_On]

Bit of iPerf3 action attached.

Some folks might be interested in how little data is going in the other direction - 8 Gb/s of TCP one way required only 19 Mb/s of acknowledgements. I've read occasionally how Virgin Media are cutting it very close with their asymmetry at times but not really the case unless there's something wrong with the TCP stack.

[burakkucat]

Some folks might be interested in how little data is going in the other direction - 8 Gb/s of TCP one way required only 19 Mb/s of acknowledgements.

Interesting. (If I was asked to predict the result I would have had to decline.) Perhaps Weaver will perform one of his calculations to see if he gets agreement with your observed result.

[XGS_Is_On]

Be interesting. 660,000 packets a second being acknowledged by 34,000.

Lots of cumulative acknowledgement going on with selective acknowledgment plugging any gaps in there.

Packet wise, 20:1 give or take. Bandwidth wise 400:1 give or take. The acknowledgements are running on a sliding scale and there are selective acknowledgments requesting and acknowledging retransmissions.

Bandwidth wise expect 40:1500 = 1:37.5 if every packet being a acknowledged so clearly that's closer to an acknowledgment per 11 packets in terms of bandwidth.

Packet-wise it reflects some background noise, some selective acknowledgements to trigger retransmissions and some overlapping cumulative acknowledgements.

400:1 explains how Virgin Media sold 53 download on 1.75 upload and it worked fine