Kitz Forum
Computers & Hardware => Networking => Topic started by: vdsl on November 07, 2020, 01:00:39 PM
-
Toying with the idea of moving part of my LAN to 10Gb ethernet. Initially would just be looking to upgrade the link between my home server and desktop PC.
I've run Cat 6 throughout the house already. It's not explicitly marked as Cat 6A, but is rated at 500 MHz rather than 250MHz.
Core switch is a Zyxel GS1920-24HP, which doesn't support 10Gb or SFP+. So I'd need a 10Gb switch between the server and PC, and then uplink would be 1Gb to the Zyxel.
The MikroTik CRS305-1G-4S+IN (https://www.broadbandbuyer.com/products/36054-mikrotik-crs305-1g-4s-in/) (~£115) looks to be one of the cheapest 10Gb switches I can find, and reviews pretty well. Bonus, it can run on PoE from the uplink port.
To convert the SFP+ ports to RJ45 and reuse the existing cabling, I'm thinking of buying a pair of ipolex 10Gb SFP+ RJ45 Module (https://www.amazon.co.uk/ipolex-Transceiver-10GBase-T-SFP-10G-T-S-Supermicro/dp/B078WLTX26/)s (~£50 each).
The ASUS XG-C100C (https://www.amazon.co.uk/ASUS-XG-C100C-PCI-Network-Interface/dp/B071JR2ZW8/) (~£85 each) looks like the cheapest way to get 10GBASE-T on the server and PC. I can't find any combination of SFP+ PCI-e cards and SFP+ modules that come close to that price.
All in, this comes to the guts of £400. Does this seem like a reasonable approach and kit selection?
-
Funnily enough I've been looking into this recently as well, considering a mix of 2.5Gbps and 10 Gbps to keep costs down - I have a server and two PC's I'd like to connect.
Going SFP+ to RJ45 gets rather expensive, so I was considering a more expensive switch in the first place, but these don't have as many 10Gbps ports, but would allow me to run 1 x 10Gbps with 2 x 2.5Gbps with the future option of 2 x 10Gbps and 1 x 5Gbps if going for the Netgear switches below.
Switches I have on my list so far.
£138 ZyXel XGS1010-12 8 x 1Gb, 2 x 2.5Gb and 2 x 10Gb SFP+ unmanaged
£188 Zyxel XGS1210-12 8 x 1Gb, 2 x 2.5Gb and 2 x 10Gb SFP+ Managed
£208 Netgear MS510TX 4 x 1G, 2 x 1G/2.5G, 2 x 1G/2.5G/5G, 1 x 1G/2.5G/5G/10G, 1 SFP+ 10G Managed
£302 MS510TXPP-100EUS As above but with POE+
For connection to the sever (which is next to the switch) I'm considering going with SFP + DAC cable (https://www.amazon.co.uk/10Gtek-10Gb-SFP-Cable-SFP-H10GB-CU2M/dp/B01DCZCP1G), and an SFP+ card (https://www.ebay.co.uk/itm/INTEL-82599ES-Chip-X520-DA1-E10G41BTDA-10G-Dual-Port-SFP-Ethernet-Server-Adapter/153110592016). Both my PC's are on CAT5e, cables are 12 & 13 meters long so should be good for 10Gbps, but will likely run 2.5Gbps to start with.
So I'll be very interested in what suggestion others have as well.
ETA. That card I linked to on eBay is a fake https://sviko.com/t/how-to-distinguish-10-gigabit-network-card-intel-x520-da2-from-chinese-fakes/37
-
If initially all you want is your home server to your desktop, then the cheapest solution is to simply add a second NIC to both and wire them directly, putting them on a different IP range to the LAN (assign the IP addresses manually as static IPs) and making sure you use that IP for the network shares on your desktop. This also has the bonus in that you can use Jumbo Frames on that network, while not screwing up the LAN.
I have to use two NICs on my desktop at the moment because its ITX so I can't add a 10Gbit NIC. So I use the onboard Gigabit for the LAN and 5Gbit for my NAS VLAN. It basically works the same way even though I only have a single NIC in the NAS (the smart-managed switch effectively makes this behave like two different NICs via VLANs).
-
Over the weekend two Asus XG-C100C cards accidentally got purchased at an average price of £62.50 each, bit more than I'd like to have paid, especially for secondhand items but there you accidents happen ;D
-
Budget 10GbE for me was initially just a Netgear GS110EMX, which was enough to link my main desktop to my main NAS. This was on the longest run of Cat5e I have at around 27m from my study to network cupboard, but I had zero issues running 10GbE over it.
This was replaced a little later when I bought an ex-press demo UniFi XG-6POE for a very friendly price. It now links all the 10GbE clients, server and NASs around the house as well as providing a 10GbE SFP+ link to my 48-port switch, which covers all the other 100Mbps & 1GbE stuff.
I only use Cat5e at home as it is far easier to work with and the short distances and low interference levels ensure a good clean 10GbE link.
From the top - 2 routers, 10 GbE switch, patch-panel, 48-port switch with servers and NASs below:
(https://www.disco3.co.uk/gallery/albums/userpics/10700/normal_9-ER3-ER4-UniFi_Switches-Patch_Panel-Mac_IMG_9933_copy.JPG)
-
Very nice, that would be ideal for my needs, but I just can't justify spending that much, don't seem to be any cheap second hand ones going through ebay either.
-
I've just ordered a second hand Netgear XS508M for 181 Euro (https://www.ebay.co.uk/itm/193738176368) 4 x RJ45 at 1G/2.5G/5G/10G and 10Gb SFP+ unmanaged. It would have been nice to have a managed switch, but I really don't need it.
I narrowly missed out on a used Intel X520-SR1 (variation of the X520-DA1) yesterday for £40, there was four left out of 24, wasn't sure if it would work with a DAC, seller replied back this morning saying it would but they had all sold :no:
Only need the SFP+ card for my server now and that will be me done once it all arrives.
Just realised after all these years that the exchange rate the eBay use and Paypal use are different. eBay uses the actual exchange rate to give an approximate UK price, Paypal use their rather poor exchange rate that ends up costing quite a bit more than eBays approximate price. Always knew Paypals rate was poor, but never realised eBay used the actual exchange rate.
-
I've just ordered a second hand Netgear XS508M for 181 Euro (https://www.ebay.co.uk/itm/193738176368) 4 x RJ45 at 1G/2.5G/5G/10G and 10Gb SFP+ unmanaged. It would have been nice to have a managed switch, but I really don't need it.
Have you tried Gigabit to 10Gigabit? I got something like 200Mbit until I enabled flow control on my switch, which obviously is not an option for an unmanaged switch so would presumably have to be forced on the NICs instead?
The price is that unit new is hilarious, it costs more than the MS510TX smart-managed 8+2 port switch.
-
No I haven't tried 10Gbps yet, the MX510TX only has two 10G capable ports, where as the XS508M has four RJ45 10G ports and one 10G SFP+ ports, there is clearly a big premium for 10g RJ45 ports whichever way you go and very few second hand options, even if you find a second-hand switch they are usually 2/3 of new price or more.
-
Good point, I clearly was half asleep when I posted. They obviously expect all ports to be 10Gbit populated which presumably would avoid my issue.
Still, I'm curious to know if unmanaged switches have the same problem my smart managed switch did when moving data between Gigabit and 10Gigbit as it was utterly useless. Its particularly strange as it was happening across TCP which should have detected the dropped frames and slowed down accordingly.
All the information I found suggested you shouldn't need flow control in 99% of cases, but I found the opposite. ???
-
We'll have to wait a week or two to find out, delivery of the switch is estimated between Wednesday 18 Nov - Tuesday 24 Nov from Poland. There was five listed, now there's only one left, selling out very quickly. I've ordered the card for my server as well, a Supermicro AOC-STGN-I2S dual SFP+ card for £60, didn't need dual port, but was getting impatient. Just got to order some CAT 6 patch cables now and that should hopefully be everything I need.
Hopefully all this is proving somewhat useful to the OP as I seem to have hijacked their thread.
-
Hopefully all this is proving somewhat useful to the OP as I seem to have hijacked their thread.
All useful, I'm waiting eagerly for the review on the Asus XG-C100C cards :) £62.50 each seems like a bargain, they seem to hold their price very well and I'm struggling to find much under £74!
-
All useful, I'm waiting eagerly for the review on the Asus XG-C100C cards :) £62.50 each seems like a bargain, they seem to hold their price very well and I'm struggling to find much under £74!
Darn it, I completely missed that.
It was what I had in my server before I switched it out for Intel when I upgraded the switch (so I could use the SFP+ port), it was fine there. Also used it for a short while in Windows before I went back to my ITX board because the one I upgraded to had coil whine.
I will be using it again as soon as I can re-build my gaming PC with a new Ryzen.
-
All useful, I'm waiting eagerly for the review on the Asus XG-C100C cards :) £62.50 each seems like a bargain, they seem to hold their price very well and I'm struggling to find much under £74!
Set up an eBay search for XG-C100C with email notification, set the price range to say £20 to £65, then you'll get notified when one pops up. Mind you looking at the completed sales I seem to have struck lucky and got the only two recent UK one's under £65.
It would also be worth setting up a search for QXG-10G1T, which is the Qnap card, there's also another but can't think of what that one is.
-
My first Asus card arrived today, along with the DAC cable, would of thought the other card would be here as well, hopefully that will arrive tomorrow.
There is this Asus card on eBay (http://www.ebay.co.uk/itm/284049470050), might be worth an offer and mentioning the other two that recently sold for £60 & £65 including delivery ;)
-
Came across another 10Gbps RJ45 card today, no cheaper than other options though.
Syba 1 Port 10 Gigabit Ethernet Network Card - PCIe x4 10Gb 10GBASE-T NIC AQTION AQC107-10Gbps Ethernet PCI-Express x4 Adapter, SD-PEX24055
-
I am watching this thread with great interest and some jealousy; but almost all the hosts I have now are wireless-only so the speed bottleneck within the LAN lies there, in the WLSN physical layer, not at my HP switch, which is only gigabit. The Raspberry Pis and the Apple TV are wireful now but limited to gigabit too. My Firebrick FB2900 router has one SFP port in it; can’t imagine the router is very fast though as it doesn’t have lots of very expensive custom hardware routing, as far as I know anyway. And all the ethernet copper ports of the FB2900 are gigabit-only, which makes sense as it’s insane to use it as a switch.
-
My second XG-C100C finally arrived this morning, much to the annoyance of my oldest daughter, the postwomen delivered it at 07:15 and the rest of us slept through the door bell ringing :) Lucky she heard it really and the postwomen waited a little while. The other one was also delivered at 07:15 the other day, but I was literally about to leave for work, don't think I've ever had post so early.
I've just run some very basic tests - see attached screenshot.
PC 1 has a 512GB Samsung 950 Pro NVME, PC 2 has a 1TB Sabrent Rocket Gen 4 NVME.
I copied 9.5GB of data using Teracopy, with the main file being 9.48GB in size.
At 1Gbps connection via my switch it averaged 112MB/s and took 1 minute 27 seconds.
At 10Gbps with the two PC's connected by a Cat5e patch cable it averaged 396MB/s and took 24 seconds.
At 10Gbps with the two PC's connected via my patch panel it averaged 314MB/s and took 30 seconds.
I was expecting far faster transfers via 10Gbps, the drives are up to it, and the connection was at 10Gbps, could it be errors?
The current patch cables are basic Cat5e cables, I do have some decent Cat6a cables coming. The third test was running through 25 meters of Cat5e a short patch cable at the patch panel, and two cat5e cables to each PC, one being the flat variety. Once fully setup PC 1 will be running over 12 meters of Cat5e, PC 2 over 13 meters approximately using Cat6a patch cables.
Once I get my switch, new cat6a cables and the other card I can try some tests from the server.
I've also discovered that Asus do a SFP+ card XG-C100F (https://www.asus.com/Networking-IoT-Servers/Wired-Networking/All-series/XG-C100F/)
-
Maybe TCP is just too slow. Or maybe the disk controllers are a bottleneck. Might be worth trying MTU=9000 ? If your kit all supports jumbo frames, that is. Do you have L2 flow control turned on?
-
Worth using iperf3 to check bandwidth between the two machines (i.e. removing drives / file copy protocols from the equation)?
-
@Weaver The SSD's are fine I believe (see attached), no idea about the rest (I'll look into it further when I have the full setup). I was reading from the Samsung to the Sabrent.
@vdsl I have Lan Speed Test setup on the server, so once I've got the switch and the server setup I'll try that. I'll also give Iperf3 a go, tried it once before and didn't get on with it (actually might be confusing it with something else though), not sure why because the video I just watched looks really simple.
-
Maybe TCP is just too slow. Or maybe the disk controllers are a bottleneck. Might be worth trying MTU=9000 ? If your kit all supports jumbo frames, that is. Do you have L2 flow control turned on?
Strange thing I found is an MTU of 6000 performs better than 9000, but that might be due to my desktop being a 5Gbit USB adapter. It does pull 430MB/s from my NAS, which is pretty good for a USB 3.0 adapter.
I'm not entirely convinced the MTU makes much difference in the great scheme of things though, its mostly intended to reduce CPU overhead from the NIC, when I think its the network file transfer protocols that have the most overhead.
-
I think TCP is rubbish in the supercomputer environment although hardware assist can help. Jumbo frames capability is all very well but the software has to use the MTU not just ignore it I suppose. But I think supercomputer LANs use jumbo frames for a reason and giving TCP an easier time is a clear win, fewer packets lost. You’re probably right about network file transfer protocols. IPv6 over the LAN with a jumbo MTU might be very fast because the IPv6 headers are designed to be fast with their qword alignment of fields which suits modern 64-bit machines well (four 128-bit fetches to get the whole thing) and the simple alignment suits hardware processing too. With a large MTU the byte overhead disadvantage of IPv6 due to the greater header size diminishes from (60-40)/1500 to (60-40)/9000 say. That would be an interesting experiment, but it could be ruined by differences in hardware support for TCP+IPv6 vs TCP+IPv4, and an iperf test with UDP mode would also be interesting as then there would only be hardware support for IP to worry about, even though that’s very much enough. I would have thought that most modern cards that have hardware acceleration for IPv4 would have it for IPv6 too now? Can check ?
-
Couple of other thoughts:
* What happens if you start multiple file copy operations in parallel? (or use iperf3 -P option to use parallel streams)?
* Have you tried copying the file via Windows Explorer or command line (e.g. xcopy, robocopy) to rule out Teracopy?
* The Samsung SSD sequential write at 737 MB/s is 5.8 Gbps - not in the picture yet if you're constrained to 396 MB/s (3.1 Gbps), but may come up as you progress.
* Are both machines SMB 3 capable (i.e. Windows 8+ or Server 2012+, unsure on Linux)?
* I assume neither machine is bottlenecked for CPU / RAM?
-
Good point about the multiple streams idea; and also do the UDP iperf thing as I said before because that almost removes all the software apart from the two operating systems. Confession: :-[ I once did a UDP iperf over the internet and immediately regretted it; sorry, world! :-[ :-[ ??? I was (literally) paying money for every dropped packet too.
-
Couple of other thoughts:
1 What happens if you start multiple file copy operations in parallel? (or use iperf3 -P option to use parallel streams)?
2 Have you tried copying the file via Windows Explorer or command line (e.g. xcopy, robocopy) to rule out Teracopy?
3 The Samsung SSD sequential write at 737 MB/s is 5.8 Gbps - not in the picture yet if you're constrained to 396 MB/s (3.1 Gbps), but may come up as you progress.
4 Are both machines SMB 3 capable (i.e. Windows 8+ or Server 2012+, unsure on Linux)?
5 I assume neither machine is bottlenecked for CPU / RAM?
1 Haven't tried at the moment, may do in the morning if I get time
2 No, but Teracopy is generally considered far better than explorer
4 Both Windows 10 64bit Prof.
5 That'll be a no, the one with the Samsung has 32GB DDR4 2666 quad channel and a 5820k at 4.5Ghz, the other is a 5900x with 32GB DDR4 at 3800Mhz ;D
-
Oh crap, did we forget the obligatory "try from a Linux Live USB stick"? ::)
-
What is the performance of SMB3 like compared to the old SMB2 ?
-
Oh crap, did we forget the obligatory "try from a Linux Live USB stick"? ::)
Good jobs it was left out, although I have used a Linux live boot USB stick on a few occasions recently I can well imagine having to use some weird seemingly totally random hieroglyphics to run a Iperf speed test across the network ;)
-
What is the performance of SMB3 like compared to the old SMB2 ?
I don't have numbers, but SMB3 added multichannel support (i.e. multiple concurrent TCP streams and multiple processing threads). I also believe SMB3 is supposed to be a more "chunky" vs "chatty" protocol.
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/dn610980(v=ws.11)#example-configurations
Apparently the CopyFileEx API in Windows was reworked to take advantage of SMB3 features to improve throughput - would be interesting to know whether Teracopy a) uses this API, or b) have applied similar changes themselves.
Also, a registry setting that may be of interest for SMB tuning is Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\DisableLargeMtu - this is defaulted to 0 in Windows 8 only, the docs are a bit ambiguous on Windows 10 behaviour, but I suspect it may be defaulted to 1 (limiting the SMB payload to 64 KB chunks).
https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/role/file-server/
Disclaimer - this is all research, I don't have my 10 Gb LAN yet to test! :)
-
@vdsl I'll leave you to test things like that ;)
Results of running Iperf3 are much better.
First set is via my 1Gbps switch and the hardwired Cat5e network cables.
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 31478 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 112 MBytes 938 Mbits/sec
[ 4] 1.00-2.00 sec 113 MBytes 949 Mbits/sec
[ 4] 2.00-3.00 sec 112 MBytes 944 Mbits/sec
[ 4] 3.00-4.00 sec 112 MBytes 943 Mbits/sec
[ 4] 4.00-5.00 sec 112 MBytes 943 Mbits/sec
[ 4] 5.00-6.00 sec 113 MBytes 947 Mbits/sec
[ 4] 6.00-7.00 sec 109 MBytes 917 Mbits/sec
[ 4] 7.00-8.00 sec 112 MBytes 941 Mbits/sec
[ 4] 8.00-9.00 sec 110 MBytes 925 Mbits/sec
[ 4] 9.00-10.00 sec 110 MBytes 923 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec sender
[ 4] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec receiver
iperf Done.
Second set is via a patch cable between the two PC's
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 22514 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1009 MBytes 8.47 Gbits/sec
[ 4] 1.00-2.00 sec 1008 MBytes 8.46 Gbits/sec
[ 4] 2.00-3.00 sec 1005 MBytes 8.43 Gbits/sec
[ 4] 3.00-4.00 sec 1014 MBytes 8.50 Gbits/sec
[ 4] 4.00-5.00 sec 1017 MBytes 8.53 Gbits/sec
[ 4] 5.00-6.00 sec 1017 MBytes 8.53 Gbits/sec
[ 4] 6.00-7.00 sec 1016 MBytes 8.53 Gbits/sec
[ 4] 7.00-8.00 sec 1010 MBytes 8.47 Gbits/sec
[ 4] 8.00-9.00 sec 1007 MBytes 8.45 Gbits/sec
[ 4] 9.00-10.00 sec 1020 MBytes 8.55 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 9.89 GBytes 8.49 Gbits/sec sender
[ 4] 0.00-10.00 sec 9.89 GBytes 8.49 Gbits/sec receiver
iperf Done.
Last set is via a short patch cable to link the ports on the patch panel - so roughly 25 meters of cat5e
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 32015 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 997 MBytes 8.36 Gbits/sec
[ 4] 1.00-2.00 sec 1010 MBytes 8.47 Gbits/sec
[ 4] 2.00-3.00 sec 1010 MBytes 8.47 Gbits/sec
[ 4] 3.00-4.00 sec 1000 MBytes 8.39 Gbits/sec
[ 4] 4.00-5.00 sec 1002 MBytes 8.41 Gbits/sec
[ 4] 5.00-6.00 sec 981 MBytes 8.23 Gbits/sec
[ 4] 6.00-7.00 sec 990 MBytes 8.30 Gbits/sec
[ 4] 7.00-8.00 sec 1003 MBytes 8.41 Gbits/sec
[ 4] 8.00-9.00 sec 1013 MBytes 8.49 Gbits/sec
[ 4] 9.00-10.00 sec 1010 MBytes 8.47 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 9.78 GBytes 8.40 Gbits/sec sender
[ 4] 0.00-10.00 sec 9.78 GBytes 8.40 Gbits/sec receiver
iperf Done.
Overall a lot better than yesterdays basic test, but ultimately it will be read/write speeds of my server that will be the limiting factor.
Almost forgot, attached a screenshot of Task Manager showing 1159.4Mbps whilst running Iperf3 at 10Gbps but shouldn't it be MB/s ? Definitely something not right there, if I run Iperf3 across the 1Gbps network Task Manager reports just under 900Mbps ???
-
Nice, good to see that the TCP throughput is much closer to 10 Gbps! :)
Did you try iperf3 with additional streams (e.g. -P 2, -P 5)? Would be interesting to see if it can be pushed closer to the theoretical max or if you're hitting cable / card limits.
Not sure what's going on with Task Manager there, I'd agree it would seem to be showing the wrong unit.
@vdsl I'll leave you to test things like that ;)
I'll report back when I manage to bag a couple of NICs for a decent price! :)
-
Did you try iperf3 with additional streams (e.g. -P 2, -P 5)? Would be interesting to see if it can be pushed closer to the theoretical max or if you're hitting cable / card limits.
No I didn't, it's not something I'm familiar with so wasn't aware?
I'll report back when I manage to bag a couple of NICs for a decent price! :)
Can you use SFP+ cards and DAC cables?
-
I would have tested mine but I, umm, can't remember where I put it after taking it out of my desktop when I returned my motherboard. :-[
-
No I didn't, it's not something I'm familiar with so wasn't aware?
Fair enough, you can use "-P" or "--parallel" from the client, e.g.
iperf3 -c <host or IP> -P 2
Note that it's case sensitive, and "-p" means something else entirely (sets the port number IIRC).
I'd try a couple of values, e.g. 2, 5, 10 and see what (if any) effect it has.
Can you use SFP+ cards and DAC cables?
Haven't been able to find an SFP+ option that comes close to the price of the Asus 10GBASE-T cards. It's still a possibility at this stage. Long term I'm planning to move the server into a different room, hoping to reuse my existing cat 6 cables. So I'd probably be looking at 10GBASE-T SFP+ modules anyway, which seem to be about £50 each and only advertise working over 30m of cat 6 (10GbE spec suggests 55m should be possible). I think 30m would be okay, but maybe cutting a little close for my longer runs (if just patching together at the panel).
-
Definitely try UDP mode, as I said earlier.
-
For UDP mode, use:
iperf3 -c <host or IP> -u -b 10G
(-u is UDP, -b is the bandwidth limit, which for UDP, defaults to 1 Mbps)
-
Haven't been able to find an SFP+ option that comes close to the price of the Asus 10GBASE-T cards. It's still a possibility at this stage. Long term I'm planning to move the server into a different room, hoping to reuse my existing cat 6 cables. So I'd probably be looking at 10GBASE-T SFP+ modules anyway, which seem to be about £50 each and only advertise working over 30m of cat 6 (10GbE spec suggests 55m should be possible). I think 30m would be okay, but maybe cutting a little close for my longer runs (if just patching together at the panel).
If you don't mind going second hand then there a quite a few options for SFP+, I missed out on an Intel X520-SR1 for £40, which appears to be identical to an X520-DA1 because I wasn't sure if it would work with a DAC cable, but by all accounts it would. Though if you're going to be moving the server and already have CAT6 installed then that would be easier, luckily for me my server is right next to all my main networking gear. Apparently https://www.fs.com/uk/ is pretty good for cables and transceivers.
Thanks for the info on Iperf3, I'll give that a go later in the week when the other stuff has arrived.
-
My switch turned up today, to all intents and purposes it looks brand new, it came with new cables, new pack of ears, new instructions, and proper foam packaging, just wasn't in a Netgear box.
Need to tidy up the cabling, I bought a splitter cable for the two switches, but can't unplug the other switch to change it over at the moment or I'll get complaints.
(https://i.postimg.cc/L5YXf0Qh/2020-11-18-20-19-09-Copy.jpg) (https://postimg.cc/SY4kbVhp)
Some quick speed tests as requested:
This one is rather slow??
C:\Temp>iperf3 -c 192.168.0.10 -u -b 10G
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 63054 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 158 MBytes 1.32 Gbits/sec 20201
[ 4] 1.00-2.00 sec 161 MBytes 1.35 Gbits/sec 20604
[ 4] 2.00-3.00 sec 161 MBytes 1.35 Gbits/sec 20623
[ 4] 3.00-4.00 sec 164 MBytes 1.38 Gbits/sec 21039
[ 4] 4.00-5.00 sec 164 MBytes 1.38 Gbits/sec 20987
[ 4] 5.00-6.00 sec 165 MBytes 1.38 Gbits/sec 21059
[ 4] 6.00-7.00 sec 159 MBytes 1.33 Gbits/sec 20318
[ 4] 7.00-8.00 sec 165 MBytes 1.38 Gbits/sec 21107
[ 4] 8.00-9.00 sec 164 MBytes 1.38 Gbits/sec 20996
[ 4] 9.00-10.00 sec 162 MBytes 1.36 Gbits/sec 20716
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 1.58 GBytes 1.36 Gbits/sec 0.002 ms 0/207649 (0%)
[ 4] Sent 207649 datagrams
iperf Done.
Oddly running through the switch and with CAT6A patch cables its a bit slower than before, mind you one of the PC's updated to Window 10 20H2 the other day.
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 8979 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 949 MBytes 7.96 Gbits/sec
[ 4] 1.00-2.00 sec 909 MBytes 7.62 Gbits/sec
[ 4] 2.00-3.00 sec 872 MBytes 7.31 Gbits/sec
[ 4] 3.00-4.00 sec 943 MBytes 7.91 Gbits/sec
[ 4] 4.00-5.00 sec 940 MBytes 7.88 Gbits/sec
[ 4] 5.00-6.00 sec 929 MBytes 7.79 Gbits/sec
[ 4] 6.00-7.00 sec 904 MBytes 7.58 Gbits/sec
[ 4] 7.00-8.00 sec 932 MBytes 7.82 Gbits/sec
[ 4] 8.00-9.00 sec 900 MBytes 7.55 Gbits/sec
[ 4] 9.00-10.00 sec 934 MBytes 7.84 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 9.00 GBytes 7.73 Gbits/sec sender
[ 4] 0.00-10.00 sec 9.00 GBytes 7.73 Gbits/sec receiver
iperf Done.
Much closer to 10Gbps in parallel though, although the second time I ran it was 7.5Gbps
C:\Temp>iperf3 -c 192.168.0.10 -P 2
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 9120 connected to 192.168.0.10 port 5201
[ 6] local 192.168.0.11 port 9121 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 562 MBytes 4.71 Gbits/sec
[ 6] 0.00-1.00 sec 549 MBytes 4.60 Gbits/sec
[SUM] 0.00-1.00 sec 1.08 GBytes 9.32 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 1.00-2.00 sec 563 MBytes 4.72 Gbits/sec
[ 6] 1.00-2.00 sec 554 MBytes 4.65 Gbits/sec
[SUM] 1.00-2.00 sec 1.09 GBytes 9.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 2.00-3.00 sec 560 MBytes 4.70 Gbits/sec
[ 6] 2.00-3.00 sec 554 MBytes 4.65 Gbits/sec
[SUM] 2.00-3.00 sec 1.09 GBytes 9.35 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 3.00-4.00 sec 600 MBytes 5.04 Gbits/sec
[ 6] 3.00-4.00 sec 502 MBytes 4.21 Gbits/sec
[SUM] 3.00-4.00 sec 1.08 GBytes 9.25 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 4.00-5.00 sec 634 MBytes 5.32 Gbits/sec
[ 6] 4.00-5.00 sec 449 MBytes 3.77 Gbits/sec
[SUM] 4.00-5.00 sec 1.06 GBytes 9.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 5.00-6.00 sec 536 MBytes 4.50 Gbits/sec
[ 6] 5.00-6.00 sec 548 MBytes 4.60 Gbits/sec
[SUM] 5.00-6.00 sec 1.06 GBytes 9.10 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 6.00-7.00 sec 522 MBytes 4.38 Gbits/sec
[ 6] 6.00-7.00 sec 508 MBytes 4.26 Gbits/sec
[SUM] 6.00-7.00 sec 1.01 GBytes 8.64 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 7.00-8.00 sec 590 MBytes 4.95 Gbits/sec
[ 6] 7.00-8.00 sec 514 MBytes 4.31 Gbits/sec
[SUM] 7.00-8.00 sec 1.08 GBytes 9.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 8.00-9.00 sec 555 MBytes 4.65 Gbits/sec
[ 6] 8.00-9.00 sec 558 MBytes 4.68 Gbits/sec
[SUM] 8.00-9.00 sec 1.09 GBytes 9.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 4] 9.00-10.00 sec 564 MBytes 4.73 Gbits/sec
[ 6] 9.00-10.00 sec 553 MBytes 4.64 Gbits/sec
[SUM] 9.00-10.00 sec 1.09 GBytes 9.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 5.55 GBytes 4.77 Gbits/sec sender
[ 4] 0.00-10.00 sec 5.55 GBytes 4.77 Gbits/sec receiver
[ 6] 0.00-10.00 sec 5.16 GBytes 4.44 Gbits/sec sender
[ 6] 0.00-10.00 sec 5.16 GBytes 4.44 Gbits/sec receiver
[SUM] 0.00-10.00 sec 10.7 GBytes 9.21 Gbits/sec sender
[SUM] 0.00-10.00 sec 10.7 GBytes 9.21 Gbits/sec receiver
iperf Done.
The PC with 20H2 version of Windows actually reports the network speed correctly.
Just waiting on the card for the server now, Royal Mail 2nd class seems to take for ever now :-(
-
Is that an ONT, Ronski?
And a small NAS or Intel NUC or similar?
And I’m wondering what the brightly coloured thing on the left is.
-
The brightly coloured thing is a Dyson cordless vacuum. The black object bottom centre is my UPS, on the left end of the shelf is my pfSense router (a Qotom QOTOM-Q190G4-S02 Mini PC) , other end of the shelf is the Virgin SH3 and my server below, which has a quad satellite tuner for TV recording. I still have a redundant HG612 screwed to the wall, hopefully I'll never need that again.
-
Finally received my SFP+ card today, it wasn't supposed to include any transceivers, I guess someone hadn't noticed the two that were plugged in to it. Both Intel units, a FTLX8571D3BCV-IT and a AFBR-703SDZ-IN2, I may just have to get a cable and see if they work.
(https://i.postimg.cc/r0mdKtj3/2020-11-20-18-44-09-Copy.jpg) (https://postimg.cc/r0mdKtj3)
and the card which is a Supermicro AOC-STGN-I2S Rev 1
(https://i.postimg.cc/LgP81k1C/2020-11-20-20-55-08-Copy.jpg) (https://postimg.cc/LgP81k1C)
Tomorrow I'll install it and see what happens.
-
I installed the card this morning and it worked without installing any drivers, but Iperf3 reported only about 2Gbps transfer speeds.
Went through some fun and games trying to update the drivers, but got there in the end, but that didn't seem to make much difference, no idea why I'm not seeing the speeds I did the other day between the two PC's with the Asus cards either.
From the X99 system to the server I get:
C:\Temp>iperf3 -c 192.168.0.20
Connecting to host 192.168.0.20, port 5201
[ 4] local 192.168.0.11 port 2154 connected to 192.168.0.20 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 319 MBytes 2.68 Gbits/sec
[ 4] 1.00-2.00 sec 350 MBytes 2.93 Gbits/sec
[ 4] 2.00-3.00 sec 362 MBytes 3.03 Gbits/sec
[ 4] 3.00-4.00 sec 349 MBytes 2.93 Gbits/sec
[ 4] 4.00-5.00 sec 378 MBytes 3.17 Gbits/sec
[ 4] 5.00-6.00 sec 357 MBytes 2.99 Gbits/sec
[ 4] 6.00-7.00 sec 360 MBytes 3.02 Gbits/sec
[ 4] 7.00-8.00 sec 369 MBytes 3.09 Gbits/sec
[ 4] 8.00-9.00 sec 377 MBytes 3.16 Gbits/sec
[ 4] 9.00-10.00 sec 367 MBytes 3.08 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 3.50 GBytes 3.01 Gbits/sec sender
[ 4] 0.00-10.00 sec 3.50 GBytes 3.01 Gbits/sec receiver
iperf Done.
The X570 pc to server achieves more
C:\Temp>iperf3 -c 192.168.0.20
Connecting to host 192.168.0.20, port 5201
[ 4] local 192.168.0.10 port 61192 connected to 192.168.0.20 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 447 MBytes 3.75 Gbits/sec
[ 4] 1.00-2.00 sec 562 MBytes 4.71 Gbits/sec
[ 4] 2.00-3.00 sec 559 MBytes 4.69 Gbits/sec
[ 4] 3.00-4.00 sec 569 MBytes 4.77 Gbits/sec
[ 4] 4.00-5.00 sec 584 MBytes 4.90 Gbits/sec
[ 4] 5.00-6.00 sec 582 MBytes 4.88 Gbits/sec
[ 4] 6.00-7.00 sec 574 MBytes 4.82 Gbits/sec
[ 4] 7.00-8.00 sec 559 MBytes 4.69 Gbits/sec
[ 4] 8.00-9.00 sec 556 MBytes 4.66 Gbits/sec
[ 4] 9.00-10.00 sec 576 MBytes 4.83 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 5.44 GBytes 4.67 Gbits/sec sender
[ 4] 0.00-10.00 sec 5.44 GBytes 4.67 Gbits/sec receiver
iperf Done.
and X99 to X570 system
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 4] local 192.168.0.11 port 33823 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 624 MBytes 5.23 Gbits/sec
[ 4] 1.00-2.00 sec 611 MBytes 5.13 Gbits/sec
[ 4] 2.00-3.00 sec 657 MBytes 5.51 Gbits/sec
[ 4] 3.00-4.00 sec 657 MBytes 5.52 Gbits/sec
[ 4] 4.00-5.00 sec 648 MBytes 5.43 Gbits/sec
[ 4] 5.00-6.00 sec 657 MBytes 5.51 Gbits/sec
[ 4] 6.00-7.00 sec 657 MBytes 5.51 Gbits/sec
[ 4] 7.00-8.00 sec 675 MBytes 5.67 Gbits/sec
[ 4] 8.00-9.00 sec 668 MBytes 5.60 Gbits/sec
[ 4] 9.00-10.00 sec 680 MBytes 5.70 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 6.38 GBytes 5.48 Gbits/sec sender
[ 4] 0.00-10.00 sec 6.38 GBytes 5.48 Gbits/sec receiver
iperf Done.
When I run Iperf3 in parallel it does perform better, but still not as high as it did the other day.
File transfers to/from the server do seem roughly 3 times faster though.
-
I found this guide last night for updating the firmware on the Asus cards, I've updated one, but haven't done the other yet, I'll also install the Marvell drivers later. There is also some tips for improving the through put as well, which I'll give a try later.
https://www.rashedtalukder.com/updating-asus-xg-c100cs-aquantia-aqc107-firmware-and-drivers/
This thread also looks useful, lists some newer firmware and drivers https://www.overclock.net/threads/aquantia-10g-lan-firmware-update-aqc107-3-1-90.1735798/
-
I don't recall having any issues once flow control was enabled, but maybe this firmware actually fixes the bug that caused me to need flow control in the first place?
-
Was that on Linux or Windows Alex?
-
I received my LC to LC OM3 fibre cable today, and the transceivers work, so I'm now running fibre between the server and switch.
I've moved the cards around, both in my X99 system and the server as the cards were in slots not able to fully utilise maximum speed, but it made no difference to my Iperf3 results. I think it's a Windows thing, and may be also Iperf3, I found some talk of it having some problems on Windows. At some point I'll try Linux Live USB stick on the two desktop systems.
The motherboard on the server only has PCIe 2 slots (it is 10 years old), but as far as I can work out the x8 slot the NIC is in should be plenty fast enough. I'm considering changing the motherboard, and at some point installing Windows 10 Pro on there instead of WHS2011 which is now out of support.
-
Was that on Linux or Windows Alex?
I think I tested on both but to be safe I will assume Linux, the NAS I was testing against would always be Linux though.
https://forum.kitz.co.uk/index.php/topic,24581.msg413332.html
-
I'm now hitting around 9Gbps between the two Windows 10 machines, but getting substantially lower to the server, between 4 to 6Gbps. This is with Jumbo Packets of 9014 bytes on all three machines.
The server is about 10 years old, running WHS2011 on an i3 540 CPU, but I don't see any increase in CPU usage during the tests. The NIC is a Supermicro AOC-STGN-I2S SFP+ which uses an Intel 82599 chipset. It's plugged into the x16 socket, but is a x8 card, socket is PCIe v2.
Microsoft Windows [Version 10.0.19042.630]
(c) 2020 Microsoft Corporation. All rights reserved.
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 5] local 192.168.0.11 port 29224 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 998 MBytes 8.37 Gbits/sec
[ 5] 1.00-2.00 sec 1.06 GBytes 9.12 Gbits/sec
[ 5] 2.00-3.00 sec 1.00 GBytes 8.62 Gbits/sec
[ 5] 3.00-4.00 sec 980 MBytes 8.22 Gbits/sec
[ 5] 4.00-5.00 sec 1.05 GBytes 9.03 Gbits/sec
[ 5] 5.00-6.00 sec 1.08 GBytes 9.25 Gbits/sec
[ 5] 6.00-7.00 sec 1.09 GBytes 9.36 Gbits/sec
[ 5] 7.00-8.00 sec 1.09 GBytes 9.32 Gbits/sec
[ 5] 8.00-9.00 sec 1.09 GBytes 9.37 Gbits/sec
[ 5] 9.00-10.00 sec 1.04 GBytes 8.95 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 10.4 GBytes 8.96 Gbits/sec sender
[ 5] 0.00-10.05 sec 10.4 GBytes 8.91 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.10
Connecting to host 192.168.0.10, port 5201
[ 5] local 192.168.0.11 port 29228 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 1.02 GBytes 8.78 Gbits/sec
[ 5] 1.00-2.00 sec 1.07 GBytes 9.19 Gbits/sec
[ 5] 2.00-3.00 sec 1.06 GBytes 9.10 Gbits/sec
[ 5] 3.00-4.00 sec 1.02 GBytes 8.75 Gbits/sec
[ 5] 4.00-5.00 sec 1.12 GBytes 9.60 Gbits/sec
[ 5] 5.00-6.00 sec 1.07 GBytes 9.15 Gbits/sec
[ 5] 6.00-7.00 sec 1018 MBytes 8.54 Gbits/sec
[ 5] 7.00-8.00 sec 1.02 GBytes 8.76 Gbits/sec
[ 5] 8.00-9.00 sec 1.09 GBytes 9.32 Gbits/sec
[ 5] 9.00-10.00 sec 1.11 GBytes 9.57 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 10.6 GBytes 9.08 Gbits/sec sender
[ 5] 0.00-10.04 sec 10.6 GBytes 9.04 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.10 -P2
Connecting to host 192.168.0.10, port 5201
[ 5] local 192.168.0.11 port 29230 connected to 192.168.0.10 port 5201
[ 7] local 192.168.0.11 port 29231 connected to 192.168.0.10 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 569 MBytes 4.76 Gbits/sec
[ 7] 0.00-1.00 sec 574 MBytes 4.80 Gbits/sec
[SUM] 0.00-1.00 sec 1.12 GBytes 9.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 558 MBytes 4.69 Gbits/sec
[ 7] 1.00-2.00 sec 570 MBytes 4.79 Gbits/sec
[SUM] 1.00-2.00 sec 1.10 GBytes 9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 576 MBytes 4.83 Gbits/sec
[ 7] 2.00-3.00 sec 583 MBytes 4.89 Gbits/sec
[SUM] 2.00-3.00 sec 1.13 GBytes 9.72 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 545 MBytes 4.58 Gbits/sec
[ 7] 3.00-4.00 sec 546 MBytes 4.59 Gbits/sec
[SUM] 3.00-4.00 sec 1.07 GBytes 9.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 575 MBytes 4.82 Gbits/sec
[ 7] 4.00-5.00 sec 571 MBytes 4.78 Gbits/sec
[SUM] 4.00-5.00 sec 1.12 GBytes 9.60 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 553 MBytes 4.64 Gbits/sec
[ 7] 5.00-6.00 sec 585 MBytes 4.90 Gbits/sec
[SUM] 5.00-6.00 sec 1.11 GBytes 9.54 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 544 MBytes 4.56 Gbits/sec
[ 7] 6.00-7.00 sec 552 MBytes 4.63 Gbits/sec
[SUM] 6.00-7.00 sec 1.07 GBytes 9.20 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 543 MBytes 4.56 Gbits/sec
[ 7] 7.00-8.00 sec 537 MBytes 4.51 Gbits/sec
[SUM] 7.00-8.00 sec 1.05 GBytes 9.07 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 519 MBytes 4.35 Gbits/sec
[ 7] 8.00-9.00 sec 484 MBytes 4.07 Gbits/sec
[SUM] 8.00-9.00 sec 1004 MBytes 8.42 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 551 MBytes 4.62 Gbits/sec
[ 7] 9.00-10.00 sec 541 MBytes 4.54 Gbits/sec
[SUM] 9.00-10.00 sec 1.07 GBytes 9.15 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 5.40 GBytes 4.64 Gbits/sec sender
[ 5] 0.00-10.04 sec 5.40 GBytes 4.62 Gbits/sec receiver
[ 7] 0.00-10.00 sec 5.41 GBytes 4.65 Gbits/sec sender
[ 7] 0.00-10.04 sec 5.41 GBytes 4.63 Gbits/sec receiver
[SUM] 0.00-10.00 sec 10.8 GBytes 9.29 Gbits/sec sender
[SUM] 0.00-10.04 sec 10.8 GBytes 9.25 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.20
Connecting to host 192.168.0.20, port 5201
[ 5] local 192.168.0.11 port 29256 connected to 192.168.0.20 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 338 MBytes 2.83 Gbits/sec
[ 5] 1.00-2.00 sec 414 MBytes 3.48 Gbits/sec
[ 5] 2.00-3.00 sec 409 MBytes 3.43 Gbits/sec
[ 5] 3.00-4.00 sec 419 MBytes 3.52 Gbits/sec
[ 5] 4.00-5.00 sec 403 MBytes 3.38 Gbits/sec
[ 5] 5.00-6.00 sec 426 MBytes 3.58 Gbits/sec
[ 5] 6.00-7.00 sec 356 MBytes 2.99 Gbits/sec
[ 5] 7.00-8.00 sec 376 MBytes 3.15 Gbits/sec
[ 5] 8.00-9.00 sec 384 MBytes 3.22 Gbits/sec
[ 5] 9.00-10.00 sec 408 MBytes 3.42 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 3.84 GBytes 3.30 Gbits/sec sender
[ 5] 0.00-10.03 sec 3.84 GBytes 3.29 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.20 -P4
Connecting to host 192.168.0.20, port 5201
[ 5] local 192.168.0.11 port 29258 connected to 192.168.0.20 port 5201
[ 7] local 192.168.0.11 port 29259 connected to 192.168.0.20 port 5201
[ 9] local 192.168.0.11 port 29260 connected to 192.168.0.20 port 5201
[ 11] local 192.168.0.11 port 29261 connected to 192.168.0.20 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 156 MBytes 1.31 Gbits/sec
[ 7] 0.00-1.00 sec 152 MBytes 1.28 Gbits/sec
[ 9] 0.00-1.00 sec 140 MBytes 1.18 Gbits/sec
[ 11] 0.00-1.00 sec 137 MBytes 1.15 Gbits/sec
[SUM] 0.00-1.00 sec 586 MBytes 4.92 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 194 MBytes 1.63 Gbits/sec
[ 7] 1.00-2.00 sec 185 MBytes 1.55 Gbits/sec
[ 9] 1.00-2.00 sec 163 MBytes 1.37 Gbits/sec
[ 11] 1.00-2.00 sec 159 MBytes 1.33 Gbits/sec
[SUM] 1.00-2.00 sec 701 MBytes 5.88 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 199 MBytes 1.67 Gbits/sec
[ 7] 2.00-3.00 sec 184 MBytes 1.55 Gbits/sec
[ 9] 2.00-3.00 sec 163 MBytes 1.37 Gbits/sec
[ 11] 2.00-3.00 sec 161 MBytes 1.35 Gbits/sec
[SUM] 2.00-3.00 sec 708 MBytes 5.94 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 207 MBytes 1.73 Gbits/sec
[ 7] 3.00-4.00 sec 186 MBytes 1.56 Gbits/sec
[ 9] 3.00-4.00 sec 163 MBytes 1.37 Gbits/sec
[ 11] 3.00-4.00 sec 153 MBytes 1.29 Gbits/sec
[SUM] 3.00-4.00 sec 709 MBytes 5.95 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 187 MBytes 1.57 Gbits/sec
[ 7] 4.00-5.00 sec 183 MBytes 1.54 Gbits/sec
[ 9] 4.00-5.00 sec 126 MBytes 1.06 Gbits/sec
[ 11] 4.00-5.00 sec 123 MBytes 1.03 Gbits/sec
[SUM] 4.00-5.00 sec 620 MBytes 5.20 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 218 MBytes 1.83 Gbits/sec
[ 7] 5.00-6.00 sec 200 MBytes 1.68 Gbits/sec
[ 9] 5.00-6.00 sec 160 MBytes 1.34 Gbits/sec
[ 11] 5.00-6.00 sec 154 MBytes 1.29 Gbits/sec
[SUM] 5.00-6.00 sec 732 MBytes 6.14 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 200 MBytes 1.68 Gbits/sec
[ 7] 6.00-7.00 sec 172 MBytes 1.44 Gbits/sec
[ 9] 6.00-7.00 sec 165 MBytes 1.38 Gbits/sec
[ 11] 6.00-7.00 sec 157 MBytes 1.31 Gbits/sec
[SUM] 6.00-7.00 sec 694 MBytes 5.82 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 196 MBytes 1.65 Gbits/sec
[ 7] 7.00-8.00 sec 181 MBytes 1.52 Gbits/sec
[ 9] 7.00-8.00 sec 160 MBytes 1.34 Gbits/sec
[ 11] 7.00-8.00 sec 157 MBytes 1.31 Gbits/sec
[SUM] 7.00-8.00 sec 694 MBytes 5.82 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 211 MBytes 1.77 Gbits/sec
[ 7] 8.00-9.00 sec 192 MBytes 1.61 Gbits/sec
[ 9] 8.00-9.00 sec 154 MBytes 1.29 Gbits/sec
[ 11] 8.00-9.00 sec 150 MBytes 1.26 Gbits/sec
[SUM] 8.00-9.00 sec 708 MBytes 5.94 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 213 MBytes 1.79 Gbits/sec
[ 7] 9.00-10.00 sec 192 MBytes 1.61 Gbits/sec
[ 9] 9.00-10.00 sec 167 MBytes 1.40 Gbits/sec
[ 11] 9.00-10.00 sec 164 MBytes 1.37 Gbits/sec
[SUM] 9.00-10.00 sec 736 MBytes 6.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.94 GBytes 1.66 Gbits/sec sender
[ 5] 0.00-10.01 sec 1.94 GBytes 1.66 Gbits/sec receiver
[ 7] 0.00-10.00 sec 1.78 GBytes 1.53 Gbits/sec sender
[ 7] 0.00-10.01 sec 1.78 GBytes 1.53 Gbits/sec receiver
[ 9] 0.00-10.00 sec 1.53 GBytes 1.31 Gbits/sec sender
[ 9] 0.00-10.01 sec 1.52 GBytes 1.31 Gbits/sec receiver
[ 11] 0.00-10.00 sec 1.48 GBytes 1.27 Gbits/sec sender
[ 11] 0.00-10.01 sec 1.48 GBytes 1.27 Gbits/sec receiver
[SUM] 0.00-10.00 sec 6.73 GBytes 5.78 Gbits/sec sender
[SUM] 0.00-10.01 sec 6.73 GBytes 5.77 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.20 -P8
Connecting to host 192.168.0.20, port 5201
[ 5] local 192.168.0.11 port 29266 connected to 192.168.0.20 port 5201
[ 7] local 192.168.0.11 port 29267 connected to 192.168.0.20 port 5201
[ 9] local 192.168.0.11 port 29268 connected to 192.168.0.20 port 5201
[ 11] local 192.168.0.11 port 29269 connected to 192.168.0.20 port 5201
[ 13] local 192.168.0.11 port 29270 connected to 192.168.0.20 port 5201
[ 15] local 192.168.0.11 port 29271 connected to 192.168.0.20 port 5201
[ 17] local 192.168.0.11 port 29272 connected to 192.168.0.20 port 5201
[ 19] local 192.168.0.11 port 29273 connected to 192.168.0.20 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 68.4 MBytes 573 Mbits/sec
[ 7] 0.00-1.00 sec 66.6 MBytes 559 Mbits/sec
[ 9] 0.00-1.00 sec 132 MBytes 1.10 Gbits/sec
[ 11] 0.00-1.00 sec 143 MBytes 1.20 Gbits/sec
[ 13] 0.00-1.00 sec 66.5 MBytes 558 Mbits/sec
[ 15] 0.00-1.00 sec 62.8 MBytes 526 Mbits/sec
[ 17] 0.00-1.00 sec 63.8 MBytes 535 Mbits/sec
[ 19] 0.00-1.00 sec 60.9 MBytes 510 Mbits/sec
[SUM] 0.00-1.00 sec 663 MBytes 5.56 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 76.9 MBytes 645 Mbits/sec
[ 7] 1.00-2.00 sec 74.0 MBytes 621 Mbits/sec
[ 9] 1.00-2.00 sec 153 MBytes 1.28 Gbits/sec
[ 11] 1.00-2.00 sec 146 MBytes 1.22 Gbits/sec
[ 13] 1.00-2.00 sec 85.0 MBytes 713 Mbits/sec
[ 15] 1.00-2.00 sec 73.0 MBytes 613 Mbits/sec
[ 17] 1.00-2.00 sec 75.1 MBytes 630 Mbits/sec
[ 19] 1.00-2.00 sec 75.4 MBytes 632 Mbits/sec
[SUM] 1.00-2.00 sec 758 MBytes 6.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 83.5 MBytes 700 Mbits/sec
[ 7] 2.00-3.00 sec 79.5 MBytes 667 Mbits/sec
[ 9] 2.00-3.00 sec 146 MBytes 1.22 Gbits/sec
[ 11] 2.00-3.00 sec 129 MBytes 1.08 Gbits/sec
[ 13] 2.00-3.00 sec 81.6 MBytes 685 Mbits/sec
[ 15] 2.00-3.00 sec 80.2 MBytes 673 Mbits/sec
[ 17] 2.00-3.00 sec 78.0 MBytes 654 Mbits/sec
[ 19] 2.00-3.00 sec 82.5 MBytes 692 Mbits/sec
[SUM] 2.00-3.00 sec 760 MBytes 6.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 78.0 MBytes 654 Mbits/sec
[ 7] 3.00-4.00 sec 75.4 MBytes 632 Mbits/sec
[ 9] 3.00-4.00 sec 145 MBytes 1.22 Gbits/sec
[ 11] 3.00-4.00 sec 134 MBytes 1.13 Gbits/sec
[ 13] 3.00-4.00 sec 76.9 MBytes 645 Mbits/sec
[ 15] 3.00-4.00 sec 72.8 MBytes 610 Mbits/sec
[ 17] 3.00-4.00 sec 74.2 MBytes 623 Mbits/sec
[ 19] 3.00-4.00 sec 69.9 MBytes 586 Mbits/sec
[SUM] 3.00-4.00 sec 727 MBytes 6.10 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 78.1 MBytes 655 Mbits/sec
[ 7] 4.00-5.00 sec 63.5 MBytes 533 Mbits/sec
[ 9] 4.00-5.00 sec 163 MBytes 1.37 Gbits/sec
[ 11] 4.00-5.00 sec 148 MBytes 1.24 Gbits/sec
[ 13] 4.00-5.00 sec 80.6 MBytes 676 Mbits/sec
[ 15] 4.00-5.00 sec 74.4 MBytes 624 Mbits/sec
[ 17] 4.00-5.00 sec 76.8 MBytes 644 Mbits/sec
[ 19] 4.00-5.00 sec 74.2 MBytes 623 Mbits/sec
[SUM] 4.00-5.00 sec 759 MBytes 6.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 78.8 MBytes 659 Mbits/sec
[ 7] 5.00-6.00 sec 74.8 MBytes 625 Mbits/sec
[ 9] 5.00-6.00 sec 143 MBytes 1.19 Gbits/sec
[ 11] 5.00-6.00 sec 135 MBytes 1.13 Gbits/sec
[ 13] 5.00-6.00 sec 79.8 MBytes 667 Mbits/sec
[ 15] 5.00-6.00 sec 73.0 MBytes 611 Mbits/sec
[ 17] 5.00-6.00 sec 76.8 MBytes 642 Mbits/sec
[ 19] 5.00-6.00 sec 74.1 MBytes 620 Mbits/sec
[SUM] 5.00-6.00 sec 734 MBytes 6.14 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 74.9 MBytes 630 Mbits/sec
[ 7] 6.00-7.00 sec 66.0 MBytes 555 Mbits/sec
[ 9] 6.00-7.00 sec 157 MBytes 1.32 Gbits/sec
[ 11] 6.00-7.00 sec 141 MBytes 1.19 Gbits/sec
[ 13] 6.00-7.00 sec 74.6 MBytes 628 Mbits/sec
[ 15] 6.00-7.00 sec 68.1 MBytes 573 Mbits/sec
[ 17] 6.00-7.00 sec 71.6 MBytes 603 Mbits/sec
[ 19] 6.00-7.00 sec 70.2 MBytes 591 Mbits/sec
[SUM] 6.00-7.00 sec 723 MBytes 6.09 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 77.9 MBytes 653 Mbits/sec
[ 7] 7.00-8.00 sec 76.6 MBytes 642 Mbits/sec
[ 9] 7.00-8.00 sec 152 MBytes 1.27 Gbits/sec
[ 11] 7.00-8.00 sec 144 MBytes 1.21 Gbits/sec
[ 13] 7.00-8.00 sec 79.4 MBytes 666 Mbits/sec
[ 15] 7.00-8.00 sec 73.9 MBytes 619 Mbits/sec
[ 17] 7.00-8.00 sec 75.6 MBytes 634 Mbits/sec
[ 19] 7.00-8.00 sec 73.6 MBytes 617 Mbits/sec
[SUM] 7.00-8.00 sec 753 MBytes 6.31 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 82.5 MBytes 692 Mbits/sec
[ 7] 8.00-9.00 sec 83.4 MBytes 700 Mbits/sec
[ 9] 8.00-9.00 sec 144 MBytes 1.21 Gbits/sec
[ 11] 8.00-9.00 sec 146 MBytes 1.22 Gbits/sec
[ 13] 8.00-9.00 sec 79.6 MBytes 668 Mbits/sec
[ 15] 8.00-9.00 sec 72.9 MBytes 611 Mbits/sec
[ 17] 8.00-9.00 sec 72.1 MBytes 605 Mbits/sec
[ 19] 8.00-9.00 sec 75.0 MBytes 629 Mbits/sec
[SUM] 8.00-9.00 sec 755 MBytes 6.34 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 81.6 MBytes 684 Mbits/sec
[ 7] 9.00-10.00 sec 77.6 MBytes 650 Mbits/sec
[ 9] 9.00-10.00 sec 153 MBytes 1.28 Gbits/sec
[ 11] 9.00-10.00 sec 150 MBytes 1.26 Gbits/sec
[ 13] 9.00-10.00 sec 82.1 MBytes 688 Mbits/sec
[ 15] 9.00-10.00 sec 73.8 MBytes 618 Mbits/sec
[ 17] 9.00-10.00 sec 79.1 MBytes 663 Mbits/sec
[ 19] 9.00-10.00 sec 74.1 MBytes 621 Mbits/sec
[SUM] 9.00-10.00 sec 771 MBytes 6.46 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 780 MBytes 655 Mbits/sec sender
[ 5] 0.00-10.01 sec 780 MBytes 654 Mbits/sec receiver
[ 7] 0.00-10.00 sec 737 MBytes 618 Mbits/sec sender
[ 7] 0.00-10.01 sec 737 MBytes 618 Mbits/sec receiver
[ 9] 0.00-10.00 sec 1.45 GBytes 1.25 Gbits/sec sender
[ 9] 0.00-10.01 sec 1.45 GBytes 1.25 Gbits/sec receiver
[ 11] 0.00-10.00 sec 1.38 GBytes 1.19 Gbits/sec sender
[ 11] 0.00-10.01 sec 1.38 GBytes 1.19 Gbits/sec receiver
[ 13] 0.00-10.00 sec 786 MBytes 659 Mbits/sec sender
[ 13] 0.00-10.01 sec 786 MBytes 659 Mbits/sec receiver
[ 15] 0.00-10.00 sec 725 MBytes 608 Mbits/sec sender
[ 15] 0.00-10.01 sec 724 MBytes 607 Mbits/sec receiver
[ 17] 0.00-10.00 sec 743 MBytes 623 Mbits/sec sender
[ 17] 0.00-10.01 sec 743 MBytes 623 Mbits/sec receiver
[ 19] 0.00-10.00 sec 730 MBytes 612 Mbits/sec sender
[ 19] 0.00-10.01 sec 730 MBytes 612 Mbits/sec receiver
[SUM] 0.00-10.00 sec 7.23 GBytes 6.21 Gbits/sec sender
[SUM] 0.00-10.01 sec 7.23 GBytes 6.20 Gbits/sec receiver
iperf Done.
C:\Temp>
-
I'd definitely try booting the server from a Linux USB stick, just to confirm its not a hardware problem.
-
I've now tested with Linux Mint on the server and get slightly better results, but still no where near what it should be.
Iperf3 Windows 10 to Linux Mint on my server.
Microsoft Windows [Version 10.0.19042.630]
(c) 2020 Microsoft Corporation. All rights reserved.
C:\Temp>iperf3 -c 192.168.0.209
Connecting to host 192.168.0.209, port 5201
[ 5] local 192.168.0.11 port 9274 connected to 192.168.0.209 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 530 MBytes 4.44 Gbits/sec
[ 5] 1.00-2.00 sec 442 MBytes 3.71 Gbits/sec
[ 5] 2.00-3.00 sec 458 MBytes 3.84 Gbits/sec
[ 5] 3.00-4.00 sec 484 MBytes 4.06 Gbits/sec
[ 5] 4.00-5.00 sec 544 MBytes 4.57 Gbits/sec
[ 5] 5.00-6.00 sec 448 MBytes 3.76 Gbits/sec
[ 5] 6.00-7.00 sec 401 MBytes 3.36 Gbits/sec
[ 5] 7.00-8.01 sec 427 MBytes 3.56 Gbits/sec
[ 5] 8.01-9.00 sec 446 MBytes 3.77 Gbits/sec
[ 5] 9.00-10.00 sec 473 MBytes 3.97 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 4.54 GBytes 3.90 Gbits/sec sender
[ 5] 0.00-10.05 sec 4.54 GBytes 3.88 Gbits/sec receiver
iperf Done.
C:\Temp>iperf3 -c 192.168.0.209 -P2
Connecting to host 192.168.0.209, port 5201
[ 5] local 192.168.0.11 port 9276 connected to 192.168.0.209 port 5201
[ 7] local 192.168.0.11 port 9277 connected to 192.168.0.209 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 335 MBytes 2.79 Gbits/sec
[ 7] 0.00-1.01 sec 335 MBytes 2.79 Gbits/sec
[SUM] 0.00-1.01 sec 670 MBytes 5.58 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.01-2.00 sec 338 MBytes 2.85 Gbits/sec
[ 7] 1.01-2.00 sec 337 MBytes 2.84 Gbits/sec
[SUM] 1.01-2.00 sec 674 MBytes 5.70 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 334 MBytes 2.80 Gbits/sec
[ 7] 2.00-3.00 sec 297 MBytes 2.49 Gbits/sec
[SUM] 2.00-3.00 sec 631 MBytes 5.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 332 MBytes 2.79 Gbits/sec
[ 7] 3.00-4.00 sec 326 MBytes 2.74 Gbits/sec
[SUM] 3.00-4.00 sec 658 MBytes 5.52 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 301 MBytes 2.52 Gbits/sec
[ 7] 4.00-5.00 sec 299 MBytes 2.51 Gbits/sec
[SUM] 4.00-5.00 sec 600 MBytes 5.03 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 336 MBytes 2.81 Gbits/sec
[ 7] 5.00-6.00 sec 331 MBytes 2.78 Gbits/sec
[SUM] 5.00-6.00 sec 667 MBytes 5.59 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 345 MBytes 2.89 Gbits/sec
[ 7] 6.00-7.00 sec 341 MBytes 2.86 Gbits/sec
[SUM] 6.00-7.00 sec 686 MBytes 5.75 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 296 MBytes 2.49 Gbits/sec
[ 7] 7.00-8.00 sec 292 MBytes 2.45 Gbits/sec
[SUM] 7.00-8.00 sec 589 MBytes 4.94 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 330 MBytes 2.77 Gbits/sec
[ 7] 8.00-9.00 sec 328 MBytes 2.75 Gbits/sec
[SUM] 8.00-9.00 sec 658 MBytes 5.52 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 348 MBytes 2.92 Gbits/sec
[ 7] 9.00-10.00 sec 338 MBytes 2.83 Gbits/sec
[SUM] 9.00-10.00 sec 686 MBytes 5.74 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 3.22 GBytes 2.76 Gbits/sec sender
[ 5] 0.00-10.05 sec 3.22 GBytes 2.75 Gbits/sec receiver
[ 7] 0.00-10.00 sec 3.15 GBytes 2.70 Gbits/sec sender
[ 7] 0.00-10.05 sec 3.15 GBytes 2.69 Gbits/sec receiver
[SUM] 0.00-10.00 sec 6.37 GBytes 5.47 Gbits/sec sender
[SUM] 0.00-10.05 sec 6.36 GBytes 5.44 Gbits/sec receiver
iperf Done.
C:\Temp>
-
Actually this is all getting rather confusing, Iperf3 from the X99 to the server generally seems to be slower than from the X570 PC, and I've just discovered if I do -R -P2 from the X570 I can hit an average of 8.4Gbps and even 9.37Gbps, then another time it will barely manage 4.16Gbps, most odd.
-
I run a more windows optimised test, which equates to around 9Gbps ;D
C:\NTttcp>ntttcp.exe -s -m 8,*,192.168.0.10 -l 128k -a 2 -t 15
Copyright Version 5.33
Network activity progressing...
Thread Time(s) Throughput(KB/s) Avg B / Compl
====== ======= ================ =============
0 14.999 144496.033 131072.000
1 15.000 144989.867 131072.000
2 14.999 144922.728 131072.000
3 14.999 144956.864 131072.000
4 15.000 145058.133 131072.000
5 14.999 144538.703 131072.000
6 14.999 145221.415 131072.000
7 14.999 144547.236 131072.000
##### Totals: #####
Bytes(MEG) realtime(s) Avg Frame Size Throughput(MB/s)
================ =========== ============== ================
16972.750000 15.000 1456.187 1131.517
Throughput(Buffers/s) Cycles/Byte Buffers
===================== =========== =============
9052.133 2.438 135782.000
DPCs(count/s) Pkts(num/DPC) Intr(count/s) Pkts(num/intr)
============= ============= =============== ==============
16160.867 2.489 21234.200 1.894
Packets Sent Packets Received Retransmits Errors Avg. CPU %
============ ================ =========== ====== ==========
12221792 603255 16 8 23.582
C:\NTttcp>
-
iperf3 can be weird sometimes, I have seen it report lower than actual file transfers before.