Kitz Forum
Broadband Related => Broadband Technology => Topic started by: XGS_Is_On on August 24, 2022, 03:39:46 AM
-
So I'm fortunate enough to have uncapped XGSPON now. Whatever's left from other folks on the PON I can draw on.
As the geeks may be aware XGSPON is symmetrical 10G with compulsory FEC taking it down to about 8.5G then the various overheads at Ethernet and IP layer.
I'm seeing 8.2-8.3 Gbit/s reliably. Bufferbloat is good - joys of 8.3G on a 10GbE port.
It just works and works well. Any questions I'll try and answer them however it's basically exactly the same as GPON just symmetrical and bigger numbers.
-
I forget, what alt-net are you on again?
-
YouFibre via Netomnia.
-
YouFibre via Netomnia.
I am very encouraged to hear their network is strong even though you can pull 8.3Gb through it. As soon as it is available here I'll be signing up for 1Gbps symmetric, I cannot really justify 10Gbps at this point (not to say I can't afford it, but would have to spend a lot of time and effort upgrading everything else to support 10Gbps), however I applaud the trailblazers who are going that route.
-
Yes, I didn't have the expense of upgrading anything as I made the decision to have 10GbE available to all wired devices when I put the home network in, which was lucky!
I have, however, brought in some 25G now, just to a couple of devices. I don't plan on going to 25G across the board for a while: those switches are expensive.
More likely I'll get 100G kit when 50GPON becomes a thing, so around 2025 when my contract is up. :fingers:
-
I will be looking with curiosity to see if the the alt-nets have the sense to use NBASE-T ONTs.
My current, relatively cheap router, should be able to handle 2.5Gbit (although if it can handle this over PPP I'm not sure).
While its not essential and Gigabit symmetrical is really my aim, if they offered a 2Gbit or 2.5Gbit service that would be a really nice sweet spot for years to come I think.
Its rare I can even saturate 10Gbit on the LAN never mind WAN, as you need high-end NVMe drives at both ends as the normal affordable ones hit their SLC cache limit so fast that unless your file transfer is smaller than this cache, you might as well have just moved the file at a slower more consistent speed.
-
I just bought some new switches, all GbE with PoE for AP's. Somehow, I think I should have gone for 2.5GbE ones and kept up my tradition of having 2 x connections.
I suppose I could do 1000/1000 with YF and a slower package for failover...
Or another one of those £10 SIM cards :)
-
Yeah that's why I just went for it with the MS510TXUP. It was a bit pricey, but I can't see it needing to replaced for a long long time, maybe never.
Incidentally I have:
- Netgear MS510TXUP (Server/NAS 10Gbit DAC, WiFi 2.5Gbit [PoE], Router 2.5Gbit [PoE], main PC 10Gbit and 10Gbit to other switches)
- Netgear MS510TXPP (Fibre to main switch 10Gbit, Gaming PC 10Gbit, Games Consoles, MiSTer Multisystem [PoE], Bluray player, TV, etc)
- Netgear GS110EMX (stuff that mostly doesn't need much bandwidth)
It happened this way as I started with the latter, and then upgraded as newer models came out, replacing existing unmanaged Gigabit switches.
-
Steam appears to have a ~300MB/s limit in place which is probably a good idea given how their CDN works.
Nothing gets close to saturating the entire thing other than speedtests.
I'm starting to take advantage of it a bit more as time goes on. The port forwarding rules on the routers are getting more in number.
-
Steam appears to have a ~300MB/s limit in place
Yup, already seen that.
https://youtu.be/wWltASCJO-U?t=1067
-
@Alex where did you get your latest switch from?
[apologies for going off-topic]
-
https://www.broadbandbuyer.com/products/42122-netgear-ms510txup-100eus/
It came in stock very briefly in February so I grabbed it at £571.21 in (I wouldn't usually pay that much, but as this may last me a lifetime it was worth it), the price is creeping up due to stock availability.
The big deal was its relatively quiet with a high PoE budget, as there is nowhere in the house to put whiney cheaper ex-business switches, plus they don't support PoE++ which I figured might be useful in the future.
-
Something I need to fix in my setup I reckon but pretty good.
(https://www.speedtest.net/result/c/ca88c092-a505-432e-bd7c-3211484a3cff.png)
-
Asymmetric DS/US ? :-\
-
Take any of it with a pinch of salt given the server might only have 10Gbit in total.
The only way to test 10Gbit is to fire up multiple concurrent tests to different servers and monitor the traffic at the router.
-
iperf ?
-
Iperf seems kinda unpredictable for me, it often comes up short compared to browser speed tests.
-
Add more threads and ensure you've enough CPU to handle it. Far more reliable than browser tests.
-
Agrees with XGS.
-
Add more threads and ensure you've enough CPU to handle it. Far more reliable than browser tests.
I have some of the most powerful hardware on the market and increasing threads doesn't help when the problem occurs.
Its not that it universally doesn't work, its inconsistent and doesn't seem to be related to threads or CPU power.
-
Strange. iPerf3 is what we use in work to test enterprise networks from edge to edge. Controlling both the receiver and sender gives way more visibility than an Internet speed test someone else owns.
It's also what I've used troubleshooting connections between users and ISPs. It's very simple code. It uses the same APIs everything else does to open sockets and push data down them. The two sides exchange a bit of telemetry so there is a clearer picture of link quality.
Far less to go wrong than Ookla. Ram enough connections through something and it'll certainly look consistent as any issues will be compensated by other connections ramping up to cover up deficiencies but how realistic a guide that is is debatable.
At home I run these tests from clients not the edge device to keep the test in the same datapath and IP chain as my traffic. Work software can manipulate which path it takes depending on source interface specified. I would like to think that our business isn't using inconsistent metrics with our customers.
-
Yeah its weird, most of the time it works great but there are just some outlier scenarios where it gives odd results that are inconsistent with the real-world performance.
Its still my primary tool for testing how the internal network is performing though, but I don't 100% rely on it - not that you should ANY synthetic test as it doesn't tell you how different protocols might react.
I actually stumbled onto SSHFS while migrating my VPS over to one that supports newer kernels with the security mitigations. Was really surprised to see it comfortably maxed out my upstream while moving the Virtualmin backup files over.
-
Does UDP iperf3 work for you? Without destroying civilisation, that is?
-
I'll have to try if/when I encounter the problem again.
-
Good plan. I've just been doing some work and happily saturated 3 links consisting of 2 x 10G and 1 x 25G with it reliably, so not sure why it's problematic for you.
-
Don't get me wrong, 99% of the time it works as expected, but the fact its not 100% is concerning as in the past I've seen it claim it was doing faster than the link it was running over. Of course that might have been a bug long since squashed, but it highlights how you should never rely on just a single tool for testing.
-
I had a big problem with it, and with my own ignorance, when I tried UDP mode across the internet. Nearly destroyed the civilised world, as I said. So I need to find out what I should be doing before I go near that again. And I can’t see any point in using it in TCP mode, given my objections to the use of TCP in speedtesters, where imho they only measure TCP, not the actual path.
-
Most of the time all you want to test is TCP, as its where the majority of the traffic is and where you can identify single-thread speed issues.
I've just been trying to test between my desktop and server over 10Gbit using UDP on the LAN and it seems to become CPU bottlenecked on an i5 8600K at 4.2Gbit. I don't understand this when TCP can do 9.4Gbit absolutely fine and surely TCP should use more CPU than UDP?
-
TCP offload engine would potentially account for some of the difference. No UDP offload in network cards or drivers so all CPU-based.
-
Okay this is weird, sending from a Ryzen 9 5950X (Win11) to an Intel i5 12700F (Linux) I can push 9.57Gbit with a 1M window size. Receiving it only does 3.4Gbit. Increasing the windows size doesn't help, although reducing it tanks the receive speed to 4.4Gbit.
However if I run the iperf3 server on Windows, performance tanks both ways.
I think I'm going to give up on this and just say its useless for testing in UDP mode, which strengthens what I said before - we test TCP for a good reason.
This does highlight something though, wont this be a problem with QUIC given the drivers will only see it as UDP traffic? Or is this mostly down to a bad implementation in iperf3?
-
There has to be something odd about the UDP implementation.
-
Ah. I don't use iPerf via Windows.
-
Is it safe / legal / allowed /sensible to do iperf/UDP over the internet ? I tried it over IPv6 UDP between my hosted Raspberry Pi out on the internet (hosted by Mythic Beasts) and my iPad here, and all hell broke loose. It just shows that I didn’t know what I was doing.
-
Yeah, sending from your end to the Internet is probably okay but from the Internet to you could be rather problematic unless you specify the bandwidth correctly. Although by default it only pushes 1Mbit, presumably to minimise the chances of someone accidentally killing the network on their first try.
That said I can't get it to work from either of my VPS over 3Mbit, one at IONOS and one at Mythic Beasts. It also throws an error if I try to change the socket buffer size.
-
Before I try again in future, I’ll ask the community to sanity check my command line parameters.
-
Yeah I can't get my head around it. I tried changing the buffer length but it rejects the values even though they are in range according to the documentation.
-b, --bandwidth n[KM] Set target bandwidth to n bits/sec (default 1 Mbit/sec for UDP, unlimited for TCP).
If there are multiple streams (-P flag), the bandwidth limit is applied separately to
each stream. You can also add a ’/’ and a number to the bandwidth specifier. This
is called "burst mode". It will send the given number of packets without pausing,
even if that temporarily exceeds the specified bandwidth limit.
-l, --length n[KM] The length of buffers to read or write. iPerf works by writing an array of len bytes
a number of times. Default is 128 KB for TCP, 8 KB for UDP.
-w, --window n[KM] Sets the socket buffer sizes to the specified value. For TCP, this sets the TCP
window size. (this gets sent to the server and used on that side too)
iperf3 -ub 4M -c <IP>iperf3: error - unable to read from stream socket: Resource temporarily unavailable
iperf3 -ub 3M -c <IP>
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 330 KBytes 2.70 Mbits/sec 240
[ 4] 1.00-2.00 sec 367 KBytes 3.01 Mbits/sec 267
[ 4] 2.00-3.00 sec 366 KBytes 3.00 Mbits/sec 266
[ 4] 3.00-4.00 sec 366 KBytes 3.00 Mbits/sec 266
[ 4] 4.00-5.00 sec 367 KBytes 3.01 Mbits/sec 267
[ 4] 5.00-6.00 sec 366 KBytes 3.00 Mbits/sec 266
[ 4] 6.00-7.00 sec 366 KBytes 3.00 Mbits/sec 266
[ 4] 7.00-8.00 sec 367 KBytes 3.01 Mbits/sec 267
[ 4] 8.00-9.00 sec 366 KBytes 3.00 Mbits/sec 266
[ 4] 9.00-10.00 sec 366 KBytes 3.00 Mbits/sec 266
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 3.54 MBytes 2.97 Mbits/sec 0.018 ms 0/2637 (0%)
[ 4] Sent 2637 datagrams
Ah, it seems the block size has to be lower than the MSS?
iperf3 -u -l 16K -c server
warning: Warning: UDP block size 16384 exceeds TCP MSS 1408, may result in fragmentation / drops
iperf3 -u -l 1408 -b 900M -c server
Connecting to host server, port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 102 MBytes 859 Mbits/sec 76304
[ 4] 1.00-2.00 sec 103 MBytes 865 Mbits/sec 76798
[ 4] 2.00-3.00 sec 105 MBytes 878 Mbits/sec 77965
[ 4] 3.00-4.00 sec 102 MBytes 852 Mbits/sec 75641
[ 4] 4.00-5.00 sec 82.5 MBytes 692 Mbits/sec 61472
[ 4] 5.00-6.00 sec 88.1 MBytes 739 Mbits/sec 65593
[ 4] 6.00-7.00 sec 97.2 MBytes 815 Mbits/sec 72365
[ 4] 7.00-8.00 sec 103 MBytes 865 Mbits/sec 76825
[ 4] 8.00-9.00 sec 97.5 MBytes 818 Mbits/sec 72579
[ 4] 9.00-10.00 sec 83.0 MBytes 696 Mbits/sec 61815
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 963 MBytes 808 Mbits/sec 0.021 ms 325/717357 (0.045%)
[ 4] Sent 717357 datagrams
The packet loss is probably contention on the VPS host at Mythic Beasts rather than the broadband.
-
In IPv6 as you know, there is no fragmentation in middleboxes, ie intermediate nodes in the path. Fragmentation only happens at the sending machine, and it is all cool. Indeed some good hardware can handle ‘large send’ offload, maybe even for IPv6 and maybe even for UDP now. You just give it a huge block of data and tell the hardware to get on with shifting it out. I would hope that with UDP the combination of hardware and software in your own sending machine doesn’t overdrive the max egress rate of the interface and first link. I would like to try this large block thing with UDP and compare it with small blocks of say 1280 bytes and 1500 bytes. I feel a lot happier about IPv6 fragment handling than IPv4, because the software involved is likely to be cleaner having been redesigned for IPv6. Evil receiver-end firewalls may remain a problem though.
-
IPv6 is interesting, it claims the MSS is higher - how?
iperf3 -u -l 1408 -b 900M -6 -c ping6
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 90.0 MBytes 755 Mbits/sec 67054
[ 4] 1.00-2.00 sec 97.4 MBytes 817 Mbits/sec 72500
[ 4] 2.00-3.00 sec 100 MBytes 841 Mbits/sec 74706
[ 4] 3.00-4.00 sec 104 MBytes 872 Mbits/sec 77418
[ 4] 4.00-5.00 sec 94.4 MBytes 792 Mbits/sec 70307
[ 4] 5.00-6.00 sec 106 MBytes 889 Mbits/sec 78907
[ 4] 6.00-7.00 sec 107 MBytes 894 Mbits/sec 79405
[ 4] 7.00-8.00 sec 95.1 MBytes 798 Mbits/sec 70827
[ 4] 8.00-9.00 sec 104 MBytes 869 Mbits/sec 77187
[ 4] 9.00-10.00 sec 106 MBytes 888 Mbits/sec 78799
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 1003 MBytes 842 Mbits/sec 0.022 ms 307/747100 (0.041%)
[ 4] Sent 747100 datagrams
I have absolutely no idea what is going on here:
iperf3 -u -l 3000 -b 900M -6 -c ping6
warning: Warning: UDP block size 3000 exceeds TCP MSS 1428, may result in fragmentation / drops
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 98.4 MBytes 825 Mbits/sec 34378
[ 4] 1.00-2.00 sec 108 MBytes 910 Mbits/sec 37910
[ 4] 2.00-3.00 sec 110 MBytes 920 Mbits/sec 38321
[ 4] 3.00-4.00 sec 111 MBytes 931 Mbits/sec 38788
[ 4] 4.00-5.00 sec 106 MBytes 892 Mbits/sec 37154
[ 4] 5.00-6.00 sec 109 MBytes 914 Mbits/sec 38065
[ 4] 6.00-7.00 sec 105 MBytes 878 Mbits/sec 36581
[ 4] 7.00-8.00 sec 110 MBytes 922 Mbits/sec 38435
[ 4] 8.00-9.00 sec 108 MBytes 907 Mbits/sec 37792
[ 4] 9.00-10.00 sec 107 MBytes 901 Mbits/sec 37521
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-10.00 sec 1.05 GBytes 900 Mbits/sec 1.205 ms 225480/225491 (1e+02%)
[ 4] Sent 225491 datagrams
-
1428 implies a header which is 40+20+12 = 72 if your IP MTU is 1500, which is IPv6 + TCP + TCP_timestamp, and that should be 20 bytes longer than IPv4. So your IPv4 MSS should be 20 bytes less than IPv6 of course.
-
Think you've that the wrong way around, Weaver. The headers will be 20 bytes larger than IPv4 so the v4 MSS will be 20 bytes larger than v6. Lower headers, more room for payload, higher MSS.
-
Think you've that the wrong way around, Weaver. The headers will be 20 bytes larger than IPv4 so the v4 MSS will be 20 bytes larger than v6. Lower headers, more room for payload, higher MSS.
That's what I thought which is why its so confusing.
-
Now I reread it, what I wrote was ambiguous. I was referring to the headers to begin with, not the MSS. So bad writing.
-
Bit of iPerf3 action attached.
Some folks might be interested in how little data is going in the other direction - 8 Gb/s of TCP one way required only 19 Mb/s of acknowledgements. I've read occasionally how Virgin Media are cutting it very close with their asymmetry at times but not really the case unless there's something wrong with the TCP stack.
-
Some folks might be interested in how little data is going in the other direction - 8 Gb/s of TCP one way required only 19 Mb/s of acknowledgements.
Interesting. (If I was asked to predict the result I would have had to decline.) Perhaps Weaver will perform one of his calculations to see if he gets agreement with your observed result.
-
Be interesting. 660,000 packets a second being acknowledged by 34,000.
Lots of cumulative acknowledgement going on with selective acknowledgment plugging any gaps in there.
Packet wise, 20:1 give or take. Bandwidth wise 400:1 give or take. The acknowledgements are running on a sliding scale and there are selective acknowledgments requesting and acknowledging retransmissions.
Bandwidth wise expect 40:1500 = 1:37.5 if every packet being a acknowledged so clearly that's closer to an acknowledgment per 11 packets in terms of bandwidth.
Packet-wise it reflects some background noise, some selective acknowledgements to trigger retransmissions and some overlapping cumulative acknowledgements.
400:1 explains how Virgin Media sold 53 download on 1.75 upload and it worked fine :)
-
Be interesting. 660,000 packets a second being acknowledged by 34,000.
400:1 explains how Virgin Media sold 53 download on 1.75 upload and it worked fine :)
Yes, may have worked a while back, before the proliferation of multi device households.
Yes interesting, but I don't think typical these days. The typical household would have multiple users using multiple devices, communicating to multiple services on multiple networks, all with packets of different sizes.
Just for kicks I looked at my router interface (uptime 25 days), I see following ratios:
packets Download:1 Upload:1.3
bytes Download:1 Upload:2.8
-
Was more a reference to the concerns some had about 1.75 not being enough to download at 53 due to acknowledgements overwhelming the upload than a reference to individual usage patterns.
-
On gig1, the ack's are nowhere near close to saturating the upstream, I think I remember you telling me a while back VM do this magic and it seems to work reasonably well.
At 963mbps down in dumeter, it reported 8mbps up of ack bandwidth. Approx 120:1 ratio. But thats just to my hub right, I think you explained the magic is done 'after' I see it, so will be less than actual 8mbit of US used?
-
Yes. Standard for DoCSIS kit to act as a layer 4 proxy. Ack suppression.
-
Agreed. Assuming one TCP ACK PDU for two DATA PDUs typical. Assuming the worst case inefficiency, IPv6+TCP with timestamps would be a 72 byte IP PDU containing a TCP ACK vs 1500 byte data IP PDU say. I make that 2.4% IP ACK PDUs / IP DATA PDUs byte count ie bandwidth. So bandwidth 1/x = 20.83 approx. Now SACK could of course increase that latter figure, as would IPv4 or no timestamps.
If they are doing L4 proxying then that’s potentially very naughty as the L4 reliability guarantee is broken. Unless of course the proxy is like a ‘custodian’ in the sense of the term in the bundle protocol, and in that case there needs to be a chained guarantee of successful forwarding. Not impressed by that at all.
-
The cable modem ack suppression just accumulates acknowledgments. As it's a TDMA upstream it'll have a bunch of acknowledgements in its buffer that are unnecessary to send. It accumulates them and acknowledges all received data in one message. It forwards selective acknowledgements as necessary.
Acknowledging 2:1 throughout is entirely unnecessary. Only need to acknowledge with enough frequency to ensure windows don't fill. Other side is only concerned by the quantity of unacknowledged data, not by how many PDUs it came in. 2:1 only really useful at the start if slow start is being used to ensure windows grow.
-
Agrees with XGS, I didn’t put it very well at all. What I was trying to say, and failed miserably, is that generating fake ACKs is naughty. Delaying them so that you can ACK for more recvd data is entirely sensible, as XGS says, as long as you don’t leave it way too late and blow it.
-
CityFibre are now blowing fibre into their ducts serving me. Might well replace my Openreach backup with CityFibre. Not least because it means a diverse physical path following the CityFibre ducting rather than the Openreach ducts once out of the, small, development.
-
Asymmetric DS/US ? :-\
It's still >:D awesome!
-
I'm sure some would be offended that it doesn't even average 0.1% utilisation but it's good to have the capacity there on the rare occasion I need it for sure.
-
CityFibre are now blowing fibre into their ducts serving me. Might well replace my Openreach backup with CityFibre. Not least because it means a diverse physical path following the CityFibre ducting rather than the Openreach ducts once out of the, small, development.
Will be interesting to see if you get activated earlier than my street, my street was done months ago. Please keep us updated. :)
-
On Tuesday they were enjoying the weather building the contents of the primary nodal cabinet: the one across the road. The secondary that serves us is already built.
I presume next will be a splicing crew, then the blowing of fibre to light CBTs.
-
Because I'm an idiot and I've some 25G ports that need terminating I've purchased a QNAP QSW-M5216-1T. It is disappointing. It's had zero software updates since release and its support and status information for LAGs is unacceptable given the mass of information and options on the NAS they make. Indeed it could actually hamstring their NAS.
They are almost certainly using a Marvell Prestera switch chip so could be doing it better with hardware acceleration.
For now it's acceptable though this won't last much longer as the WAN breaches the 10G mark.
Wouldn't recommend that switch. A great bit of hardware with awful firmware support.
-
Because I'm an idiot and I've some 25G ports that need terminating I've purchased a QNAP QSW-M5216-1T.
. . .
Wouldn't recommend that switch. A great bit of hardware with awful firmware support.
Thank you for the latest update and the advice as what to avoid.
-
Will be interesting to see if you get activated earlier than my street, my street was done months ago. Please keep us updated. :)
:hmm:
(https://iili.io/H0BxcGe.md.jpg)
-
I guess they work faster when is already competition ;)
I got a vodafone letter through the door, but when ringing them they said its automatic when they notified of build and they know nothing else, so still the 9+ months I was quoted (when work started several months ago).
-
I guess they work faster when is already competition ;)
I got a vodafone letter through the door, but when ringing them they said its automatic when they notified of build and they know nothing else, so still the 9+ months I was quoted (when work started several months ago).
Not round here, we have VM and OR, CF don't seem to be doing any jobs on my exchange right now. I think its more they realised they can't keep up and switched to the wealthier estates. I'll be livid if we drop out of the plan.
-
Not round here, we have VM and OR, CF don't seem to be doing any jobs on my exchange right now. I think its more they realised they can't keep up and switched to the wealthier estates. I'll be livid if we drop out of the plan.
Here I see the work spreading to many other parts of the city, I think they going to turn on the remaining 75% all in one go, they well past the point of turning on in blocks now here, no new postcodes activated for about a year, but more works going on in new areas. The most recent stuff through the letter box is offering people a chance to win a PC if they register interest, pointless as when I registered my interest 2 years ago, I havent had any automated emails giving me updates other than the misleading one saying its coming soon when its 9+ months away.
-
I guess they work faster when is already competition ;)
I got a vodafone letter through the door, but when ringing them they said its automatic when they notified of build and they know nothing else, so still the 9+ months I was quoted (when work started several months ago).
They are working out in a spiral from their FEX. Netomnia know how they work and are preempting. Openreach are also busy.
https://bidb.uk/ Pop in 340 WF2 8JQ. Not my address I might add but not too far away. Most of the work around there is done and they're now having a competition over Horbury, Ossett and Gawthorpe with some VM works too.
-
That's interesting as they started top-left of the exchange here, I saw them moving clockwise round it then suddenly bam they are far-left. I suppose that could still be a spiral (if this is how they do it everywhere) if were closer and they are still on the outer edge.
I can't check on bidb.uk right now sadly:
Despite offering assistance to fetch the data that Better Internet Dashboard provides, Grepsr are sustaining an attack on our APIs.
For this reason, this feed is disabled.
-
Working now.
-
With apologies to all the folks who are stuck on slow services.
You really don't use it that much more just because you can.
-
With apologies to all the folks who are stuck on slow services.
You really don't use it that much more just because you can.
Speak for yourself. :P
-
That's a fair chunk!
Mine below but don't really have any stats from the Sky line I just ceased - so double all the figures below (very roughly) as I load balanced.
-
We all use exactly the amount we need to. I'm not generating much to back up and don't run any services reachable from outside.
That and my numbers are more standard network graphs of capacity in use at any one time rather than total throughput.
-
Sync rates:
=== Links’ sync rates: ===
Link #1: downstream: 2.763 Mbps, upstream: 692 kbps
Link #3: downstream: 3.242 Mbps, upstream: 755 kbps
My AA speedtest:
(https://i.postimg.cc/Jz6SX0xg/ACBF1973-B40-F-4890-8-AED-F88-E84894-BEC.jpg)
My usage, reported by AA:
(https://i.postimg.cc/xjFP2msc/C4205-B32-F944-4-C34-82-D2-50-EB9761-A884.png)
-
Should mention my usage I don't host files I share with others, backups are incremental and usage is pretty average generally.
That's also of course a snapshot of a single week.
On another matter, availability, just watched an Openreach cherry picker on the main road finding some room on a pole that already had CityFibre and Netomnia on it for a CBT have a van driven by the contractor for Netomnia go past.
-
More impressed that Weaver is pulling hundreds of GB's on 5Mbps.
-
We all use exactly the amount we need to. I'm not generating much to back up and don't run any services reachable from outside.
That and my numbers are more standard network graphs of capacity in use at any one time rather than total throughput.
Yeah I'm definitely not a typical case. As I'm always at home I spend a lot of time browsing photo sites and in that case I end up using more because instead of waiting for a photo to load, its practically instant, so naturally I end up browsing faster and using more. When I get into a fibro-fog I can spend hours on this as I'm unable to focus on anything else.
Also if I see an interesting YouTube video I'm much more inclined to archive a local copy as it wont saturate my connection while doing so. Its easier than fighting with them constantly adjusting the default to 480p. Plus I'm a lot more inclined to experiment with GamePass games now they don't take so long to download.
I'm also more inclined to just leave torrents seeding for the same reason, it has zero impact on my other usage.
-
I'm also more inclined to just leave torrents seeding for the same reason, it has zero impact on my other usage.
All Linux ISOs, and public domain material, natch. ;)
I tried seeding a Linux ISO torrent and couldn't get enough peers connected to get anywhere near stressing the connection. It was kinda disappointing if I'm honest.
-
Asymmetry issue resolved. Currently not at maximum speeds as running through an SD-WAN VM doing DPI on a not especially fast server.
https://www.speedtest.net/my-result/d/b1ae3097-bd5a-4793-b04b-db64e9ce0dc0 is where we are.
-
Asymmetry issue resolved.
I had previously pawed --
Asymmetric DS/US ? :-\
Now looking at the latest result, I'll call that symmetric. :)
-
I've also had to do some rearrangement due to incompatibility between wife's work VPN and, well, anything.
Have had to go from the previous design to two layers of routing to make sure her traffic always takes the backup.
Looks similar to a datacentre/hubsite now logically. 4 routing nodes forming 2 layers in a mesh, chatting to one another via BGP.
So much more complicated than it was, but a very familiar build.
-
I suspect this will be a very familiar kind of set up to at least one of the members of the forum :)
iBGP to the SD-WAN not allowed so had to do a little something.
-
Some more diagrams for me to study (and archive)! :D
-
All Linux ISOs, and public domain material, natch. ;)
I tried seeding a Linux ISO torrent and couldn't get enough peers connected to get anywhere near stressing the connection. It was kinda disappointing if I'm honest.
Yeah its a chicken and egg situation. I used to download ISOs over torrent but it took longer than from the website due to lack of peers, which means I don't bother any more, meaning less peers.
Its very rare no matter what you download over torrent to hit line rate. The problem with home broadband getting more and more asymmetrical over time.
-
I'm talking about the upload speed not getting close, bizarrely. However many slots I opened up there just weren't enough speedy connections.
EDIT: While I'm here I should mention that the diagrams above weren't produced just for you guys: going towards both an internal knowledge base and my own academic work. Nothing personal!
-
I'm talking about the upload speed not getting close, bizarrely. However many slots I opened up there just weren't enough speedy connections.
Depends on the content but generally it is very peaky when its new, but then there are a lot more people seeding so even hitting my 4000KiB torrent upload limit doesn't necessarily happen. Especially as anything new will likely also be hosted on a dedicated seedbox. Then over time there will be more traditionally hosted mirrors available, and people will assume nobody is going to be seeding so torrents would be a last resort.
I also think a lot of people who torrent choke their downloads and have tons of other downloads going at the same time.
I suspect high seas private torrents would be very different, but so not going there.
-
On a seedbox I rarely find my upload anywhere near its limit, the majority are people from america seemingly on slow DSL connections or with not enough buffers for the RTT. When someone fast does connect, they will also only do so via one thread.
I have thought about will I host stuff from home (bunch of stuff not torrents specifically) on FTTP, I am someone who has always had the mindset, content should be hosted in a data centre, due to its redundancy, and always on by design. But annual costs are creeping up, IP address's in particular have quadrupled in cost in 12 months, Considerably more expensive than what the likes of AAISP supply them for. Still thinking about it, but the money saved would be used to fund new storage etc. for my NAS.
-
My usage is so low. :'(
-
You're clearly not watching enough por..... cat videos.
-
Yes very familiar to me :)
Makes mine look very simple, though that's mostly because it is now - single Firebrick doing all the routing across the 4 VLAN's at my place. Easy peasy stuff.
Need to break up some more stuff on my LAN for some research though :)
-
Didn't have any choice. There's a route for the wife's work VPN on the study router telling it to just send immediately to the comms router, where it hits the PBR and leaves the premises.
Can't do that with a single layer of routing so was order, wait, then install a new AP single homed to comms router or build a slipstream which essentially forms a DMZ.
The two routers can happily be configured with stateful firewalling on their 222 slipstream facing interfaces. :) I can at my convenience add another device into the 222 range to be DMZ firewall and be destination for everything with no firewall state as it's inbound.
If it's new, send it to....
-
Didn't have any choice. There's a route for the wife's work VPN on the study router telling it to just send immediately to the comms router, where it hits the PBR and leaves the premises.
Just trying to understand this a little more. Is your LAN all private IP networks, and you are NATing on the edge router? Or do you have some public addresses locally?
Could IPv6 solve the VPN issue, assuming all equipment was IPv6 enabled?
-
NAT on edge. I have a /29 of public IP addresses I could use on the backup. They are, where necessary, destination NAT into the LAN. There is no mixing of public and private addressing in the LAN, only IP/port mapping.
Wife's VPN is v4 only, old school, and touchy. It is fundamentally incompatible with the SD-WAN so has to be kept away else it doesn't work.
Most decent VPNs using IPSEC NAT-T will be sourced from ephemeral ports because it shouldn't really matter. Wife's employer insists on 4500 as source and destination and the SD-WAN will not do this: source port 4500 is sacred.
It was a good thing anyway. It made me build the thing properly in coherent layers instead of flat.
-
On the use of mapping:
No need to burn IPs routing them - no gateway or broadcast.
Can either forward everything, unwise, or specific ports.
The LAN is a simplification in the diagrams. There are other VLANs not featured. The DMZ using the public IPs is a /28: no public IP mapping to hosts in the subnet where the regular devices live, those are masquerade / SNAT only.
-
Carl, are you also an IPv6 fan? I’m seeing a lot of IPv4 (we all of us still love it sometimes, if truth be told).
-
I'm starting to become a fan of CGNAT seeing as I now operate an ISP network...
-
It’s just that I didn’t see any IPv6 addresses on XGS’s network diagram, but maybe doing everything internally twice is just madness?
-
I'm starting to become a fan of CGNAT seeing as I now operate an ISP network...
BAN THE HERETIC!
-
Quite right Alex. Speaking as someone with a real IPv4 address here.
-
Carl, are you also an IPv6 fan? I’m seeing a lot of IPv4 (we all of us still love it sometimes, if truth be told).
Indifferent, Sir. The backup link is dual stack however this is problematic given the difference in capacity between the two links.
Awaiting full dual stack support from SD-WAN software and ISP then I'll implement it.
-
It’s just that I didn’t see any IPv6 addresses on XGS’s network diagram, but maybe doing everything internally twice is just madness?
IPv6 that's publicly routable as delivered by regular providers would break things as it has to be tied to a single ISP. A major point of the network I describe in the diagrams is that it isn't dependent on a single ISP.
Before I implement I need a way of ensuring that a similar level of resilience is in place as with v4 NAT.
As above not opposed and will implement when available but has to deliver the same redundancy as v4 else not happening.
-
Quite right Alex. Speaking as someone with a real IPv4 address here.
I do wonder why you have more than one though?
I had a block years ago on Plusnet but never found I needed them.
-
BAN THE HERETIC!
Yes but when IPv4 costs $50 per address and you are servicing thousands of customers, you suddenly understand the need for CGNAT...
Me personally, I could not abide CGNAT as a user. I have a routed /29.
-
[Apologies for wandering off-topic. Might want to split this conversation off, my friends?]
Agreed. I have had all my machines inside a routed /26 for over ten years.
What about simply getting rid of IPv4 completely and using DNS64 + NAT64, is that viable ? (AA can do this but I think they don’t advertise it much.) And customers who want real IPv4 addresses could pay extra for them?
-
[Apologies for wandering off-topic. Might want to split this conversation off, my friends?]
For me it's fine. Nerd on.
-
Weaver due to google's daft happy eyes balls policy on their browser I only browse single stacked now. IPv6 is back on my PC but all IPv6 set to deny on windows firewall for the executable.
Had to also do same on steam (seems they inherited it with chrome framework), as without it, downloads went over 200mbit AAISP tunnel instead of gigabit.
-
Wrote this elsewhere but thought it'd be good here, too.
You see Mikrotik, QNAP, their 25G switch was a steal despite awful software, and you see a server running Aruba EdgeConnect (https://www.arubanetworks.com/en-gb/products/sd-wan/edgeconnect/). That fella in turn connects via hubs in Amsterdam, London and Warsaw to about a hundred other devices via an SD-WAN fabric, along with taking some of my Internet traffic to Amsterdam and some being allowed to break out locally.
As you can see there are two Internet connections there and the EdgeConnect uses both - the second one is sent to the secondary Internet connection via a dedicated VLAN and access port on the switch at the other end.
(https://iili.io/HEbBbDX.md.jpg)
The main home network lives in 192.168.0.0/22. I'm lazy.
There are 2 DHCP servers, one handing out 192.168.0.2-192.168.0.254 the other 192.168.2.1-192.168.2.254. Both in the same broadcast domain so both answer DHCP requests. I don't care which offer a device takes: it doesn't matter. It does mean if one dies the other can continue to provide addresses.
The 2116 and the RB5009 have a VRRP VIP of the default gateway, 192.168.0.1, and actual IPs in 192.168.1.0-255 - a range reserved for static addressing.
The 2116 receives its default gateway from the SD-WAN appliance. It has a higher metric default route to the RB5009. These are in a logically separate network, 192.168.222.0/29 in its own VLAN on the switches: the slipstream network.
The SD-WAN appliance also advertises a bunch of routes from the SD-WAN fabric to both 2116 and RB5009.
The RB5009 has two routing tables, a main one for most traffic and one for traffic arriving from the SD-WAN. There are also two BGP sessions, one for each routing table to ensure that traffic heading to the SD-WAN gets there whatever address it arrives on knowing that it'll be encapsulated and then can be sent out the RB5009 Internet connection via the tunnel.
The alternative table besides those routes on the SD-WAN sends everything out of its connection. If traffic has reached it there is a problem with the 2116.
If after it's sent to the SD-WAN via the Slipstream it still has nowhere to go when it comes back via the WAN and VLAN 4 it goes into the alternate table that sends everything out of its Internet connection to ensure that there's no loop. Traffic hitting there should be destined for the Internet always. The SD-WAN learns the internal networks via the BGP session across the slipstream.
Lastly as noted in the diagram both 2116 and RB5009 advertise to the SD-WAN, however the 5009 uses AS-Path prepending so it advertises a longer path than the 2116: it'll only be used if the session to the 2116 goes down.
That happens if VRRP is still as normal traffic will traverse both routers on its way to the SD-WAN and will come back to the client via the 5009 directly.
It's more of a lab than a home network, it's obviously ridiculously complicated for one.
-
See the attachment for some of the detail I can see now. Great product for network visibility.
-
I'm starting to become a fan of CGNAT seeing as I now operate an ISP network...
Which variety are you using?
-
Design changed again, to better replicate a high availability pro DC set up.
I forgot I had an RB5009 next to the existing one. It's now alive, peered with everything via BGP and doing what it should be. It's now an edge router: it advertises that is has a route to the Internet available though it's not as good as the other guy. Doesn't even use its own Internet link for own traffic: goes across the slipstream to the primary.
The previous 5009 sits waiting for VRRP to send things its way as the 2116 isn't working then will use a 2.5G port to talk on the 222 train and know whether to send across the switching plane to the sexay server or to send to its buddy 5009.
I ran out of 10G ports so, rescued from eBay, a CRS309-1G-8S-IN is ready to rumble and has been able to offload everything bar a couple of clients running GbE from the t existing switch.
If I'm honest that switch's lifespan isn't looking too promising. It's going to be replaced by some cheap unmanaged gigabit switch.
-
Do the rest of the users in the household experience a highly available network with constant tinkering/engineering of a home network lab? :)
-
Do the rest of the users in the household experience a highly available network with constant tinkering/engineering of a home network lab? :)
I often break mine, I just have to inform my wife and she's usually fine with some quick tinkering :)
-
Apart from the brief blips when I moved cabling from one switch to another there was zero downtime to either this or the earlier activity.
I work on one half at a time and point all the client machines to the other half. I logically separate them by switching off the routing protocol between them as necessary.
Even installing a new router as I did wasn't noticed because I did it on the backup. All our traffic was happily flowing through the main router and the YouFibre connection.
It's very easy - change a VRRP priority number to change which is primary and secondary then hit the disable button on BGP sessions and the job's done. Everything goes through the backup. Put them back the way they were and the primary is back.
-
I inform my wife when I’m about to break the network. Usually she doesn’t see any problem when I’m merely taking one modem down, as with my IP-bonded setup traffic is very quickly redirected to the other lines, within 5 secs or so.
-
When she's working from home now my better half has her own access point. I can do what I want with the main one without breaking her work connection.
Obviously a hit moving to a different Internet connection. Unavoidable unless I were to buy PI space and persuade my ISPs to route it to me.