Kitz Forum
Computers & Hardware => Networking => Topic started by: niemand on January 25, 2020, 12:52:48 AM
-
Would you good people be bothered by what I've designed for the new place?
Offering the option as I know the odd person will be interested but not going to through a load of diagrams and junk at you all for no reason.
This was multi-tasking. I was upgrading my lab anyway to work on higher capacity customer networks so this worked well alongside.
There is nothing complicated in here - KISS, always.
-
Yes, please. (Kittehs are always curious.)
-
Then here you are. Physical diagram only - logical one on request.
https://drive.google.com/open?id=1143N2M-oCpu1Fk89fHxc3_ex9B8OB14S
EDIT: I am aware that there are single points of failure on the left of the diagram. It's the living room and I can always install a wireless bridge in there connecting to the main AP if the kit proves problematic :)
BTW the secondary router, a Mikrotik hEX, will be running VRRP with the pfSense box LAN interface. In case it loses reachability it will assume the box is down and will gratuitously ARP the LAN gateway alongside running a script to activate PPPoE sessions to the providers.
Once the pfSense takes over via VRRP again it downs its PPPoE sessions.
The VLAN arrangement means that the only time the wireless bridge will be used is in case of failure of one of the right-hand switches. The pfSense box keeling over cuts the WAN to whatever throughput the hEX can muster but there's still 2 Gb to that hEX and the 10G LAN remains.
-
Thank you. :)
Something for me to consider when the brain is cooperating!
-
Nice to have this info, am glad you posted it. :)
-
Logical incoming. Most of this had to be done anyway as part of other things so all good.
-
https://drive.google.com/open?id=1mkAxSRC3TzTSIe7LP5BfYbl-7VMLOhrs
-
Have you calculated a likely overall power/energy consumption, or set yourself any particular goals?
-
No. Our consumption is fully carbon offset and/or renewable anyway, alongside which I receive a tax credit due to being home based.
Each switch maxes out at 18 watts fully loaded. The server running various things consumes 200 watts. All in I reckon the VM host plus all the network kit will be in the region of 400 watts.
-
Worth noting there is, in my experience, substantial variation in consumption between functionally similar kit from different manufacturers. I think some devices for example will sense idle network ports and shut them down, though - don’t know - there might be a tiny performance impact.
400W sounds a lot. As a very rough rule of thumb, for apparatus that’s on 24/7, each watt is about £1 per year in supply costs, so might be roughly £400 per year. 400W may also be enough, in a summer heatwave, to add enough heat to make a room significantly more uncomfortable.
I’ve no particular interest in carbon footprints btw, not much of a believer in all that. My only interest is that from an engineering perspective, low power & energy usage, imho adds to elegance of design. :)
-
All the 10G switches will be using SFP+ for all connectivity besides a single built-in gigabit port so if it's not in use it won't be active.
The 10GBase-T is between 3 different rooms - the ONTs, two of the switches, the backup router and one of the two wireless bridges will be in the cupboard under the stairs, just the two switches in the lounge, 10G and 8 x GE, all the rest in my home office.
I could simplify profoundly by removing all the resiliency but that's not the best idea given how critical connectivity is to the home.
Sorry to hear of your indifference to anthropogenic climate change but to each their own.
-
Sorry to hear of your indifference to anthropogenic climate change but to each their own.
That’s not what I said. I am certainly sceptical about equations based on carbon footprints because I strongly believe that such equations can be, and are, manipulated for political convenience. I am also sceptical over so-called renewable energy sources too, as I don’t believe they are always as environmentally friendly as they are cracked up to be. Whereas if we can simply reduce energy consumption in the first place as a reasonable engineering objective, the environment wins as a side-effect, and these become relatively moot points.
Basically, if I were in your shoes, I’d simply be building a target annual energy usage into my requirements, alongside all the performance requirements. Maybe you’ve already done so, apols for ruffling of feathers. :)
-
I would welcome suggestions on how to do it more efficiently without ballooning costs.
That isn't sarcasm. If there's something I've missed I would appreciate it.
Just FYI the 4 core switches will all be running STP so will be a loop free environment despite the multiple paths. The secondary router will be using SVIs rather than physical ports for everything so could bridge the two core switches in the unlikely event their DAC fails.
The only thing lacking resilience will be the living room. This could be trivially accomplished via another wireless bridge.
Power is an 'issue' however I have solutions for that, too.
The diagram is the end result of how the network will end up. Components will be added incrementally.
February - single 330 Mb connection - All Gigabit Ethernet. ONT <> Cupboard switch <> Office and Lounge switches <> Netgear R9000 with GigE uplink to Office switch. Cost - Zero, I have all the kit.
March - 2 of the 10G switches go in place, one replacing the single cupboard switch, another the office. 10GBase-T SFP+ x 3 - not enough ports available on single switch without extra SFP+, may as well get 10G as will need later and can run in GigE for now.
April - Single gigabit broadband connection. Server gets its 2 x SFP+ NIC. 3 x SFP+ DACs, pfSense installation, Netgear to Access Point mode only.
May - Second gigabit broadband connection. Second 10G switch under stairs. 1 x SFP+ DAC to cross-connect.
Whenever, it's not important: Complete resiliency, capacity upgrade to lounge. Install lounge 10G switch. 1 x 10GBase-T SFP+. Install 2 x 1000Base-T SFP+ into cupboard switches. Install hEX backup router. Install wireless bridges.
Install UPS in both office and under stairs - to connect kit under stairs and wireless AP / bridge / switches in office but not the server.
Once all of this is done I'm good for the foreseeable. At some point perhaps a second Ethernet run to each of lounge and office to allow for core switch resilience will be run, retiring the wireless bridge solution, but no rush to attack the walls. Hopefully by then a decent 10G router will be available that isn't a server and that can go under the stairs too - where it should be!
-
The paragraph at the bottom of my post is still something that's frustrating.
If there were a dedicated 2 port router that could route at line rate or something close to it I'd be content.
Don't need about 16 10Gb ports on the thing - port density is what switches are there to supply.
-
Excellent. Seems I got it about right. Should go into this stuff for a living: if I can manage a basic branch office HA network across 3 segments with very limited cabling there's hope for me yet.
-
There is nothing complicated in here - KISS, always.
As my eyes glaze over at the description of what you are doing. ;D
-
How are you load sharing your Internet connections?
-
I was originally planning pfSense however plans are subject to change and I have a better solution.
https://www.silver-peak.com/sites/default/files/userdocs/edgeconnect_host_system_requirements_r8-1_revh_april2018.pdf
I was originally going to have an EC-V behind the pfSense however this way around I've another masterplan I can complete - hosting another instance in the cloud and having my own on-demand VPN service plus a subnet of public IPv4 address space I may consume as I see fit in a DMZ.
A provider is happy to host a VM for me in their Geneva DC and will mostly charge me a base VPS fee alongside bandwidth charges at cost.
It's simpler to configure it via the EC-V and have it doing everything.
Well, it's simpler for me - YMMV.
-
> when the brain is cooperating!
It isn’t, in my case, so I might need some help/pointers. For very stupid old people.
Let us know what UPS you have your eye on?
-
Will do.
Things have been simplified somewhat by that I have minimal chance of being able to have dual-WAN at the new property.
With that in mind this entire thing is put on hold with the exception of the 10G deployment in the home office.
I'm not entirely sure how they are doing it but Openreach appear to literally be building capacity as each unit goes live and no more. The network is perpetually at capacity so no new ONT may be ordered.
I can't be bothered with the hassle and expense of getting a reactive build order placed so if Openreach really have no concept that the odd property may want multiple feeds :shrug2:
Life's too short to debate with some faceless Openreach guy about a restriction that probably doesn't exist given the components come in fixed port counts and it's actually quite impossible for the entire estate to have zero spare ports, it's a flag on a computer screen claiming there are none.
-
I was not surprised to read your latest posting on this subject. ;)
Hence my request is that you provide updates as the master-plan develops and phases thereof are deployed, please.
-
To be perfectly honest I can't be bothered with the masterplan. I have better things to do than argue with idiotic infrastructure providers that pinch pennies over ports on PON splitters.
Best guess for new build the charges are per plot and some bean counter insists on no more capacity being made available than the exact number of plots regardless of cost.
Whatever turns them on. BT / Openreach have a long history of being penny-wise and pound-stupid and that evidently continues. :)
-
I should clarify. The primary driver behind the 10G upgrade is the need for resilient connectivity at >1G.
In the absence of 2 ONTs and total WAN capacity of over a gigabit it's basically a pointless waste of money.
I am not going to buy it for the sake of it or to play with it. A network this basic won't teach me anything or be any fun to set up.
I'll hook up a 4G dongle as a backup on an active:passive basis.
Hopefully at some point another operator will rock up using PIA or their own kit.
My concerns about being stuck in an Openreach only area have proven true before I even moved in. That likely single figures pounds saved provisioning the bare minimum CBT ports is so worth it.
::)
Just to see off a possible criticism nothing to do with optical budgets. Pretty close to the exchange, 4 PONs required on standard planning rules so should be loads of ports free or they're running PONs with zero spares and a fibre break will result in far more expense than necessary.
Purely some planning rules written by someone with a rod up their hindmost per Openreach / BT / Post Office Telephones standard. Laughable that overprovisioning routinely done for copper but not optical.
Plus ca change, plus la meme chose.
-
Let me know how you get on with 4G dongle ?
-
Not a huge amount to say about that one but will let you know for sure.
-
I’m stuck at the moment with only a 3G dongle because of the limitations of various 4G dongles : they’re NAT, I can’t live with NAT; not speaking PPPoE, which my connected router prefers (dongle is plugged into a Firebrick FB2900 router). The interface between a 3G dongle and a connected router seems ideal for the Firebrick’s preferences.
-
Is the firebrick supplied in a way to lock people into aaisp services? As all of the things you just described are essentially whats keeping you at aaisp.
Now I love aaisp, however I dont think I love them enough that it would make me use ADSL with them over a much superior technology for a primary internet connection..
However maybe the firebrick is capable of more than you describe.
Have you tried these things.
1 - Asking aaisp if they can do a 4g dongle, it seems a logical step that they should be able to manage if they can do 3g.
2 - If they cannot do it is there a way to utilise another provider's 4g and routing the traffic via the firebrick still.
Surely the firebrick is not limited to pppoe, because aaisp use it for other things aside from consumer broadband, obviously the NAT thing is a preference but I have learnt over the years, NAT is not as bad as it seemed to originally be. Also if you are prepared to only use 4g on a single device, you should be able to disable NAT e.g. I have managed to use my EE 4G routed direct without NAT, it involved rooting the phone (which was already done), and messing with some firewall rules.
-
Ah, I see.
So I can easily run a combination of NAT and NO-NAT via anything as I can use my own software either side of connection.
Per earlier discussions I can have a PA range if I want but some NAT doesn't really concern so no need to worry either way.
Unfortunately not applicable to your situation, Weaver, where you are entirely dependent on A&A.
The PPPoE part, however, you can run that over anything. You will be getting some kind of IP address to encapsulate the PPP session within and get it to A&A as the mobile operator aren't going to be inspecting PPPoE. They'll use an IP network to send all your traffic to A&A encapsulated in whatever the Firebrick puts around it. I doubt A&A quite have the scale to be running their own MVNO with own RAN, etc, but am happy to be wrong.
It's probably the same kinda way L2TP customers connect.
-
> Is the firebrick supplied in a way to lock people into aaisp services? As all of the things you just described are essentially whats keeping you at aaisp.
No :-)
I’m with AA because I just love them. And can’t imagine going elsewhere. And I’ve only recently had the dongle and was a customer for many years without. They would like to sell Bricks to as many users as possible and I believe some other ISPs use Firebricks. Some AA users iirc may be using two different ISPs bonded together by a Firebrick, for reliability.
They can support a 4G dongle but the support website says that this requires putting the dongle into a mode where it acts like a home gateway/home user router, and does unwanted NATing of traffic. I just need a dongle to be a modem, and just pass PPP frames straight through, so that IPv4 and IPv6 traffic is completely unmolested.
Since the dongle is only used in a failover situation, once in a blue moon, then I decided that because if the hassle, I would have to stick with 3G mode which acts as a PPPoE modem.
Firebricks are sold by AA, by Watchfront and by Firebrick Ltd which iirc is a joint venture owned by Watchfront and Firebrick both. If I have got this wrong, then I’m hoping someone will correct me. Watchfront itself is another ISP and so there’s one immediate example of a Firebrick being used with a non-AA ISP
-
Yeah as Carl pointed out you can mix NAT with no NAT, so a NAT'd 4g wouldnt force ADSL to be also NAT'd.
-
Unlike the solution suggested by some of the setups AA describe for failover, I was a perfectionist and insisted on keeping all IP addresses constant and keeping TCP connections intact during the failover. So that means that addresses inside the Firebrick’s LAN range don’t suddenly change from what they were before to some RFC1918 crappy new replacement addresses; existing IP addresses carry on exactly as before. My addresses are all globally routable and eternal, unchanged during failover. I use a smaller IPv6 MTU of 1408 too so that TCP connections can continue without being disrupted because the MTU is suddenly forced to drop substantially. It would have to drop because of increased overhead because currently there’s no 3G/4G support for IPv6 so a tunnel is used. I have the MTU permanently low, so it does not reduce - it’s lower than necessary before failover so that when the failover happens nothing actually changes; the increased overhead just causes an MTU to be chosen that matches that already in use, so nothing breaks. IPv4 works differently, uses MTU 1500+8=1508 normally and after failover goes down to MTU 1408. These quite low MTU values are chosen because they are perfection for ATM cell packing efficiency with the DSL overhead that I have in this particular case.
-
I could certainly arrange seamless failover but genuinely don't have any real driver for it. Having connections reestablish is fine.
So from that point of view the solution is quite inferior to yours :)
If I really wanted seamless I'd be using a pair of tunnels to an SD-WAN appliance hosted on a VPS and be pushing all my traffic that needed seamless through there - that way no IP address changes as encapsulation takes care of the change.
Could have it switch over within at most 100 ms via our fast fail feature but that's probably excessive.
If I had 2 wired LAN circuits I could run either all traffic or a critical subset through them and have a failover time of zero - for that critical, sensitive traffic one link carries the data the other FEC on a 1:1 basis so loss of one link means not a single bit dropped.
But that's excessive, expensive and Openreach won't let me have 2 access lines so academic.
-
Unlike the solution suggested by some of the setups AA describe for failover, I was a perfectionist and insisted on keeping all IP addresses constant and keeping TCP connections intact during the failover.
I believe you said at some point that you use actual Internet addressing (AA PA) on you internal network. Is that correct? If so then only AA are going to route that address space. However could you you use an AA L2TP connection for your backup, giving you a completely free hand for 4G provider? I'd be very surprised if their service wouldn't pass over NAT. If you really wanted to avoid NAT you could use one of the 4G routers that can bridge the WAN connection to the LAN. Presumably the Firebrick would be OK receiving a DHCP address, I find it hard to believe it can only work using PPPoE.
-
> Do you use actual internet addressing?
Yes. I use global routable constant IPv4 and IPv6 addresses for all the hosts on my main LAN, no NAT, no RFC1918 addressing required.(Although I do use RFC 1918 addresses internally; for talking to modems. The modems’ admin interface addresses are exposed as 192.168.n.1 when n is 1,2,3,4 the nth modem. The Firebrick routes these addresses across from the link between the Firebrick and each modem’s admin interface and passes traffic onto the main LAN so that the modems can be queried and so forth)
> L2TP:
Yes, I thought about that. But then there’s the cost of AA L2TP, charged per byte of traffic. And the hassle of setting it up given my pain and concentration levels. That would give me 4G instead of 3G though. And it’s only used once in a blue moon anyway.
The current system keeps me going and is truly seamless because the src IP addresses don’t change so no TCP connections that are established need to be broken; so no protocol breakages at all.
IPv6 MTU does not change at failover, but for the normal DSL state, PPP MTU=1500+8=1508 therefore normal IPv4 PDU MTU=1500 bytes and that will drop, a lot, when in the failed-over state, because of the limited MTU of the 3G ‘dongle’ USB NIC (IP PDU MTU is only 1430 or 1440 or something like that). Because IPv4 can not only change MTU on the fly even while a flow is established so that I’m assuming that PMTUD will detect the sudden reduction in path MTU and fix the problem, but IPv4 can fragment packets at intermediate routers not just at the source. So for this reason, I’m assuming that even before PMTUD kicks in and changes the PMTU, the first IPv4 packets can simply get fragmented if need be and again I’m assuming, perhaps rather optimistically, that that isn’t going to be the end of the world if there’s a bit of IPv4 fragmentation on failover, perhaps only short-term, but I’m not willing to reduce the IPv4 MTU permanently even in the normal case and just for the sake of the smoothness if the failover when fragmentation should I hope work and may only be needed for a while. Does a sudden onset of fragmentation get detected and trigger a change in PMTU that some transport layer by Eg TCP can take advantage of to fix the temporarily or permanently limited MTU problem?
It’s really annoying because the reduced MTU of the USB 3G NIC is way below 1500 bytes, for no good reason at all. If anything it should be oversized something like 1600 bytes, to support tunnel overhead. What is incredibly annoying is that AA’s 4G/3G carrier network AQL/Three does not support IPv6 and this from AA, the leader in IPv6 provision, with IPv6 first offered in something like 2002 iirc. Have to use AA’s 6in4 tunnelling to get it to work
-
PMTUD relies on the do not fragment bit being set so you'll either have a stall in the connection while the new MTU is worked out or fragmentation.
Fragmentation in the path will break certain traffic. Some applications don't respond well to it.
Only way around this is tunneling and fragmentation, coalescing and reassembly before presentation to the end application.
Firewalls in path may not be fans of fragments either, especially if they arrive out of order.
If it were that easy otherwise I'd be out of a job!
-
Regarding the original I may have a possible solution to the dual-WAN dilemma, however I can't go into any detail just yet.
Things are getting interesting. :angel:
-
Perhaps I’ve done a bad thing allowing occasional IPv4 fragmentation then, but the failover state or even just the immediate post-failover period are very rare occurrences so I thought I had better not let such things compromise the normal state of affairs.
It would indeed I suspect be useful if some routers could coalesce fragments in case of a problem with fragment-unfriendly firewalls or apps further upstream.
-
You've done the best you could without specialised, and expensive, hardware.
The cigar-box size piece of kit on my desk that'll do this for 100 Mb/s of traffic across pretty much as many WAN links as you want to throw at it probably costs about the same as your Firebrick. That you need another one in a data centre somewhere to terminate the traffic or a virtual machine adds to the cost. The licensing is where it gets really fun.
-
So I've had a bunch of issues and had to have a rethink.
Ubiquiti kindly started selling a fantastic bit of equipment in the interim meaning I can outsource much of the work.
I have one of these (https://store.ui.com/collections/routing-switching/products/udm-pro) on their way from them. It'll serve as router, switch under stairs, gateway and wireless controller.
I will supplement it with their access points as I go, adding one or two a month depending on budget until every room has superb wireless coverage.
I am hoping to get cabling to a couple of key places in the property, however this may end up being unjacketed ClearCurve (https://www.corning.com/emea/en/products/communication-networks/products/fiber/clearcurve-multimode-fiber.html) along the skirting board :)
-
When the moment is ripe, I (and I suspect other members) would be interested to see another (pictorial) diagram of how your LAN is now planned to be deployed.
-
This depends on how/if I can get cabling to some rooms. That's something that's far from clear just now!
-
Ubiquiti kindly started selling a fantastic bit of equipment in the interim meaning I can outsource much of the work.
I have one of these (https://store.ui.com/collections/routing-switching/products/udm-pro) on their way from them. It'll serve as router, switch under stairs, gateway and wireless controller.
I read this, and completely by coincidence read this (https://forums.thinkbroadband.com/fibre/t/4637546-re-routerfirewall-for-300mbs-fttp.html), don't know what his problem is with it though.
-
Inevitably going to be software. Not too worried: that can be fixed.
-
I was very put off Ubiquiti WAPs because of RevK - the boss of AA- ’s experience with Apple and roaming and Ubiquiti. (Described in Revk’s blog (https://www.revk.uk/search?q=ubiquiti). Basically he couldn’t get roaming to happen with an iPhone despite huge effort. This might not happen with you though because you’re using all Ubiquiti gear and it could be the mixture of Ubiquiti and Firebrick that somehow triggered his badness.
It would be a death situation for me, an all Apple shop with a Firebrick.
-
I was reading that Ubiquiti have pretty poor manufacturing quality and firmware compared to such as Zyxel (their comparable products, not consumer).
Reading their forums you can see the firmware issues, they seem to release products in beta or even alpha state and use their customers to bug fix.
I get the appeal of using a central controller to manage everything, but I'm having second thoughts on their products after really thinking about this.
-
As long as it has the SFP+ ports and can achieve the throughput it claims it will be fine.
-
Controller installed.
Ooh it is buggy :lol:
Model UniFi Dream Machine Pro
Version 1.6.4.2306
Board Revision 5
-
Ubiquiti release buggy software? But they NEVER do that. ::)
-
It has some really obvious defects that were noticed immediately.
I have barely scratched the surface and can already list 5+ defects of functionality or reporting.
-
I'll update this thread with the network as I build it. So very much in tactical solution territory.
-
I'll update this thread with the network as I build it. So very much in tactical solution territory.
Thank you, that will be appreciated.
-
Maybe you’ve already mentioned this - I haven’t gone back through the whole thread and my memory is shot - could you tell us about your choice of addressing scheme? IPv6 ? NAT or real addresses? And what are your MTUs going to be like for IPv4/IPv6?
-
Routed IPv6, NAT IPv4, relying on router to fragment and coalesce as it needs to. It is massively overpowered so will be fine.
-
Switched off the Ubiquiti for now.
I've 300/50 and can't use anywhere near all of it so rather pointless having that router.
The network right now is the ONT connecting to a regular WiFi router with a Raspberry Pi connected wired and everything else wireless.
I don't think a diagram is needed for that.
-
The network right now is the ONT connecting to a regular WiFi router with a Raspberry Pi connected wired and everything else wireless.
So the "grand design of the master-plan" has been reduced to very simple, basic, elements. I can easily "see" that infrastructure layout! :D
-
I have no internal cabling outside of the cupboard under the stairs and no realistic prospect of doing so without notable damage to the property.
The quotes I've taken indicate how complex and disruptive this is going to be.
I'm stuck.
This written from our bedroom, where I currently have the below performance due to the wireless speed to my then £400 and even now £350 access point being 14 Mbps.
https://www.speedtest.net/my-result/a/5837526641
It is what it is. I don't have the energy to rage against it.
I've already bought components. Anyone want some Cat 8.1, brushed steel face plates, back plates or 2 GHz RJ-45 connectors?
-
Carl sorry to hear that. It is absurd that they are building new houses without any cabled networking built in.
It is possible to install cabling with minimal damage, round access holes can be cut in chip board floors, small holes in plasterboard so cables can be fed behind, but it's always disruptive and will require some redecoration. Whilst not ideal it may be possible to run cables externally, up to the loft or other rooms to get better WiFi coverage.
-
Hi
I have no internal cabling outside of the cupboard under the stairs and no realistic prospect of doing so without notable damage to the property.
The quotes I've taken indicate how complex and disruptive this is going to be.
I'm stuck.
This written from our bedroom, where I currently have the below performance due to the wireless speed to my then £400 and even now £350 access point being 14 Mbps.
https://www.speedtest.net/my-result/a/5837526641
It is what it is. I don't have the energy to rage against it.
I've already bought components. Anyone want some Cat 8.1, brushed steel face plates, back plates or 2 GHz RJ-45 connectors?
Can you not go outside the property with cabling? In our house to get Ethernet from the study one end to the lounge was simply a case of taking Ethernet from inside to outside and running the cabling around the house. Our house (new but not new build) the cable was easily buried just underneath the stone chippings running around the house next to the outside bricks. A similar thing was done to take Ethernet to the loft for the access point, fortunately already routed out by co-ax TV aerial cable we didn't need so it replaced that.
Regards
Phil
-
The broadband comes in under the stairs right in the middle of the house.
That's life. It was a nice little project idea.
-
Solution found. Can't keep me down for too long.
802.11ax, 160 MHz 4x4 wireless bridge. Careful placement will produce minimal attenuation, permitting >1Gbps throughput. Best case close to 2.5 Gbit/s.
Depending how it performs adding a second load of wireless bridging is an option and using switches and static LAG to load balance between the two bridges.
Then in a few years time when 2 - 5 Gbit/s isn't enough my beautiful wife will be more accommodating to using discrete fibre. The house it a bit new right now.
Will build, iPerf and update.
-
It's good that you've found a workable solution, only the other day in some BT document I was reading it warned builders to install network cabling to other parts of the house if terminating the fibre under the stairs!
As soon as your wife mentions decorating, that's the time start installing the hardwired network, well if we're going to decorate it would be silly not to install this important cabling..........
-
802.11ax, 160 MHz 4x4 wireless bridge. Careful placement will produce minimal attenuation, permitting >1Gbps throughput.
I was wondering how long it would be before you considered the possibility of using a wireless bridge. :D
-
Accommodation reached with the developer. Bridge components, 10G switch and SFP arriving tomorrow.
See subject change.
-
Accommodation reached with the developer. Bridge components, 10G switch and SFP arriving tomorrow.
I shall be watching (https://elrepo.org/people/ajb/I_Am_Watching_You.jpg), with interest.
-
Will run with one of the AX routers routing. When I get the second ISP in place will revert to the Ubiquiti kit.
Once I get the AX kit installed I'll iPerf it and see how good the link actually is. A 160 MHz channel should be good for 4.6 Gbit/s PHY rate, hoping for around 2.5 Gbit/s half-duplex Ethernet rate: more than enough for 2 x 1000 / 115 links.
-
I will certainly be curious. Quite annoyed my nanoHD with a 1300Mbit down PHY and 1560Mbit up, rate still only does 700Mbit throughput in either direction on my laptops, despite my Galaxy S10 also doing the same with an 866Mbit PHY.
CPU load on the nanoHD is really low, makes me wonder if its a bug in the Intel AX200 cards in the laptops or Ubiquiti put some weird speed cap in the firmware.
Although I've long since given up trying to understand the weirdness that can happen with WiFi to Ethernet bridges. On some WiFi APs in the past I used to struggle to max out my broadband speed, even though I could do several times faster than that to LAN clients. I can't even begin to understand what weird network tomfoolery was going on there.
-
What model is the 802.11ax hardware? What’s the range of 802.11ax like?
-
Going to be using a pair of https://www.asus.com/us/Networking/ROG-Rapture-GT-AX11000/ Weaver.
They outperform AC as they use OFDMA. Will let you know how it compares once I've installed them.
Alongside these a 4 SFP+ port switch is going in. The AX11k has 2.5Gbase-T which I will take full advantage of once I've moved to > 1 Gb broadband on the WAN and will use immediately on the LAN.
-
Going to be using a pair of https://www.asus.com/us/Networking/ROG-Rapture-GT-AX11000/ Weaver.
They outperform AC as they use OFDMA. Will let you know how it compares once I've installed them.
Alongside these a 4 SFP+ port switch is going in. The AX11k has 2.5Gbase-T which I will take full advantage of once I've moved to > 1 Gb broadband on the WAN and will use immediately on the LAN.
Good god, I think a bit of Ethernet cable or thin fibre would be a lot less intrusive than those things :-) Talk about designed by men for marketing to boys. Presumably you'll be hiding them. :lol: Dread to think how much 2 of those will cost a year in electricity costs.
Regards
Phil
-
Going to be using a pair of https://www.asus.com/us/Networking/ROG-Rapture-GT-AX11000/ Weaver.
They outperform AC as they use OFDMA. Will let you know how it compares once I've installed them.
Alongside these a 4 SFP+ port switch is going in. The AX11k has 2.5Gbase-T which I will take full advantage of once I've moved to > 1 Gb broadband on the WAN and will use immediately on the LAN.
Will be interesting to see if the firmware has improved since https://www.smallnetbuilder.com/wireless/wireless-reviews/33220-wi-fi-6-performance-roundup-five-routers-tested as their tests seem to fall short of what I can get on the nanoHD using AC.
That does seem to contradict https://hothardware.com/reviews/asus-rog-rapture-gt-ax11000-router-review so I'm really not sure.
-
So, PHY rate is more than adequate. Can't reliably test the throughput yet: will update more next week. :)
-
Sorry Carl, please forgive me, being a bit slow here. What plugs into the 2.5G ethernet copper port?
-
Found a bug in switch software - SFP+ was not auto-negotiating properly. Upgraded and fixed. Throughput now stable, was a duplex mismatch there.
The 2.5G port is going to a Mikrotik CRS305-1G-4S+ switch, Weaver.
-
And you have an SFP module for 2.5G copper plugged in to one of those slots?
If I ever get round to upgrading my WAPs I don’t have any clients that can speak 802.11ax anyway, but I could take advantage of 802.11ac wave 2. Dee.jay (hero :angel:), iirc, offered to sort out my Cisco APs for me, but I can’t get Janet to ever remember to post them off to him.
-
Yes. It's a 10G SFP+ that can do 100 Mb, 1, 2.5, 5 and 10 Gbit.
Due to a software bug it was previously at 10 Gbit, it's now correctly negotiating 2.5 with the Asus.
-
This is extremely helpful stuff, many thanks for your write-ups.
-
I'll write up some more tomorrow.
I will probably get some fibre just in case, and will carry on my testing with this solution.
Should be able to give it more of a workout some time around Monday of next week.
-
Okay.
So temporarily I'm running with an Asus GT-AX11000 (https://www.asus.com/uk/Networking/ROG-Rapture-GT-AX11000/) connected directly to my ONT.
This provides 802.11ac 2.4 GHz connectivity and 802.11ac/ax 5 GHz connectivity to the property.
It also connects via a dedicated 160 MHz 802.11ax bridge to the same type of device in the office.
That device goes via 2.5 Gbit Ethernet to a Mikrotik CRS305-1G-4S-IN switch.
Longer term I'm looking at getting 10 Gigabit Ethernet between Ubiquiti UDM Pro and another CRS305-1G-4S-IN which will then feed one of the GT-AX11000s via 2.5 Gbit Ethernet and then the office CRS305 via ultra-fine fibre, again at 10 Gbit.
Acquiring the fibre isn't the easiest thing but is a work in progress. The AX11000s can then do wireless AP downstairs and mesh AP upstairs duty :)
-
These wireless routers are garbage.
The 2.4-2.8 Gbit PHY speed translates to about 1/10th of that in actual throughput.
I was able to conduct a fair test and it failed it abysmally.
The wireless performance on 2 of the 3 radios is inferior to my previous 802.11ac device, even on ax clients, the other one claims huge PHY throughput but can't shift more than 280 Mbit/s sustained with occasional bursts to 320.
In short the 4x4 802.11ax radio in this device running with a 160 GHz bandwidth performs about the same as a 1x1 802.11ac radio connecting to an 80 MHz channel.
-
These wireless routers are garbage.
So that's another idea crossed of the list. :(
-
Fibre it is. The refund from the wireless should cover the necessary.
-
I should mention further testing provided confusing results.
Nearly saturating the gigabit port on the server uploading to it, seeing about 640 Mb/s downloading from it.
I can't exclude the VM and host as a cause of the bottleneck, however neither can I prove the wireless.
I will test it via Internet early next week.
-
The 2.4 GHz radio on this thing isn't great and the first 5 GHz radio is artificially attenuated.
So I thought I'd switch those off and replace with an 802.11ac device.
The Asus responded by dropping the point to point AX link from 160 MHz to 20 MHz even though there was no channel overlap.
So I've responded by returning one of the Asus tomorrow and the other one once its replacement arrives Tuesday.
Using my old router for right now. It delivers better performance than the Asus on 2.4 GHz and the first 5 GHz radio, replacing AX with AC has resulted in slightly lower throughput but more sustained and consistent - I can hit the same speed on my 300 Mbit consistently over the media bridge I can wired now.
Replacing with Netgear Nighthawk AX12 12-Stream WiFi 6 Router (RAX120) one at a time. Will test with to gauge performance to AC devices, then if it's okay will install a second one as the other half of the bridge.
Not going to chance the same wireless chipset used before - see if the 8x8:8 5 GHz will deliver. Can't see any reason why it wouldn't.
-
As an aside another Mikrotik switch is on the way: needed to handle the fibre connection - 10GBase-T to switch then fibre to my home office and 2 SFP+ ports and a GigE port for whatever.
Will need single mode BiDi fibre SFP+ x 2. This isn't too inordinately expensive. :)
-
Will need single mode BiDi fibre SFP+ x 2. This isn't too inordinately expensive. :)
Will they be 1310/1550 nm & 1550/1310 nm or 1310/1490 nm & 1490/1310 nm ? (Presumably SC/PC connectors.)
-
Has to be LC as SC/PC are too large to be used with an SFP+ - check the size of the SC connector compared to an SFP+ and remember that BiDi is literally a transceiver on one side of a bidirectional SFP+.
The wavelength doesn't really matter as long as it's the opposite on both sides :)
-
I'm not familiar with SFP+ . . . have never handled one. And I automatically think of pairs of mono-directional fibres & connectors for LC. Quite wrong, of course.
The wavelength doesn't really matter as long as it's the opposite on both sides :)
I was wondering if you had already made a decision, hence the query.
-
I can assure my fingers are decidedly average in size at most, if not on the small side in length and rather fat. Gecko fingers!
-
Happy days. 2 x OFS EZ-Bend Invisilight Fiber Optic Wiring WCC Kit 20 M Version 3 on the way.
1FT LC Male SC Female Duplex Singlemode 9/125 Fiber Optic Adapter Cable that I'll separate into 2 simplex on the way.
Once I've the delivery dates I can consider my choice of SFP. The Mikrotik switches happily take anything.
What a mission!
-
I'm seeing a lot of confusing information regarding SFP+ modules.
I'm thinking of moving from RJ45 in my NAS/Server to SFP+, to reduce heat and power consumption.
Is there anything important to think about as I constantly see people saying you need Intel specific SFP+ with Intel cards, Netgear ones for the Netgear Switch. Is this just sellers guaranteeing support or are there major compatibility issues here?
If there are, how do you get an SFP+ DAC that will work between a Netgear Switch and Intel NIC?
Would you have a recommendation to look for regarding a NIC (doesn't have to be Intel) that doesn't cost the earth for optimal performance on Linux and is there a specific 10Gbit DAC I'd need or will they all simply be straight-through wired the same?
I'm kinda trying to decide if to go DAC or Fibre, as the latter would mean a surge from the router would at least be less likely to reach the NAS (though this should be a none-issue once I switch to FTTP) and I'm kinda concerned that I will end up breaking the DAC during routine cleaning of the NAS.
-
I'm not aware of the criteria being so strict. I suspect Intel NICs are fine with pretty much anything. You should be fine with generic SFP+.
I have no recommendations. I'm using switches and a router to provide a 10G backbone with some 2.5GBase-T stuff spurring off that backbone.
The Netgear kit is likely to be more strict than the Intel card. Find a Netgear DAC that is confirmed to work then find a generic unbranded compatible one. :)
-
Will they be 1310/1550 nm & 1550/1310 nm or 1310/1490 nm & 1490/1310 nm ? (Presumably SC/PC connectors.)
None of those: 1270/1330 nm. LC connectors.
-
I can assure my fingers are decidedly average in size at most, if not on the small side in length and rather fat. Gecko fingers!
Thank you for posting the image. I note that the UF-RJ45-10G SFP+ interface, with an 8P8C female socket, looks just like an SFP equivalent. (Though I don't have one here, to create a similar photograph.)
Looking at an Adva SFP/GBE/1310S/SM/LC optic (the most convenient optic for me to measure), it is (approx) 13mm wide, 8mm high by 47mm deep. (They are the dimensions of the optic that fit into the host cage.) Positioning the optic on a human left-hand, in an attempt to emulate the subject of your photograph, I can measure a further 39mm from the end of the optic to the finger-tip.
So it appears, to me, that an SFP and an SFP+ interface/optic are virtually the same size.
None of those: 1270/1330 nm. LC connectors.
Noted, with thanks.
-
Oh I see. Physically the two are identically sized.
-
Oh I see. Physically the two are identically sized.
That accounts for my puzzlement with regards to the available space for either SC or LC connectors. (I'm only familiar with SFP optics, those with either one SC or two LC connectors.)
-
The BiDi optics are nothing more than a simplex LC connector rather than the duplex you'd usually see with a pair of fibres.
Either way I'll get it in place.
Replaced one of the Asus devices with Netgear. This is over WiFi:
(https://www.speedtest.net/result/9178099030.png)
-
The BiDi optics are nothing more than a simplex LC connector rather than the duplex you'd usually see with a pair of fibres.
Agreed. :)
For fun, this is what I see --
[bcat ~]$ speedtest-cli --simple
Ping: 50.704 ms
Download: 4.19 Mbit/s
Upload: 1.01 Mbit/s
[bcat ~]$
The latency is no problem for my mode of operation and if I could have 5/5 Mbps (i.e. symmetric) then I'd be perfectly happy.
But now I am hijacking your thread.
-
The BiDi optics are nothing more than a simplex LC connector rather than the duplex you'd usually see with a pair of fibres.
Either way I'll get it in place.
Replaced one of the Asus devices with Netgear. This is over WiFi:
(https://www.speedtest.net/result/9178099030.png)
Oh that's not fair, I can't get that high download off the nanoHD from my NAS.
-
Oh I see. Physically the two are identically sized.
And in fact some gear can accept a 1 meg SFP in their SFP+ slots. Some Nexus models for example.
-
And in fact some gear can accept a 1 meg SFP in their SFP+ slots. Some Nexus models for example.
All SFP should be fine in an SFP+ slot. Backwards compatible.
-
"Should" maybe. But one of our early servers has SFP+ slots that don't support 1 gig transceivers. https://tmgmatrix.cisco.com/?npid=3
I did try anyway but no joy.
-
Lockdowns by manufacturer based purely on software excepted, of course. That's a decision Cisco made to lock it down so that they can sell overpriced SFPs rather than people buying generic ones. :)
The standards are backwards compatible but just as not all SFPs work in all hardware that supports SFPs so the same goes for SFP+ and SFPs into SFP+ slots.
That server could certainly support 1.25 Gbit transceivers if it's standards compliant, Cisco just saw fit not to allow them or didn't comply with standards.
Flash an SFP appropriately and it should work as long as the port is compliant. SFP+ slots should support 1.25 Gbit/s and 12.5 Gbit/s lanes.
-
Lockdowns by manufacturer based purely on software excepted, of course. That's a decision Cisco made to lock it down so that they can sell overpriced SFPs rather than people buying generic ones.
I'm not convinced that's the sole reason, for example that device doesn't support Cisco 1gig or 100Fx SFPs. And in fact doesn't even support all of their 10gig options either.
-
I'm not convinced that's the sole reason, for example that device doesn't support Cisco 1gig or 100Fx SFPs. And in fact doesn't even support all of their 10gig options either.
I am. Cisco made a decision not to support that stuff in their software. As long as it's standards compliant the interface itself is well documented.
-
Looking at SFP+ 10Gbit NICs on eBay/Amazon and it seems all the cheap ones either run hot or all the information I can find on them are people having problems getting them working. :'(
-
Running hot rather comes with the territory. 12.5 Gbit/s symmetrical of throughput runs warm :)
-
Curse you all for mentioning NICs. A 2 SFP+ port card has fallen into my shopping cart.
-
Curse you all for mentioning NICs. A 2 SFP+ port card has fallen into my shopping cart.
I was very tempted to get a 2 port but I figured I might need that money for, you know, food, bog roll, etc. ::)
I ended up getting an Intel X520-DA1 which hopefully will work with the 10Gtek DAC cable. They claim it works with their version of that card, but then they charge twice as much for theirs as I got that one for new off eBay. (I could have gotten a 2 port card for the price of their 1 port)
Reading the Linux kernel module documentation is kinda scary, as it implies by default it blocks all SFP+ modules/DACs that have not been tested which seems kinda silly IMO. I know adding module options isn't hard, but I'm always expecting them not to work.
-
For personal reasons this project has been indefinitely suspended and there'll be no updates for the foreseeable future.
Thank you, everyone, for your comments, feedback and interest. :)
-
Wow, the DAC arrived and the pictures make it look super thick, I was worried about the bend radius - but its tiny, way thinner than ethernet.
I mean the bend radius is still bigger due to the SFP+ connectors but getting round the back of the server without kinking is going be no issue at all.
-
Guys what kind of card are we talking about, in hardware platform terms? Are we talking about a desktop class PC? Or server or what?
-
Intel call it a "server adapter" but of course it could be put into a desktop PC as long as you had a suitable slot.
-
Guys what kind of card are we talking about, in hardware platform terms? Are we talking about a desktop class PC? Or server or what?
Anything with a PCIe slot, if I am remembering correctly.
-
(https://i.ebayimg.com/images/g/kqgAAOSwjYxdhhPz/s-l1600.jpg)
(https://images-na.ssl-images-amazon.com/images/I/61WrPUmXWuL._AC_SL1008_.jpg)
-
Thank you for posting those images. They are exactly as I expected. However we do seem to have hijacked CarlT's thread.
-
Thank you for posting those images. They are exactly as I expected. However we do seem to have hijacked CarlT's thread.
Well he did say:
there'll be no updates for the foreseeable future.
-
Could always split the thread off, to keep the last sort of it more visible.
-
I don't feel we went that far off-topic, it is a forum not a blog after all so discussing 10Gbit seemed relevant. But its not my forum nor thread.
-
False alarm, all is well, please carry on with the discussion.
-
Nice and simple at the moment - here's what we have attached.
-
When ready to move to dual ISPs the UDM Pro will handle it with 2 WAN ports. Then a 10 Gbit Ethernet link to the switch which will, in turn, feed the RAX 120 in access point mode via the 5GBase-T link and the LR optical link to the other switch.
The UDM Pro has a number of copper ports of its own supplying port density there, no need for the 2 Gb LAG, and the office switch will be okay doing LAG detail to a normal GigE switch.
No need for dual ISP networking for the foreseeable though, of course, as no installs are being done within the home.
-
Single mode fibre between the two Mikrotik switches?
-
Ya. Single mode Invisilight.
-
Single homed. For obvious reasons a second ONT or ONT replacement isn't happening right now.
(https://i.ibb.co/Sf8h1BD/Physical.png)
-
Putting the router in place there has had the added bonus that there's no need to have external and internal VLANs going across the switch which simplifies things a ton.
The lab box is listed as having 2 interfaces though it actually has 3 and only one of them gets to access the Internet, the others are sink holed.
Anyway that's my network building done. That was not cheap.
-
Various bits of kit migrated to home office. The Harry Potter room now has ONT, switch and WiFi AP and that's all.
The 4 RU cabinet has been moved to the home office where ventilation is better.
All is ready for dual-WAN now as far as the switches go. The switch ports are isolated so that no LAN device sees WAN traffic and, likewise, no WAN port sees anything other than WAN traffic.
Under stairs switch:
0 GE-1 - ONT - 1 GbE
1 SFP-1 - Empty - Future ONT 2
2 SFP-2 - 10Gbase-BX to Office
3 SFP-3 - Empty
4 SFP-4 - 5Gbase-T to WiFi AP
Office switch:
0 GE-1 - Empty - Future WAN 2
1 SFP-1 - UDM Pro WAN - 10G DAC
2 SFP-2 - Server - 10G DAC
3 SFP-3 - 10Gbase-BX to Stairs
4 SFP-4 - UDM Pro LAN - 10G DAC
Security - unless specified connectivity denied:
Stairs:
0 <> 2
1 <> 2
2 <> All
3 <> 2, 4
4 <> 2, 3
Office:
0 <> 3
1 <> 3
2 <> 3, 4
3 <> All
4 <> 2, 3
The UDM Pro has 8 x GbE ports of its own all on a LAN switch. 2 of these are a tagless LAG to the GigE switch in the office, another is connected to the Raspberry Pi on PiHole duties.
Highly not recommended to be done regularly. Hasn't been cheap.
-
Highly not recommended to be done regularly. Hasn't been cheap.
But fun. ;)
-
Good for you!
How are you liking the UDM Pro? And how are you finding the wireless throughout?
-
From my phone:
https://www.speedtest.net/my-result/a/5979582922
The UDM is running test software which seems to fix issues. It's okay for right now!
-
But fun. ;)
Fun isn't the first word that came to mind. It's been far more pleasing relating the trials and tribulations to you guys and letting you enjoy them vicariously than actually doing the work. It was all pretty trivial stuff.
It's done now and in its completed state. Port isolation takes care of things locally on each switch and the respective WANs are using VLAN tagging
Traffic using WiFi goes along the fibre then back again but that's not a big deal. The 10 Gb link is only going to be used for Internet and WiFi and that's going to max out for the foreseeable at perhaps 4 Gbit/s: a gigabit each for the 2 WANs and 2 Gbit/s for the wireless mesh that I'll complete.
Just for your entertainment I can see my Huawei ONT's MAC address on both switches and I can see the Nokia 7750 SR router in the exchange for obvious reasons - it's PPP over Ethernet after all.
-
I've kinda soured to Ubiquiti after reading up on them removing the ability to set static DNS overrides by manually editing the config file and the fact I had to turn off the broadcast limiter on my nanoHD as it was erroneously blocking ARP traffic causing an inability to talk from LAN to WiFi clients.
-
Couple of real-time load snapshots of saturating the broadband downstream and also carrying out a LAN transfer - ensured that >1 Gb/s throughput was a thing.
Seems all good.
-
We're done. Thanks for reading.
(https://i.ibb.co/YLn3ZgB/Final-Lab-Home.png)
-
That's a Netgear R9000 (X10) in the photo, not a RAX 120 ;)
-
Couldn't be bothered to change the picture. :P
-
Right then.
With the core in place for normal, regular home service it has been time for fun with the build of the lab network.
Due to what needs to be built and a desire to minimise the network junk outside of my cabinet alongside using the physical appliances there is some great fun here involving the UDM Pro, one of the 10G switches, another physical GigE switch, 2 more physical routers, 2 SD-WAN appliances.
The logical construct involves an L2TP tunnel to simulate more direct Internet access to the SD-WAN , 4 BGP sessions to simulate an MPLS network and route import/export, a basic 3 node OSPF area to exchange LAN routes and obviously some logical and physical segmentation in terms of VLANs and vNICs.
Then the easiest part: the SD-WAN IKE-less IPSEC/UDP tunnels.
So that's how I spent a part of my working day.
-
Incidentally the dual-WAN will be active-passive so don't expect any super-high speed tests.
-
Carl, just been reading about you and your house in PCPro, it's ludicrous that they installed 7 phone points and no network sockets, absolutely clueless and incompetent idiots.
-
Hah forgot about that. ;D
-
As time has gone I've been busy. There remains zero copper between rooms, Invisilight fibre connects them.
Everything I'm using is in my signature. I'm not going to shove an enormous speed test link in there, however depending on the time of day, how much other people on my split are using their connections, etc, I see between 2000 and 2130-ish Mb/s with odd drops to the 1600s.
GPON is good for about 2,250 Mb/s of IP.
-
Everything I'm using is in my signature.
Upon review of your signature block, I see --
ISP: BT Full Fibre 900. Zen Full Fibre 900. Faelix FTTP 300
So you now have three separate services delivered over the one fibre to a 4+2 ONT? Or have I misinterpreted it? :-\
-
That's exactly what I'm using.
2 are over BTW, the other not BTW.
-
Thanks for the confirmation. :)
-
:o :o :o :'(
-
Basic physical diagram attached.
-
As you can tell from the physical structure there's a secondary switched path via WiFi so if I lose a big link I still have connectivity, albeit over WiFi at a gigabit give or take.
Connectivity is attached in PDF.
I've segmentation to do, but have been busy with a lot of other things related to work, study and home.
Mangle file is a section of the firewall rules. Inbound connections are tagged with the link they came in on so that they go out of the same one, outbound connections have source/destination IP and port hashed. 3 of the 7 rules take BT, 3 Zen and 1 Faelix providing 3/7, 3/7, 1/7 flow split.
https://wiki.mikrotik.com/wiki/Manual:PCC
A few other bits I'll keep to myself as they include some tweaks and specifics that I'm not going to advertise as they include public IP addressing I'd prefer not to advertise but the interested get the idea.
With the complex policy-based routing I am using the CRS2004 (https://mikrotik.com/product/ccr2004_1g_12s_2xs) can handle at least 4 Gb/s I guesstimate. As I am able to drop connections I will be able to reduce the rule set and retrieve some capacity.
Have had to spend a few quid but have kept the home network discrete as required by SWMBO.
EDIT: Neighbours have now largely gone to bed so I can nom the PON again.
https://www.speedtest.net/result/c/c8ab1b3e-e4bf-4ea0-91e7-eaeebd61c63b
-
So is this NOT over an SD-WAN right now?
-
No. Load balancing as above. There's an SD-WAN running in the lab with an interface for each WAN link.
It runs beside the normal network.
Unlikely as it is maxing the whole lot via AWS would quickly get expensive. Would need to spend a fortune just on the fixed costs to avoid potential bottlenecks.
I don't run the lab device or the remote network it connects to now.
-
:o :o :o :'(
You and Carl opposite ends of the tech spectrum for connectivity. O_o
-
Why I have the 3 is a bit of a story but mostly focused around that I wasn't paying for one of them and required another for resiliency.
I'm happy to drop one of the 900s as soon as the contract allows. The Faelix is run over Zen's on-net network, the Zen 900 is run over BT Wholesale.
-
Are the three ISPs IP-bonded (ie each individual flow is split across links) as I have it?
How are you getting on with the Netgear WAPs?
-
The ISPs are flow balanced. Every time a new flow is started source and destination IP and port are hashed mod 7. Remainder 0-2 go via BT, 3-5 Zen and 6 Faelix.
I'm using a Netgear router in AP mode and it's fine. It's an 802.11ax enabled AP and provides 700+ Mb/s wireless throughout much of the ground floor.
I'll be adding a couple of Aruba APs.
-
From the little I’ve read on the Netgear, specs look impressive. I’m thinking about getting Aruba APs because my ZyXEL NWA-3560n 802.11n APs are over ten years old now, and will die at some point, but they have been superb.
The flow-hash method is a very sensible one, stateless and fast. You had to set up the modulo->interface assignments yourself ? Because the router doesn’t know the relative speeds of the links.
My FB2900 router splits single flows across interfaces in the correct split according to each link’s upstream speed and it (now) gets the upstream speeds from a message delivered to the river by the ISP at PPP connect-time (suffixed to the normal connect string as <downstream>/<upstream>) and this is multiplied by a fraction given the config file which converts the PPP-specified speed into a true IP PDU bps speed. If a link goes down then the router recalculates the split according to the remaining links’ speeds.
-
Assignments set manually though not stateless - new flows are assessed and marked. Once marked subsequent packets in the stream fall through the rules and hit the routing.
I could use an overlay but there's the small matter of needing something on the remote side to terminate it on. Using the same ISP for everything leaves a bunch of single points of failure so would have to be provider independent.