Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 2 [3] 4 5 ... 11

Author Topic: 10 Gb, Dual-WAN, segmented home / home office Setup  (Read 11201 times)

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 9919
  • Retd s/w dev; A&A; 3x7km lines; Firebrick; IPv6
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #30 on: February 03, 2020, 07:29:18 PM »

Unlike the solution suggested by some of the setups AA describe for failover, I was a perfectionist and insisted on keeping all IP addresses constant and keeping TCP connections intact during the failover. So that means that addresses inside the Firebrickís LAN range donít suddenly change from what they were before to some RFC1918 crappy new replacement addresses; existing IP addresses carry on exactly as before. My addresses are all globally routable and eternal, unchanged during failover. I use a smaller IPv6 MTU of 1408 too so that TCP connections can continue without being disrupted because the MTU is suddenly forced to drop substantially. It would have to drop because of increased overhead because currently thereís no 3G/4G support for IPv6 so a tunnel is used. I have the MTU permanently low, so it does not reduce - itís lower than necessary before failover so that when the failover happens nothing actually changes; the increased overhead just causes an MTU to be chosen that matches that already in use, so nothing breaks. IPv4 works differently, uses MTU 1500+8=1508 normally and after failover goes down to MTU 1408. These quite low MTU values are chosen because they are perfection for ATM cell packing efficiency with the DSL overhead that I have in this particular case.
Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #31 on: February 03, 2020, 10:11:43 PM »

I could certainly arrange seamless failover but genuinely don't have any real driver for it. Having connections reestablish is fine.

So from that point of view the solution is quite inferior to yours  :)

If I really wanted seamless I'd be using a pair of tunnels to an SD-WAN appliance hosted on a VPS and be pushing all my traffic that needed seamless through there - that way no IP address changes as encapsulation takes care of the change.

Could have it switch over within at most 100 ms via our fast fail feature but that's probably excessive.

If I had 2 wired LAN circuits I could run either all traffic or a critical subset through them and have a failover time of zero - for that critical, sensitive traffic one link carries the data the other FEC on a 1:1 basis so loss of one link means not a single bit dropped.

But that's excessive, expensive and Openreach won't let me have 2 access lines so academic.
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

aesmith

  • Kitizen
  • ****
  • Posts: 1031
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #32 on: February 06, 2020, 01:39:23 PM »

Unlike the solution suggested by some of the setups AA describe for failover, I was a perfectionist and insisted on keeping all IP addresses constant and keeping TCP connections intact during the failover.
I believe you said at some point that you use actual Internet addressing (AA PA) on you internal network.  Is that correct?  If so then only AA are going to route that address space.  However could you you use an AA L2TP connection for your backup, giving you a completely free hand for 4G provider?  I'd be very surprised if their service wouldn't pass over NAT.  If you really wanted to avoid NAT you could use one of the 4G routers that can bridge the WAN connection to the LAN.  Presumably the Firebrick would be OK receiving a DHCP address, I find it hard to believe it can only work using PPPoE.
Logged

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 9919
  • Retd s/w dev; A&A; 3x7km lines; Firebrick; IPv6
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #33 on: February 10, 2020, 04:10:51 PM »

> Do you use actual internet addressing?

Yes. I use global routable constant IPv4 and IPv6 addresses for all the hosts on my main LAN, no NAT, no RFC1918 addressing required.(Although I do use RFC 1918 addresses internally; for talking to modems. The modemsí admin interface addresses are exposed as 192.168.n.1 when n is 1,2,3,4 the nth modem. The Firebrick routes these addresses across from the link between the Firebrick and each modemís admin interface and passes traffic onto the main LAN so that the modems can be queried and so forth)

> L2TP:

Yes, I thought about that. But then thereís the cost of AA L2TP, charged per byte of traffic. And the hassle of setting it up given my pain and concentration levels. That would give me 4G instead of 3G though. And itís only used once in a blue moon anyway.

The current system keeps me going and is truly seamless because the src IP addresses donít change so no TCP connections that are established need to be broken; so no protocol breakages at all.

IPv6 MTU does not change at failover, but for the normal DSL state, PPP MTU=1500+8=1508 therefore normal IPv4 PDU MTU=1500 bytes and that will drop, a lot, when in the failed-over state, because of the limited MTU of the 3G Ďdongleí USB NIC (IP PDU MTU is only 1430 or 1440 or something like that). Because IPv4 can not only change MTU on the fly even while a flow is established so that Iím assuming that PMTUD will detect the sudden reduction in path MTU and fix the problem, but IPv4 can fragment packets at intermediate routers not just at the source. So for this reason, Iím assuming that even before PMTUD kicks in and changes the PMTU, the first IPv4 packets can simply get fragmented if need be and again Iím assuming, perhaps rather optimistically, that that isnít going to be the end of the world if thereís a bit of IPv4 fragmentation on failover, perhaps only short-term, but Iím not willing to reduce the IPv4 MTU permanently even in the normal case and just for the sake of the smoothness if the failover when fragmentation should I hope work and may only be needed for a while. Does a sudden onset of fragmentation get detected and trigger a change in PMTU that some transport layer by Eg TCP can take advantage of to fix the temporarily or permanently limited MTU problem?

Itís really annoying because the reduced MTU of the USB 3G NIC is way below 1500 bytes, for no good reason at all. If anything it should be oversized something like 1600 bytes, to support tunnel overhead. What is incredibly annoying is that AAís 4G/3G carrier network AQL/Three does not support IPv6 and this from AA, the leader in IPv6 provision, with IPv6 first offered in something like 2002 iirc.  Have to use AAís 6in4 tunnelling to get it to work

Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #34 on: February 10, 2020, 05:39:08 PM »

PMTUD relies on the do not fragment bit being set so you'll either have a stall in the connection while the new MTU is worked out or fragmentation.

Fragmentation in the path will break certain traffic. Some applications don't respond well to it.

Only way around this is tunneling and fragmentation, coalescing and reassembly before presentation to the end application.

Firewalls in path may not be fans of fragments either, especially if they arrive out of order.

If it were that easy otherwise I'd be out of a job!
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #35 on: February 10, 2020, 06:09:21 PM »

Regarding the original I may have a possible solution to the dual-WAN dilemma, however I can't go into any detail just yet.

Things are getting interesting. :angel:
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 9919
  • Retd s/w dev; A&A; 3x7km lines; Firebrick; IPv6
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #36 on: February 10, 2020, 11:55:11 PM »

Perhaps Iíve done a bad thing allowing occasional IPv4 fragmentation then, but the failover state or even just the immediate post-failover period are very rare occurrences so I thought I had better not let such things compromise the normal state of affairs.

It would indeed I suspect be useful if some routers could coalesce fragments in case of a problem with fragment-unfriendly firewalls or apps further upstream.
« Last Edit: February 29, 2020, 08:40:24 PM by Weaver »
Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #37 on: February 11, 2020, 08:43:30 AM »

You've done the best you could without specialised, and expensive, hardware.

The cigar-box size piece of kit on my desk that'll do this for 100 Mb/s of traffic across pretty much as many WAN links as you want to throw at it probably costs about the same as your Firebrick. That you need another one in a data centre somewhere to terminate the traffic or a virtual machine adds to the cost. The licensing is where it gets really fun.
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #38 on: February 29, 2020, 10:45:19 AM »

So I've had a bunch of issues and had to have a rethink.

Ubiquiti kindly started selling a fantastic bit of equipment in the interim meaning I can outsource much of the work.

I have one of these on their way from them. It'll serve as router, switch under stairs, gateway and wireless controller.

I will supplement it with their access points as I go, adding one or two a month depending on budget until every room has superb wireless coverage.

I am hoping to get cabling to a couple of key places in the property, however this may end up being unjacketed ClearCurve along the skirting board  :)
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

burakkucat

  • Global Moderator
  • Senior Kitizen
  • *
  • Posts: 32932
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #39 on: February 29, 2020, 02:52:23 PM »

When the moment is ripe, I (and I suspect other members) would be interested to see another (pictorial) diagram of how your LAN is now planned to be deployed.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #40 on: February 29, 2020, 04:16:51 PM »

This depends on how/if I can get cabling to some rooms. That's something that's far from clear just now!
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

Ronski

  • Helpful
  • Kitizen
  • *
  • Posts: 4103
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #41 on: February 29, 2020, 05:13:26 PM »

Ubiquiti kindly started selling a fantastic bit of equipment in the interim meaning I can outsource much of the work.

I have one of these on their way from them. It'll serve as router, switch under stairs, gateway and wireless controller.

I read this, and completely by coincidence read this, don't know what his problem is with it though.
Logged
Formerly restrained by ECI and ali,  now surfing along at 390/36  ;D

CarlT

  • Kitizen
  • ****
  • Posts: 1808
  • Random network numpty
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #42 on: February 29, 2020, 05:27:51 PM »

Inevitably going to be software. Not too worried: that can be fixed.
Logged
BT Retail Full Fibre 900 // Zen Full Fibre 900 // Faelix FTTP 300 via a Mikrotik CCR2004-1G-12S+2XS.
1 * CSS326-24G-2S+RM, 2 * CRS309-1G-8S+IN, 2 * CRS305-1G-4S+IN all connected via Invisilight SMF, wife required subtlety, and DACs.

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 9919
  • Retd s/w dev; A&A; 3x7km lines; Firebrick; IPv6
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #43 on: February 29, 2020, 08:51:49 PM »

I was very put off Ubiquiti WAPs because of RevK - the boss of AA- ís experience with Apple and roaming and Ubiquiti. (Described in Revkís blog. Basically he couldnít get roaming to happen with an iPhone despite huge effort. This might not happen with you though because youíre using all Ubiquiti gear and it could be the mixture of Ubiquiti and Firebrick that somehow triggered his badness.

It would be a death situation for me, an all Apple shop with a Firebrick.
Logged

Alex Atkin UK

  • Kitizen
  • ****
  • Posts: 2565
    • Thinkbroadband Quality Monitors
Re: 10 Gb, Dual-WAN, segmented home / home office Setup
« Reply #44 on: February 29, 2020, 11:10:14 PM »

I was reading that Ubiquiti have pretty poor manufacturing quality and firmware compared to such as Zyxel (their comparable products, not consumer).

Reading their forums you can see the firmware issues, they seem to release products in beta or even alpha state and use their customers to bug fix.

I get the appeal of using a central controller to manage everything, but I'm having second thoughts on their products after really thinking about this.
« Last Edit: March 02, 2020, 08:10:57 PM by Alex Atkin UK »
Logged
INTAKE (ECI) 1xHome Hub 5A on Zen, 1xECI Openreach on Plusnet, 1xHauwei B535-232 on Voxi 4G Router: pfSense (i5-7200U) WiFi: Ubiquiti nanoHD + Honor Router 3
My Broadband History & Ping Quality Monitors
Pages: 1 2 [3] 4 5 ... 11
 

anything