Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 2 3 [4] 5

Author Topic: Diary of My Experiences with Lightning Fibre (FTTP local ISP)  (Read 12107 times)

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #45 on: July 15, 2022, 01:31:58 PM »

Question is, does TBB do a single-threaded upload test or is that multi-threaded?

Single threaded is even more important as downloads can usually be multi-threaded quite easily, but uploads cannot.

While I'm not 100% certain, I believe upload is single threaded. I'll try to find out.

I've had a response from Lightning Fibre, unfortunately somewhat mixed. Maybe someone here can suggest something.

---

They thanked me for the information I've given them and said that it's indeed unusual.

They managed to recreate similar behaviour their end when running a single TCP stream.

They checked all network devices in the path and even tested over different backhaul links, but it all looks normal as far as they can tell.

On their laptop they saw similar speeds, between 3-400 Mbps on a single TCP stream. Running a single TCP stream over LAN with two other clients on the same LAN saw near gigabit speeds.

It was repeatable behaviour. They said given the network infrastructure is seemingly quite happy moving that traffic at that rate (and above), it all looks well otherwise. They believe that this is most likely a client issue at one or both ends. Their guess is that it relates to the TCP window sizing or similar given throughput otherwise is normal and single stream TCP tests are fine on some clients repeatably.

They aren't sure that there's anything they can alter to affect this and can't be of much more help but hope that their feedback helps give me more information on what they can see from their end.

Any suggestions?

I'm not sure what to say at this point. If Cerberus Networks works fine through the same networking equipment (router being a CCR2004-16G-2S+ by Mikrotik) and computer, just a different ONT and LAN cable, I don't understand how it can be a client side issue.
Logged

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #46 on: July 15, 2022, 03:31:19 PM »

Any suggestions?

No, nothing comes to mind.

Quote
If Cerberus Networks works fine through the same networking equipment (router being a CCR2004-16G-2S+ by Mikrotik) and computer, just a different ONT and LAN cable, I don't understand how it can be a client side issue.

s/and LAN cable/and Ethernet patch cable/  --  for clarity.

For completeness, it would be sensible to try a new Ethernet patch cable (used to connect the MikroTik router to the ONT of the service under test) and use it with both ONTs.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #47 on: July 15, 2022, 03:57:44 PM »

No, nothing comes to mind.

s/and LAN cable/and Ethernet patch cable/  --  for clarity.

For completeness, it would be sensible to try a new Ethernet patch cable (used to connect the MikroTik router to the ONT of the service under test) and use it with both ONTs.

I thought about trying the same Ethernet patch cable that I use with the Openreach ONT, but I can't because the one that's connected to the Openreach ONT goes through a hole in the wall. It was cut and rejoined. The Lightning Fibre ONT is on the other side of the wall, it's not in the same room as the Openreach ONT. I can try a new cable though. I also tried a 'cable test' on RouterOS but it said 'link ok'. I have a testing gadget too which I can try but I have a feeling it would say that there's no problem with the cable going to the Lightning Fibre ONT.

I've replied to Lightning Fibre suggesting one other idea I can try but I highly doubt it will make a difference. However if it's fine to do so then it can't hurt to try. I know with Openreach's ONT you can power cycle it without any fuss, I imagine it's the same with Lightning Fibre's ONT but I've asked them just to make absolutely certain. It's the only other idea I have right now. In the reply I also mentioned how it was fine on day one and that I don't recall anything I've done here which could've caused this. If I did then surely the Cerberus Networks connection would also be slow (unless it's a dodgy cable connected to the Lightning Fibre ONT but I'll eliminate that possibility too).

EDIT: I've had a response, they don't recommend power cycling the ONT 'very often' because there have been issues where the '10G port' has been affected by 'constant power cycles'. Doing so the once should be fine though.

I only plan to do so once, just to see if it corrects this issue. I highly doubt it will though but it's worth a try.
« Last Edit: July 15, 2022, 04:07:45 PM by Ixel »
Logged

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #48 on: July 15, 2022, 04:18:55 PM »

I thought about trying the same Ethernet patch cable that I use with the Openreach ONT, but I can't because the one that's connected to the Openreach ONT goes through a hole in the wall. It was cut and rejoined. The Lightning Fibre ONT is on the other side of the wall, it's not in the same room as the Openreach ONT.

Hmm . . . I think I understand. So just checking: The Openreach ONT is in room A, the Lightning Fibre ONT is in room B and your equipment rack (cupboard) is in room B?

Do you have an Ethernet patch cable that is long enough to (temporally) run via room A door, hallway, room B door? Scientifically, the "same cable experiment" really needs to be performed.

Power-cycling or powering off the ONT for anything between, say, 5 minutes and 23 hours should have no effect on either service.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #49 on: July 15, 2022, 04:37:23 PM »

Hmm . . . I think I understand. So just checking: The Openreach ONT is in room A, the Lightning Fibre ONT is in room B and your equipment rack (cupboard) is in room B?

Do you have an Ethernet patch cable that is long enough to (temporally) run via room A door, hallway, room B door? Scientifically, the "same cable experiment" really needs to be performed.

Power-cycling or powering off the ONT for anything between, say, 5 minutes and 23 hours should have no effect on either service.

Yes (regarding room A and room B). I may have a long enough cable, I'll look through my cables and see what I have.

It's interesting how they said that 'constant' power cycling of the ONT has caused issues with the 10G port, though I don't know how often they consider 'constant' to be. I haven't power cycled the ONT ever but I have unplugged the ethernet patch cable from either my Mikrotik router or the Lightning Fibre (Zyxel) router a few times, mostly for diagnosing the slow speed issue though.

I'll post another update here in a bit when I have something to mention.

EDIT: For the curious, the ONT looks like this: https://www.adtran.com/solutions/by-segment/products/by-category/fiber-access/optical-network-terminals-ont/621.html

I see it has a standard gigabit ethernet port as well, but I'm uncertain if the service is configured to also work over that and whether it's even wise to consider seeing if anything's different on that port if it did work.
« Last Edit: July 15, 2022, 05:06:20 PM by Ixel »
Logged

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #50 on: July 15, 2022, 08:56:05 PM »

Just an update to say that the cable made no difference. The power cycle didn't either, as expected.

After some consideration, I'm considering the option of re-contracting (and perhaps migrating) the Openreach FTTP connection to an ISP such as A&A. The Lightning Fibre connection will be kept, however...

As I use BGP and have my own block of IPv4 and IPv6 addresses I will also make use of these for the browsing the internet. I should be able to setup asymmetrical routing by having it so incoming traffic goes via Openreach FTTP (whichever ISP I end up choosing) while outgoing traffic goes via Lightning Fibre FTTP. This should be possible with two GRE tunnels, one for each connection. Before I re-contract/migrate the Openreach FTTP connection I will test this possibility with the current ISP (Cerberus Networks) to ensure it's possible, but in my mind I believe it should be.

It still works out cheaper than I'm paying Cerberus Networks at the moment (out of contract for a long time).

Who knows, Lightning Fibre may fix this issue one day in the future. The above seems like my best option I guess as I have a feeling it would be a challenge to get out of the contract early with LF without paying for the contract's term, even with the issue I'm having. Having both connections also provides some redundancy I guess. :)
Logged

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #51 on: July 15, 2022, 09:19:22 PM »

EDIT: For the curious, the ONT looks like this: https://www.adtran.com/solutions/by-segment/products/by-category/fiber-access/optical-network-terminals-ont/621.html

I see it has a standard gigabit ethernet port as well, but I'm uncertain if the service is configured to also work over that and whether it's even wise to consider seeing if anything's different on that port if it did work.

Thank you. I always appreciate sight of any relevant equipment.

Just an update to say that the cable made no difference. The power cycle didn't either, as expected.

So now all the hardware related experiments have been completed.

Quote
As I use BGP and have my own block of IPv4 and IPv6 addresses I will also make use of these for the browsing the internet. I should be able to setup asymmetrical routing by having it so incoming traffic goes via Openreach FTTP (whichever ISP I end up choosing) while outgoing traffic goes via Lightning Fibre FTTP. This should be possible with two GRE tunnels, one for each connection. Before I re-contract/migrate the Openreach FTTP connection I will test this possibility with the current ISP (Cerberus Networks) to ensure it's possible, but in my mind I believe it should be.

As you have explained it, it should be feasible.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

Alex Atkin UK

  • Addicted Kitizen
  • *****
  • Posts: 5260
    • Thinkbroadband Quality Monitors
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #52 on: July 16, 2022, 12:57:45 AM »

Its certainly an expensive solution, but given you already are using BGP it should be quite a nice solution.
Logged
Broadband: Zen Full Fibre 900 + Three 5G Routers: pfSense (Intel N100) + Huawei CPE Pro 2 H122-373 WiFi: Zyxel NWA210AX
Switches: Netgear MS510TXUP, Netgear MS510TXPP, Netgear GS110EMX My Broadband History & Ping Monitors

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #53 on: July 16, 2022, 01:03:37 AM »

Done it! Seems to work fine :)





I've masked out the ISP name in the speedtest.net result because the ISP name is effectively my 'real name' due to it being from one of my own IP addresses announced via BGP.

Download is going via Cerberus Networks FTTP currently (Openreach), but I will be migrating to A&A soon. Once I do the download speed should be much better. Cerberus Networks only has 2 x 1 gigabit ports on LONAP, not a 10 gigabit port at least, so I believe there's a little congestion going on there. Multiple connections/threads won't reach gigabit either, understandably.

Upload is going via Lightning Fibre FTTP.

As for pricing, I currently pay somewhere in the realm of £150/mo for Cerberus Networks. Even paying AAISP 5TB Home::1 1000/115 and Lightning Fibre 1000/1000 (at the deal pricing I have), it's still a bit cheaper per month.
Logged

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #54 on: July 16, 2022, 01:33:42 AM »

I’m pretty sure A&A would dig into this. They have their published "no b*llsh*t" policy and they can read a TCP protocol captured because that’s where the answer lies: the problem is why does TCP hate you so much and the answer is in: the precise timing of packets, packet loss, distribution of ACKs in time, per-connection field values, window sizes, buffer size, window scale, network pipe capacity, transit times so if someone looks at all that lot and has sufficient clue, then they can work out who is to blame within those two TCP instances.

It’s not good enough an ISP proving that their network has sufficient bandwidth by using a wise technique such as iperf or multiple TCP connections to saturate the link, because although those methods do accurately measure the network rather than the software implementation of TCP and o/s performance at the two ends, the behaviour of a single TCP connection is a ‘real world’ scenario that deserves a realistic expectation of performance, it’s not something wacky, highly unusable, or unrealistic.

I am possibly getting similar problems with the performance of upstream on my system with three internet access pipes, where there may or may not be multiple TCP connections in use in some scenarios. Many speed checkers, not just one, give disappointing values for upstream throughput, compared to the carefully calculated one for TCP PDU and TCP SDU (ie payload) total throughput. In the past the issue has varied markedly over time. It might be that the fact that there is one slow link out of the three is significant. But since there have been sizeable periods where full bandwidth has been seen, then I take it that that shows, as with you, that the network can handle the expected throughput which after all is extremely slow, we’re only talking about a few hundred kbps per link and 1.6 Mbps IP PDU throughput max; hardly challenging for multi-gigabit routers. I’m convinced the answer lies in TCP behaviour, but I don’t have the concentration or the experience to read TCP protocol captures, although I can obtain them.

I can’t remember; I may have talked to AA about this, but I haven’t pestered them to do the tcpdump analysis thing for me, because I haven’t had the energy to get into this, I’ll admit, and I’ve kept doing more digging and experiments myself. In my case not only is it AAs network but it’s also their router, a Firebrick, so that’s another big reason why I can indeed legitimately ask for help, as it may be that the Firebrick is not doing the packet tx scheduling over the three unequal links in a TCP-friendly way. And that’s a whole other can of worms that you don’t have. I could go back to them and get into it again but the truth is that I’ve been really unwell at times.
Logged

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #55 on: July 16, 2022, 10:03:04 AM »

Well, I've placed an order with A&A which should be completed around 02/08 (migration). The order system charged me the wrong setup fee but I've sent them an email about it so hopefully that will be corrected next week.

I’m pretty sure A&A would dig into this. They have their published "no b*llsh*t" policy and they can read a TCP protocol captured because that’s where the answer lies: the problem is why does TCP hate you so much and the answer is in: the precise timing of packets, packet loss, distribution of ACKs in time, per-connection field values, window sizes, buffer size, window scale, network pipe capacity, transit times so if someone looks at all that lot and has sufficient clue, then they can work out who is to blame within those two TCP instances.

Yeah if this was A&A's issue they would dig into it. I was honestly expecting a bit more from Lightning Fibre, suffice to say I'm a bit disappointed with them and I shall in the next couple of weeks leave an updated review stating what's happened as well as the currently disappointing outcome. While this is an annoying issue, which as more customers signup may become worse, it has been a fairly useful test run of their customer service. Granted this isn't an ordinary "every day" kind of problem they are likely tasked with though.

It’s not good enough an ISP proving that their network has sufficient bandwidth by using a wise technique such as iperf or multiple TCP connections to saturate the link, because although those methods do accurately measure the network rather than the software implementation of TCP and o/s performance at the two ends, the behaviour of a single TCP connection is a ‘real world’ scenario that deserves a realistic expectation of performance, it’s not something wacky, highly unusable, or unrealistic.

Indeed, I even proved to them that a 'local' connection between Lightning Fibre connections was performing fine and the problem is likely somewhere between their network and the internet. Instead they are content with the fact that 'some devices' are performing fine and that their network infrastructure is fine (at least locally anyway). They haven't spent much time investigating this matter from the sounds of it and are satisfied with also shifting the blame of the issue back to me and/or the servers I'm using to perform the speed tests (which ironically would also include their own Lightning Fibre server on Ookla I guess :D).

I am possibly getting similar problems with the performance of upstream on my system with three internet access pipes, where there may or may not be multiple TCP connections in use in some scenarios. Many speed checkers, not just one, give disappointing values for upstream throughput, compared to the carefully calculated one for TCP PDU and TCP SDU (ie payload) total throughput. In the past the issue has varied markedly over time.

Ouch, that's not good. I wish my issue showed some kind of variance but sadly it seems to be any time of day or night. The only significant variance I've had is when comparing day one of the service being live to any day thereafter. Great download speed on day one and considerably slower download speed from the following day onwards.

I can’t remember; I may have talked to AA about this, but I haven’t pestered them to do the tcpdump analysis thing for me, because I haven’t had the energy to get into this, I’ll admit, and I’ve kept doing more digging and experiments myself.

Interestingly someone on Discord suggested I try looking at the 'sniffer' function in RouterOS and doing some comparisons to see if I can notice any anomalies which might explain what's going on. It's something I will try this morning, I'll just use RouterOS's own 'bandwidth test' feature this time as it gave similar results to iperf3 and other single TCP stream speed tests anyway.
Logged

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #56 on: July 16, 2022, 11:56:14 AM »

EDIT: Sadly this is normal behaviour from what I've been told. Another person who has an IPoE/DHCP type of connection is seeing something similar on a different ISP. Sadly it's unlikely to be the reason.

---

I have some more news to share, a possible discovery but I think I need the assistance of this forum in understanding whether this is correct or not.

I went ahead with using 'sniffer'. I compared both Cerberus Networks connection and Lightning Fibre connection for receiving packets using a single stream bandwidth test.

This is on the Cerberus Networks connection:
https://i.imgur.com/osZWx5Y.png

This is on the troublesome Lightning Fibre connection:
https://i.imgur.com/TeFOgz4.png

The packet size on the Lightning Fibre connection is inconsistent and going well above 1500, unlike Cerberus Networks. Of course, Cerberus Networks is PPPoE while Lightning Fibre is IPoE/DHCP so I'm not sure if that's why I'm seeing this massive difference.

I'd like to hear what anyone else here has to say about this, is this possibly normal behaviour for an IPoE/DHCP connection and not the reason I'm getting a slow throughput?
« Last Edit: July 16, 2022, 12:47:13 PM by Ixel »
Logged

bogof

  • Reg Member
  • ***
  • Posts: 435
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #57 on: July 16, 2022, 12:52:36 PM »

Could it be jumbo frames related?
What package are you going for with AAISP?  I will perhaps be shopping for a new provider soon given what I seem to be seeing with my Zen connection at the moment.  Be interesting to see what it looks like when you get it set up.
Logged

Ixel

  • Kitizen
  • ****
  • Posts: 1282
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #58 on: July 16, 2022, 02:35:22 PM »

Could it be jumbo frames related?
What package are you going for with AAISP?  I will perhaps be shopping for a new provider soon given what I seem to be seeing with my Zen connection at the moment.  Be interesting to see what it looks like when you get it set up.

That's what I'm thinking, but from what I heard this is possibly normal. Although someone pointed out to me that there's also potentially a pattern going on where as on their connection they don't see something like that. It's very puzzling to say the least.

As for the package, I've chosen Home::1 5TB (1000/115).

Zen... I used to be with them some years ago. In the early days of using Zen it was fantastic but going back to the time just before I left Zen and switched to AAISP I remember having some throughput issues with Zen which were never resolved. I wasn't their only customer suffering similar issues at the time either. Zen released me from the contract without me needing to pay an early termination charge after I left them a detailed and factual negative review on Trustpilot.
Logged

bogof

  • Reg Member
  • ***
  • Posts: 435
Re: Diary of My Experiences with Lightning Fibre (FTTP local ISP)
« Reply #59 on: July 16, 2022, 04:00:51 PM »

That's what I'm thinking, but from what I heard this is possibly normal. Although someone pointed out to me that there's also potentially a pattern going on where as on their connection they don't see something like that. It's very puzzling to say the least.

As for the package, I've chosen Home::1 5TB (1000/115).
Thanks, that's the package I've been considering.  Our usage "only" gets up to around 2TB a month and that is with working from home and lots of streaming, so the 500/800GB limits would be too low but the 5TB tier would be fine.   I'll be watching with interest.

The only thing that really concerns me slightly with AAISP is that given they're a bit of a niche player, whether they have good links into the various content delivery peering systems that are around and improve our streaming experience.  For that reason (and, frankly, cost too) I'm quite tempted to give BT Business a go.
Logged
Pages: 1 2 3 [4] 5
 

anything