Kitz Forum

Broadband Related => FTTC and FTTP Issues => Topic started by: tommy45 on January 08, 2023, 09:26:50 PM

Title: Zen's GEA Network & Abnormal Buffer Bloat
Post by: tommy45 on January 08, 2023, 09:26:50 PM
 Please see  my link in ttb https://forums.thinkbroadband.com/zen/4728653-zens-gea-backhaul-abnormal-buffer-bloat.html#Post4728653
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: kitz on January 12, 2023, 05:30:13 AM
I have no problem with people linking to other reputable sites but the format of the above is unhelpful when it asks out members to go read a problem on another site and presumably responding over there.   There wasn't even a summary of the post so that people could comment here.  I'm certain that most forums would find this style of post unacceptable.   For future reference.. by all means ask a question here too and say youve also asked at tbb, but at least have the courtesy to actually make post content here rather than taking the whole thing off-site.   Thanks.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Chrysalis on January 12, 2023, 01:23:36 PM
Since you made the thread here tommy.

Have you considered doing your own traffic management locally?
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Alex Atkin UK on January 12, 2023, 03:22:54 PM
From what I can gather they are of the attitude "it was fine before so I shouldn't have to manage it locally".  They resent the idea they might need to cap their speeds, losing a little in the process.

As I posted there, they've been lucky, I've always had to use QoS on Zen, Plusnet, and the multitude of others ISPs I've used over the years to manage bloat under load.

I agree an ISP should try to throttle downloads to never exceed the pipe (and they do, its in their interest not to waste backhaul capacity), but as others have posted, keeping the buffers small/empty would be detrimental to an ISP and customers speed overall.  There has be a balance to allow heavy downloaders to get full benefit, while people needing low latency use local QoS to smooth out the experience.  Also, upstream QoS can only be managed locally, as you are the one sending, which is why the BT HG612 config by default chops ~1Mbit off upstream.

I'm in both camps, which is why I consistently pay for the fastest broadband I can get and spread the load over 5G too so even heavy downloads rarely max out my primary connection these days.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: XGS_Is_On on January 12, 2023, 06:15:18 PM
Don't use TBB anymore so will ask here: the example of the 'issue' given was a speedtest. Do many people actually notice bufferbloat in normal usage?

It's not a coincidence that complaints about it jumped when Ookla started incorporating it into tests.

I see very high bloat when using WiFi or gigabit Ethernet as the broadband connection is so much faster than them but can't say it really impacts anything.

Nothing personal, Tommy, I just really despise people chasing numbers for numbers' sake. Doesn't seem a great use of time or emotional energy.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Alex Atkin UK on January 12, 2023, 06:32:35 PM
Don't use TBB anymore so will ask here: the example of the 'issue' given was a speedtest. Do many people actually notice bufferbloat in normal usage?

I used to have web pages very slow to load if another client was maxing the line, but that's more about fair queuing than bloat so naturally can only be handled locally.

I mostly used to use QoS a long time ago to actually prioritise certain clients, for example one year I watched E3 on Xbox and that box had priority so I was able to download on another without the stream glitching/dropping quality, as it had higher priority.

Over time I tweaked it to reduce bloat mostly because I could, not out of needing it.

Its a strange thing, as seeing bloat when fully loaded isn't really telling you anything, other than you're managing to max out the connection.  Like you said, its totally expected and working as designed.  Its the reason games consoles and Steam deliberately limit the download speed (or don't download at all by default in Steam I believe) when running games.

I do still QoS on upload just to cap it a bit below line rate, just in case a VoIP call comes while doing a heavy download, but honestly its kinda hard to max out the line most of the time anyway.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: XGS_Is_On on January 13, 2023, 12:21:47 AM
As I said over there before I took my bat and ball home it's either bloat, increase risk of loss by using a tiny buffer or reduce throughput. Buffer bloat is controlled by dropping packets more quickly to trigger congestion control to slow heavy connections before they max the line, while potentially allowing other traffic through.

This can be hidden on a BQM or the Ookla test by using fair queueing so prioritising pings but won't change how the actual data bearing packets flow. To do fair queueing or anything else requires limiting a connection slightly below full throughput. It reaches full throughput all bets are off and it's every flow of data for themselves and the higher bandwidth flows will crowd out the smaller ones.

The earlier carriers TTB and BTW were doing this. Looks as though Zen aren't, which goes some way to explaining the higher download speed on the Zen test shared.

Carriers can absolutely do things to their networks to both smooth out BQMs and make the buffer bloat numbers on Ookla look good without it making any difference at all to normal traffic.

Still: after how warmly my comments were received on the other place I guess what do I know? Not like building hierarchical QoS policies for anything from heavily overbooked hub appliances balancing hundreds or thousands of spokes with many times more bandwidth available than the hub to trying to manage to preserve QoE in branch offices running over ADSL, 3/4G and sometimes 2 Mbit geostationary satellite is a notable part of what I do for a living or anything.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: XGS_Is_On on January 13, 2023, 12:32:18 AM
I used to have web pages very slow to load if another client was maxing the line, but that's more about fair queuing than bloat so naturally can only be handled locally.


Yep. Can only be managed outbound from router towards LAN that one.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: XGS_Is_On on January 13, 2023, 04:20:16 PM
Still: after how warmly my comments were received on the other place I guess what do I know? Not like building hierarchical QoS policies for anything from heavily overbooked hub appliances balancing hundreds or thousands of spokes with many times more bandwidth available than the hub to trying to manage to preserve QoE in branch offices running over ADSL, 3/4G and sometimes 2 Mbit geostationary satellite is a notable part of what I do for a living or anything.

I apologise, everyone. That was a blatant appeal to authority and was not appropriate.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Weaver on January 13, 2023, 04:34:39 PM
> I apologise, everyone. That was a blatant appeal to authority and was not appropriate.

You are forgiven. :)
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: dee.jay on January 13, 2023, 04:40:24 PM
I apologise, everyone. That was a blatant appeal to authority and was not appropriate.

I didn't think you needed to apologise, I read your statement, breathed a sigh and thought "Yeah, I've been there before"
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: burakkucat on January 13, 2023, 08:15:29 PM
Still: after how warmly my comments were received on the other place I guess what do I know? Not like building hierarchical QoS policies for anything from heavily overbooked hub appliances balancing hundreds or thousands of spokes with many times more bandwidth available than the hub to trying to manage to preserve QoE in branch offices running over ADSL, 3/4G and sometimes 2 Mbit geostationary satellite is a notable part of what I do for a living or anything.

I apologise, everyone. That was a blatant appeal to authority and was not appropriate.

Upon reading I didn't get one single "tingle in the whiskers" so, as far as I am concerned, all was (and still is) good.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: meritez on January 13, 2023, 09:52:15 PM
Don't use TBB anymore so will ask here: the example of the 'issue' given was a speedtest. Do many people actually notice bufferbloat in normal usage?


No, but I'm a fan of CAKE, quite partial to CAKE actually.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Alex Atkin UK on January 14, 2023, 01:54:39 AM
Still: after how warmly my comments were received on the other place I guess what do I know? Not like building hierarchical QoS policies for anything from heavily overbooked hub appliances balancing hundreds or thousands of spokes with many times more bandwidth available than the hub to trying to manage to preserve QoE in branch offices running over ADSL, 3/4G and sometimes 2 Mbit geostationary satellite is a notable part of what I do for a living or anything.

I for one very much appreciate having someone so knowledgeable about this stuff around to confirm that finally there is something I actually somewhat understand in networking. ;)

That's one of the good things about this forum over the others, we're a smaller group of people who (mostly) want to understand the specifics rather than get defensive when were proven wrong.  Its sad the bigger broadband forums have devolved away from that mindset, but then that seems to be the Internet in general sadly.

It makes me so sad to see the general Internet has turned into people blaming everyone else for their problems, expecting the impossible, rather than people looking for the knowledge to do things themselves.  Fortunately I've gotten kinda used to just focusing on the useful people, even if I will fall into the trap of trying to correct people sometimes.  Its been a good tool to learn to ignore people calling me names.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: XGS_Is_On on January 14, 2023, 07:07:46 PM
The fundamentals don't change whether it's a home broadband connection or a datacentre with 4 ECMP (https://en.wikipedia.org/wiki/Equal-cost_multi-path_routing) links.

What's happened here is previously TalkTalk Business / BT Wholesale were rate limiting the connection to Zen a little below maximum. Over GEA and Zen's Plexus network they evidently aren't doing this and are feeding their customer a 'clean' unshaped connection. This is perfectly legitimate but certainly may be annoying, especially for people who use one of the many speed tests showing buffer bloat.

Either, or: higher throughput or lower buffer bloat, lower throughput, potential for packet loss earlier when loading the link up.

When I'm building a policy I'll have queues for various types of traffic. VoIP / video conferencing gets top priority and has a short maximum wait time it may be buffered for before drop, interactive applications like SSH and RDP go next with a higher maximum wait time. Both of them also get guaranteed bandwidth if they need it, so a CIR where other traffic may not. Random junk gets no CIR/guarantee, lowest priority and can sit in buffers for up to half a second.

If a broadband customer is really concerned about the quality of their service they could take back control, take 5% from their maximum speed and implement a QoS policy. Bufferbloat numbers from speed tests are not representative of actual experience. Pings get through more quickly when the circuit is loaded: so what? Your web browsing for example isn't using small packets so the usual way of making the buffer bloat numbers look nice, some fair queuing that lets small packets use the gaps in between bigger ones, isn't going to help.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Chrysalis on January 14, 2023, 10:25:25 PM
Sounds like we agree, sums up what I replied to him on TBB.

"Bufferbloat is not necessarily bad, probably easier to manage via shaping, than if an isp has tiny buffers and chooses to drop packets instead."
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: tommy45 on January 31, 2023, 11:49:45 PM
UPDATE

I have been in contact with support and after explaining a few times what the issue was Excessive buffer bloat whilst downloading , They set up monitoring of the latency but their graph didn't really show the detail needed , so i sent them snapshots of my tbqm and f8lure graphs and they agreed this was not normal,

so to rule out my equipment, they asked if i would mind if they sent me a fritzbox to rule this out , i set up the new router and tested to see if it had resolved the issue, which It had, so out of curiosity i swapped back to my old set up, and the problem did not re-occur, to the cause ???? maybe some kind of glitch with the DSLAM /line card or the modem that cleared as a result of connecting to a different modem ?

And the old hardware had had several re synchs following the network migration in April ,although it had increased since then , as a result i lost the full synch and 5db snr i obtained following a re-synch following a brief (under 30sec) power outage, But buffer bloat back to normal ,infact a slight improvement on upstream buffer bloat when several FTP threads are active
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: burakkucat on February 01, 2023, 01:14:53 AM
Very strange.  :-\
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: Alex Atkin UK on February 01, 2023, 05:40:57 AM
I wonder, could the old modem have a problem that will only show after its been powered up for a certain period of time?  I know capacitors can change depending on if they are warm/cold, its usually in favour of being warm I think but could always be the other way around?

I'd always tend to side on swapping out the PSU if a modem is behaving oddly.
Title: Re: Zen's GEA Network & Abnormal Buffer Bloat
Post by: tommy45 on February 01, 2023, 02:04:59 PM
Well, if i see the  same or  other issues again, i have the PSU from the ECI modem i haven't used  since replacing it with the HG612, So will try  that 1st, Maybe also try a direct connection to PC to isolate  the billion router