Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[GigabitEthernet]

Hi All,

I have my 500Mbps TalkTalk connection now installed.

However I am still seeing bufferbloat, I thought this wouldn't be an issue on FTTP? I am using the TalkTalk Hub. On download I see the ping go from 5ms to 100 or more and upload is similar.

Do I need a new router?

[Ixel]

Bufferbloat can still happen depending on how much the connection is being used at the time. I can experience some bufferbloat on gigabit using a multiple connection speed test (typically around 40-50ms) on the download. Upload on the other hand doesn't appear to be suffering with bufferbloat here.

I could setup something like fq_codel or sfq on my router if I was really that bothered but I haven't, but bear in mind that at such a high speed there aren't too many routers (referring to typical routers at home) out there which can use QoS and still maintain most of the speed. The CPU in the router may struggle.

How are you measuring your bufferbloat, what tool(s) are you using?

[Alex Atkin UK]

I don't seem to get any for downstream, presumably as I'm not hitting full Gigabit, it tends to peak at 915Mbit which makes me think maybe Zen have a slightly lower cap at their end which honestly is a good thing if it prevents this.

Upstream I do see some jitter, I reduced this by enabling QoS on upstream only, but during a speedtest it still shows jumps to 200ms though the final result does not reflect this (just goes to show how you can't really trust the summary).

You also have to bear in mind, bufferbloat is only a problem if you are maxing out the connection.  A speedtest deliberately is saturating the link, a faster connection makes this less likely to happen in real-world use.

[Weaver]

I hate the way that operating systems have by-default transports such as TCP always guzzling 100% of the link, because their aim is just to maximise speed at all costs. Sane operating systems ought to have an option to use x% of the current internet access link capacity, or no more than a specified fixed rate, or a low-priority mode. Apps should be able to add parameters to each connection setup that select these options for upstream or downstream, or even better do so indirectly by specifying references to named speed-control profiles, these profile objects also being correct for various different internet access link types eg "ethernet>FTTP" or "4G" or "5G". Also there should be policies so that these speed controls can be forced onto apps that just use the defaults, or where the code is too old to support these new parameters. Then a UI could control the speed of particular apps / processes / classes of apps. An important feature would be support for initial-burst mode speed control profiles, so you are allowed to do something different for the first n ms, typically going flat-out or at a high x%, and after the initial duration, your speed control parameters change typically to something slower.

Certainly routers and operating systems all need to implement sane queue management, but making ‘flat-out’ a rarity will help make life more bearable, and instead of flat-out everywhere we have a typical default factional speed setting for ‘fast’ mode of ds=98%/us=98% for fast links, or maybe ds=95%/us=95% for slower links.

[Alex Atkin UK]

I noticed they have added DSCP and WMM tagging on Xbox, but its off by default and hidden in Advanced Settings.  Its just on or off though, so I suspect game downloads would still go as fast as possible (not hitting Gigabit so far), with online gaming traffic getting a high priority tag.

Of course it wont help much given how routers capable of supporting it are usually bought for broadband faster than they can handle with it enabled.  I can't see people on DSL buying £300 routers.

Plus you need QoS at the other end of the link too, so download traffic can be dropped higher up the chain.  Not sure any ISP does that.

[Weaver]

The remote end isn’t at an ISP; the L4 far end is at some random server and once say Linux gets speed management priority fractioning and throttling into the o/s then servers will all have the ability and using policies, I suggest that the local end send a new type of ICMP message to the remote o/s telling it the speed controls that it should apply to particular processes or flows or apps. So I imagined that the local end would remote control the remote end. Remote end can slow down TCP by delaying ACKs, but that could backfire with retransmissions if one gets it wrong. And not everything is TCP; there are UDP apps, the new transport QUIC, which would be easy to upgrade to be controllable like my proposed ‘new o/s TCP’ when QUIC is just a library, before it ends up being integrated into the o/s.


AA does downstream speed throttling by ingress queue management before they hand traffic over to BT or TalkTalk, so downstream flows at 1 Gbps don’t penetrate downstream into the carrier’s network only to hit a 5 Mbps first mile bottleneck where everything gets dropped.

[meritez]

Hi All,

I have my 500Mbps TalkTalk connection now installed.



However I am still seeing bufferbloat, I thought this wouldn't be an issue on FTTP? I am using the TalkTalk Hub. On download I see the ping go from 5ms to 100 or more and upload is similar.

Do I need a new router?

If you plug your pc directly into the ont, do you still see the bufferbloat?

[Alex Atkin UK]

The remote end isn’t at an ISP; the L4 far end is at some random server and once say Linux gets speed management priority fractioning and throttling into the o/s then servers will all have the ability and using policies, I suggest that the local end send a new type of ICMP message to the remote o/s telling it the speed controls that it should apply to particular processes or flows or apps. So I imagined that the local end would remote control the remote end. Remote end can slow down TCP by delaying ACKs, but that could backfire with retransmissions if one gets it wrong. And not everything is TCP; there are UDP apps, the new transport QUIC, which would be easy to upgrade to be controllable like my proposed ‘new o/s TCP’ when QUIC is just a library, before it ends up being integrated into the o/s.


AA does downstream speed throttling by ingress queue management before they hand traffic over to BT or TalkTalk, so downstream flows at 1 Gbps don’t penetrate downstream into the carrier’s network only to hit a 5 Mbps first mile bottleneck where everything gets dropped.

The server tags the packets but only your ISP knows what speed you have provisioned so THAT is where incoming QoS has to take place, re-ordering by priority and dropping lower priority packets to let the higher priority through if you've exceeded the capacity.  They already do this but without any tags to let them know some are more important than others, so they just drop when the buffer is full.

Like you said, AAISP do it like anyone else.

[Chrysalis]

Windows does have built in QoS which I think is capable of restricting throughput, but the issue is windows has no knowledge of your internet connection speed, only the local connection speed.  So e.g. 95% would be 95% of a gigabit LAN link not a 70mbit VDSL link.

Whats better perhaps is if much less aggressive congestion algorithms were used by default, preferably ones based on RTT instead of packet loss, as ones based on RTT throttle back much earlier since latency goes up before packets start been dropped.  Its like the TCP fast open problem described on the QUIC thread, it will take a decade for defaults to change.

In addition automatic rwin and send window adjustments tend to be too aggressive.  They only really serve to ensure they not too small not the other way round in practice.

Artificially adding latency to shaper low priority pipes used for bulk downloads can work well as they slow down ramping up of speeds and require a larger congestion window to hit a given speed.

A reminder of why I want FTTP, yesterday used twitch leecher to download a twitch video, and was trying to stream another one at same time, the stream got completely starved as the download saturated my connection, a complete lack of fairness.  Once I have FTTP that will be history. Cant really put twitch leecher into a bulk download queue, it has the same source ip, dest ip/ports as twitch streams, but might ask the dev of the app to add a speed throttle feature to the app.  I have these problems on VDSL, Weavers must be crazy on ADSL.

[Weaver]

Yes, my idea would require that operating systems find the speed of the bottleneck in both directions. I thought that this might require info from middleboxes solicited and returned by a new protocol; that would make deployment really near impossible. It needs to be done with a bottleneck-finding algorithm that proves the path upstream all driven by a machine inside the LAN, or the router, but that’s a hard problem. AA nowadays tells me what the speeds down and up are for a PPPoE connection that is the internet access link bottleneck.

[EC300]

I'm using pfSense with QoS and no buffer bloat at all.  Get an A and A rating on Thinkbroadband for buffer bloat, not so when QoS is turned off.

It should be possible to use QoS on the router to remove buffer bloat, just depends on the routher.

[craigski]

The server tags the packets but only your ISP knows what speed you have provisioned so THAT is where incoming QoS has to take place, re-ordering by priority and dropping lower priority packets to let the higher priority through if you've exceeded the capacity.  They already do this but without any tags to let them know some are more important than others, so they just drop when the buffer is full.

Like you said, AAISP do it like anyone else.

You said ISPs didn't do this a few days ago on the Zen GEA thread Alex

It literally isn't, by prioritising traffic you are breaking net neutrality, the whole premise is all traffic is treated equally.

[XGS_Is_On]

I hate the way that operating systems have by-default transports such as TCP always guzzling 100% of the link, because their aim is just to maximise speed at all costs. Sane operating systems ought to have an option to use x% of the current internet access link capacity, or no more than a specified fixed rate, or a low-priority mode.

Virtually no-one would use that feature and it adds both unnecessary complexity and unnecessary overhead in terms of CPU and memory to the product for that vast majority that won't use the feature.

Apps should be able to add parameters to each connection setup that select these options for upstream or downstream, or even better do so indirectly by specifying references to named speed-control profiles, these profile objects also being correct for various different internet access link types eg "ethernet>FTTP" or "4G" or "5G".

Virtually no-one would use that feature and it adds both unnecessary complexity and unnecessary overhead in terms of CPU and memory to the product for that vast majority that won't use the feature. In addition in this case it's both prone to misuse and very few applications actually saturate links for any length of time.

there should be policies so that these speed controls can be forced onto apps that just use the defaults, or where the code is too old to support these new parameters. Then a UI could control the speed of particular apps / processes / classes of apps. An important feature would be support for initial-burst mode speed control profiles, so you are allowed to do something different for the first n ms, typically going flat-out or at a high x%, and after the initial duration, your speed control parameters change typically to something slower.

Virtually no-one would use that feature and it adds both unnecessary complexity and unnecessary overhead in terms of CPU and memory to the product for that vast majority that won't use the feature. In addition in this case it's both prone to misuse and very few applications actually saturate links for any length of time and most of those are downstream saturation so only matter once data has reached the host.

routers and operating systems all need to implement sane queue management, but making ‘flat-out’ a rarity will help make life more bearable, and instead of flat-out everywhere we have a typical default factional speed setting for ‘fast’ mode of ds=98%/us=98% for fast links, or maybe ds=95%/us=95% for slower links.

Saving the biggest issue for this comment. A single device's operating system has no way of knowing what other devices on the network are doing, so has no way of properly controlling its own throughput in order to not trigger congestion. Operating systems may mark traffic to assist the router that actually handles the WAN connectivity in making decisions but cannot shape traffic themselves with any good results unless they're the only client using that WAN. As far as marking goes it's prone to misuse as mentioned previously and should be done upstream on routers or switches.

Implementing any kind of queue management on all routers in homes isn't going to work so well. QoS is best done outbound so the QoS needs to come from whatever is upstream of the bottleneck. QoS inbound you can't actually do anything beyond just drop things and let the protocol manage that loss as it sees fit.

There's no business case for the kind of mechanisms needed here. We'd need an accurate and timely way of determining the available bandwidth on a link, devices using that data to populate a series of queues, then those devices managing traffic according to the queues with no idea what is actually most important beyond taking hints from the customer side which may or may not be accurate.

[XGS_Is_On]

The remote end isn’t at an ISP; the L4 far end is at some random server and once say Linux gets speed management priority fractioning and throttling into the o/s then servers will all have the ability and using policies, I suggest that the local end send a new type of ICMP message to the remote o/s telling it the speed controls that it should apply to particular processes or flows or apps. So I imagined that the local end would remote control the remote end. Remote end can slow down TCP by delaying ACKs, but that could backfire with retransmissions if one gets it wrong. And not everything is TCP; there are UDP apps, the new transport QUIC, which would be easy to upgrade to be controllable like my proposed ‘new o/s TCP’ when QUIC is just a library, before it ends up being integrated into the o/s.


AA does downstream speed throttling by ingress queue management before they hand traffic over to BT or TalkTalk, so downstream flows at 1 Gbps don’t penetrate downstream into the carrier’s network only to hit a 5 Mbps first mile bottleneck where everything gets dropped.

Everyone using BTW or whatever manages the traffic before they send it to the carrier, they're required to. They can only go by the connected speed, they have no idea what the actual performance of the connection is. What A&A are doing is nothing special, they just offer a button to cap it further and use basic WFQ so that small packets get priority. Nothing arcane or interesting and if actual performance drops below the artificial cap A&A provide bufferbloat will happen.

Regarding your proposal it'd be horrendously chatty and, of course, it's impossible for a remote host to signal what the local host should do with a particular process or app - it can only go by the combination of layer 3 and 4 headers. Adding something else to those in terms of app-based rules would an a huge amount of additional processing and complexity. You've just taken routers and BRAS and turned them into layer 7 gateways.

[XGS_Is_On]

Yes, my idea would require that operating systems find the speed of the bottleneck in both directions. I thought that this might require info from middleboxes solicited and returned by a new protocol; that would make deployment really near impossible. It needs to be done with a bottleneck-finding algorithm that proves the path upstream all driven by a machine inside the LAN, or the router, but that’s a hard problem. AA nowadays tells me what the speeds down and up are for a PPPoE connection that is the internet access link bottleneck.

The only way to find the capacity of a link is to saturate it all the time and measure the amount of throughput required to saturate it which in turn potentially pollutes the useful throughput measurement. There is no other way. To find the capacity limit of a link you have to hit it and know how much throughput is required to hit it.

All A&A are doing is reporting the records BTW/TT/whomever provide them as to your sync speeds. They have no idea what the actual available throughput between them and you is let alone between you and some host on the Internet.

Regarding middleboxes those don't matter. The only thing that matters is the end-to-end throughput. The only way to measure this is to saturate the link all the time. This in turn results in enormous amounts of wasted bandwidth being used purely to measure capacity.

This is actually a thing for very secure networks where they resist traffic analysis by saturating all links between all nodes as much as possible but isn't viable for regular Internet usage.