Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: modern TCP window sizes way too aggressive for DSL lines.  (Read 3123 times)

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
modern TCP window sizes way too aggressive for DSL lines.
« on: December 10, 2021, 08:13:07 PM »

I think back to when I had 1/2 mbit ADSL and the question why did that connection never have issues when downloading stuff, and its all to do with how aggressive data is sent.

Now days we have congestion providers designed to send as much as possible for fat connections, aggressive recovery from packet loss etc. may be some justification in the days of high RTT, but now we also have CDN's which keep RTT low and multi threading.

I mentioned the BBR algorithm in a thread weaver created, this aims to have "just enough" of a window size to hit high 90s capacity of a pipe.  But hardly anyone uses it currently.

So today I am downloading a game from ubisoft's launcher, I had to throttle the download to less than 30% of my throughput capacity to get packet loss low enough to an acceptable level, was still jitter.  A very aggressive throttle.  However I then tried another trick, I set windows auto tuning to disabled which is a brute force way of forcing low window size capped to 65k, the traffic graph went from a spiky mess with all its overshoots to a almost flatline at the cap I set and latency and loss were as if the connection was idle, I was able to raise the throughput to only a few meg/sec below my capacity and was still no obvious packet loss.  All from reducing the window size.

I then sniffed the traffic.  I had already confirmed a window size of 64k is enough to saturate my line, but if I left auto tuning enabled at the default normal setting it was negotiating a whopping 670kB window, over 10x what is needed, it can be capped to 256k using the highly restricted setting, but that offers only a small improvement.  I think window auto negotiation is pretty broken at the moment, as if everyone is on gigabit or multi gigabit connections.

Of course capping my receive window to 64k isnt without consequence, anything not on a local CDN will have lower throughput which may or may not matter (not so much for web browsing, but more so if downloading), as well as anything single threaded might suffer as well such as streaming, although these are usually via local CDN so the effect is typically a slower initial buffer but still able to stream high definition.

Has anyone ever tried to tame this behaviour?  The obvious way is plusnet style via packet inspection and modifying the window sizes, but I am thinking maybe just adding a high amount of artificial latency via dummynet to steam/ubisoft/origin/sony/xbox etc.
Logged

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #1 on: December 10, 2021, 09:41:11 PM »

Is this the window scale option that we’re talking about in Windows? Apologies if it isn’t - it’s just that the "highly restricted" setting rings some bells from long, long ago. I’m probably talking utter nonsense.
Logged

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #2 on: December 11, 2021, 10:48:48 AM »

Yes I am talking about windows settings, I might leave this set to disabled routinely now until I have gigabit FTTP as most things I access are low RTT via CDN, and only increase on occasions I need it.

On disabled I had no issues downloading a game, downloading new LTSC iso and watching netflix HD at same time.

For my consoles I cap their throughput in firewall.
Logged

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #3 on: December 11, 2021, 11:04:54 AM »

As far as I’m aware, this just prevents or restricts the use of the window scale value scaling mechanism in the header. This was because some routers didn’t understand the window scale fields at all, no?
Logged

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #4 on: December 11, 2021, 04:52:43 PM »

Modern versions of windows rely on automatic TCP window tuning, and your only control is adjusting the aggressiveness of that tuning, I believe the reason you can turn it off is as you said there was concerns of incompatible equipment.  It basically runs in a safe mode, the window is below 65k, (in my case 64240 a multiple of MSS) and at a fixed value.  So yes scaling is disabled.

However given how much of internet content now days comes from local CDN or even caches inside the ISP network, the need for large buffers has diminished on more modest internet speeds.  Even just now streaming a single threaded youtube video, with the scaling disabled it was still pushing the data at about 80% of my line rate.  So I will be keeping my desktop with it disabled now, and if I was on your bonded ADSL setup I definitely would do it.

The problem I have though is I believe in managing my network centrally, not on each client device.  If it were possible I would manipulate the rwin on all traffic going through my firewall but it isn't.  Sure I could cap the speeds, which would reduce the congestion window a little but ultimately the window would still be set way too high and lots of retransmissions and packet loss.

So whilst this may have been intended as a troubleshooting option its still something that can be used to manipulate performance.  In the older XP days, there was no auto tuning, you could enable scaling and override it, but it was still running at a fixed value.

Other operating systems offer far more control than windows.
Logged

Alex Atkin UK

  • Addicted Kitizen
  • *****
  • Posts: 5272
    • Thinkbroadband Quality Monitors
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #5 on: December 12, 2021, 12:43:49 AM »

Surely the window WONT be too high as that's the whole point, its supposed to scale to the conditions.  If the router is throttling then surely it will cause it to scale down?
Logged
Broadband: Zen Full Fibre 900 + Three 5G Routers: pfSense (Intel N100) + Huawei CPE Pro 2 H122-373 WiFi: Zyxel NWA210AX
Switches: Netgear MS510TXUP, Netgear MS510TXPP, Netgear GS110EMX My Broadband History & Ping Monitors

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #6 on: December 12, 2021, 04:10:30 AM »

Hence me saying its broken Alex.

It doesnt work perfect thats why we have constantly new congestion algorithms been developed.

When I did tests from one of my servers, bbr used a 46k window to max out my line single threaded FTP download, cubic used a 412k window to do the same thing, the latter caused packet loss on all internet activity as it flooded my line.
« Last Edit: December 12, 2021, 11:53:46 AM by Chrysalis »
Logged

Reformed

  • Reg Member
  • ***
  • Posts: 318
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #7 on: December 12, 2021, 11:49:48 AM »

Shouldn't TCP slow start help a bit?

https://datatracker.ietf.org/doc/html/rfc5681

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #8 on: December 12, 2021, 11:52:28 AM »

Slow start is used initially then it gets handed over to the congestion provider, however many now cheat slow start as well, e.g. jumping start to 10 packets or 44 packets without an acknowledgement.  There is also a tweak to skip slow start on linux, although I havent heard of cases of the big tech companies using it. (they do however cheat slow start, both google and cloudflare openly boast about it).

I think the reason all this has happened, is a lot of the algorithms get developed with servers and large isp's in mind, so they often optimised for multi gigabit or multi 10 gigabit pipes, the number one complaint from end users is "I want to download faster".  So raw throughput has got prioritised.
« Last Edit: December 12, 2021, 11:56:44 AM by Chrysalis »
Logged

Reformed

  • Reg Member
  • ***
  • Posts: 318
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #9 on: December 12, 2021, 12:19:32 PM »

It's certainly optimised for high BDP and always has been. A number of applications actually open more and more flows in an attempt to saturate capacity.

A router with decent QoS configuration should be able to handle this fairly gracefully. The odd one also has some more advanced shaping features either built in or available as a plugin.

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 7390
  • VM Gig1 - AAISP L2TP
Re: modern TCP window sizes way too aggressive for DSL lines.
« Reply #10 on: December 16, 2021, 06:33:45 PM »

At some point I may trial cake on a Linux gateway, just to see if it handles things better than pfSense/opnsense but for now I am just keeping the RWIN unscaled down as its working very effective for me.
Logged
 

anything