https://forums.thinkbroadband.com/zen/t/4709246-slow-speed-after-gea-migration.html
Any thoughts folk?
I had completely forgotten about that TBB forum thread and so had another three pages of posts to read through. Phew! :oIt does read like a tale of woe. Not so much for the issue itself, but for the hoop-jumping by the end users.
My only thought is "currently avoid Zen".
How can you tell if you have been gea migrated?
Thanks. At the moment it would appear I haven't been migrated. I will keep an eye on this.I guess note that this I think only indicates if you are migrated from one mechanism to the other; I'm not sure if it is possible to know whether you were perhaps already migrated into Zen GEA at the point you signed up by virtue of Zen already having the capacity in your exchange when you signed up.
This is certainly interesting going forward given Zen have said for a while now they hope to have their own cable links to all headend exchanges eventually.I tried a few different PCs today as PPPoE clients and had varying results, enough to make me think there might be a bit of a gateway lottery too thrown into the bargain. The results were still good enough to indicate though that the issue is unlikely to be my hardware - I can get close to wire speed to some places, just not Zen's own server...
What really baffles me is how the device at the end of the ONT made a difference if the problem is packet loss across the L2S.
Kinda disappointed nobody tested pfSense given MacOS seemed to be the only OS to mitigate this issue.
I guess note that this I think only indicates if you are migrated from one mechanism to the other; I'm not sure if it is possible to know whether you were perhaps already migrated into Zen GEA at the point you signed up by virtue of Zen already having the capacity in your exchange when you signed up.
If you have a connection that is currently getting you >900mbps up / >100mbps down, I'd be interested to know what your speedtest.net wired results are to Zen and to Swish Fibre's servers.
I've spent a bit of time on the phone to the (very nice) staff at Zen going through various things; it seems like my issues have been escalated to a faults team. Will be interesting to see what that brings.
Swish fibre https://www.speedtest.net/my-result/d/457bad96-77f6-4a4c-b530-bd8a4936f3e9Thanks, this confirms that there isn't a more general issue with the Zen speed tester server. Nothing I do can get me close to your Zen number, in spite of it being almost exactly what I was getting pre-migration. What a pain. We will see what they come back with.
Zen https://www.speedtest.net/my-result/d/0e8c7ca4-7f94-4622-a88f-07aafecb5bc3
both over 900 down with my own opnsense router.
I'm kinda surprised you didn't know this already.
I 'kinda' knew this, but not the detail, as I hadn't used all the speed test sites before. My post was really to highlight there can be vastly different results in theserandom number generatorsspeed test sites, and to be careful before jumping to conclusions with results.
I 'kinda' knew this, but not the detail, as I hadn't used all the speed test sites before. My post was really to highlight there can be vastly different results in theseI suppose what you're saying really is that all tools can be dangerous in the wrong hands. Speed tests are no different than anything else in this respect.random number generatorsspeed test sites, and to be careful before jumping to conclusions with results.
05/07/2022
21:28
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 10 ms 12 ms 7 ms lo0-0.bng4.thn-lon.zen.net.uk [51.148.77.132]
Trace complete.
"Zen Internet - London","40788","4.599","1.639","0","68713527","13897741","790490390","54507570","https://www.speedtest.net/result/c/232c51fd-e5bb-4111-af54-e8ad97c6217c","1"
"Swish Fibre - London","34948","4.985","1.359","0","92428045","13882102","948597120","50126332","https://www.speedtest.net/result/c/03e7b878-3cba-4856-ad8e-eb7dfe824150","1"
"Zen Internet - London","40788","4.85","0.38","0","70256688","13896955","553698831","50251484","https://www.speedtest.net/result/c/52785a3d-debf-4b71-94d5-83fd3bf1f8be","1"
"Swish Fibre - London","34948","4.82","0.845","0","87321007","13887064","498739328","51501758","https://www.speedtest.net/result/c/29d66104-f8c6-4cb6-9d6c-0fc93a35314a","1"
"Zen Internet - London","40788","4.824","0.318","0","70233056","13893151","518513204","51564334","https://www.speedtest.net/result/c/e4593dad-2c59-491f-8e89-d205cf26428c","1"
"Swish Fibre - London","34948","4.991","1.185","0","92823423","13878705","958991040","54290626","https://www.speedtest.net/result/c/3c04baeb-3784-41cc-83c0-0550a1148025","1"
05/07/2022
21:30
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 7 ms 9 ms 9 ms lo0-0.bng1.ixn-lon.zen.net.uk [51.148.77.128]
Trace complete.
"Zen Internet - London","40788","5.516","0.169","0","73296512","13820884","936173398","52992426","https://www.speedtest.net/result/c/7480c670-3145-40ea-a19a-dc1c1473e201","1"
"Swish Fibre - London","34948","5.453","0.535","0","99788453","13874785","870836416","50122976","https://www.speedtest.net/result/c/18044271-1f8e-4806-b313-600d08bf76ef","1"
"Zen Internet - London","40788","5.43","0.307","0","70143622","13895628","510078744","54500544","https://www.speedtest.net/result/c/2686b534-4b72-45a8-8ca0-e3b9cef4ec68","1"
"Swish Fibre - London","34948","5.249","1.116","0","98733113","13880743","797845024","55636787","https://www.speedtest.net/result/c/bdfa05fb-6d40-43b4-89a1-53101246ac90","1"
"Zen Internet - London","40788","3.797","0.553","0","72888619","13909016","498613017","82137737","https://www.speedtest.net/result/c/57817d84-da4a-4448-b331-c1c35b21d634","1"
"Swish Fibre - London","34948","5.42","0.542","0","98974797","13888235","759295456","50164646","https://www.speedtest.net/result/c/462bf80a-94c1-49ae-ab7d-a4f241ac8a01","1"
05/07/2022
21:31
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 5 ms 5 ms 5 ms lo0-0.bng5.thn-lon.zen.net.uk [51.148.77.133]
Trace complete.
"Zen Internet - London","40788","4.809","0.443","0","69076548","13894953","814740000","50254284","https://www.speedtest.net/result/c/6a7120a9-366a-4f6f-a081-e713e16d552d","1"
"Swish Fibre - London","34948","5.755","0.113","0","108494450","13882408","447262304","50111270","https://www.speedtest.net/result/c/feb6bc01-61b7-4928-9394-b1cfe130cd36","1"
"Zen Internet - London","40788","4.25","0.503","0","66626470","13851164","567027823","52827441","https://www.speedtest.net/result/c/1e8ef096-2314-46e2-921c-c06c24c55c33","1"
"Swish Fibre - London","34948","5.805","0.077","0","105672102","13868833","579575168","50070552","https://www.speedtest.net/result/c/e6b272c2-cddf-4a93-bb1d-ae63be6c6643","1"
"Zen Internet - London","40788","4.333","0.52","0","50001257","13732845","626466693","80739147","https://www.speedtest.net/result/c/93c1fe43-be7e-4c90-bd2d-021da2cb86d2","1"
"Swish Fibre - London","34948","5.672","0.399","0","107569095","13870838","389977898","50123776","https://www.speedtest.net/result/c/48cd30f7-fb4f-43c4-9587-e6d745ec2cf3","1"
05/07/2022
21:33
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 11 ms 11 ms 10 ms lo0-0.bng3.wh-man.zen.net.uk [51.148.77.130]
Trace complete.
"Zen Internet - London","40788","15.599","0.285","0","54731590","13891571","561375264","92911434","https://www.speedtest.net/result/c/3ca243b7-a453-43d5-80da-3a14d8a85bae","1"
"Swish Fibre - London","34948","16.761","0.464","0","108375001","13895217","642339392","102418574","https://www.speedtest.net/result/c/554ea44b-df8d-40d2-ba27-0a756a68beee","1"
"Zen Internet - London","40788","15.956","0.908","0","32650857","13899257","282542294","94542917","https://www.speedtest.net/result/c/c6b829a8-8ba4-4197-b89c-a5617f83e417","1"
"Swish Fibre - London","34948","16.993","0.397","0","111666553","13895690","1098058420","102356099","https://www.speedtest.net/result/c/e0aad77b-2d24-4044-9f6d-354415f48232","1"
"Zen Internet - London","40788","15.961","0.265","0","50596218","13899846","677621440","97262408","https://www.speedtest.net/result/c/5c168f7c-7120-4b14-abfe-e54f6069ff0c","1"
"Swish Fibre - London","34948","16.931","0.215","0","115647039","13894365","1030767488","95780731","https://www.speedtest.net/result/c/d6f14f25-292a-4132-98b9-4a0583f4ab27","1"
05/07/2022
21:35
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 26 ms 9 ms 13 ms lo0-0.bng1.ixn-lon.zen.net.uk [51.148.77.128]
Trace complete.
"Zen Internet - London","40788","4.724","1.081","0","63920955","13888102","872380857","56875832","https://www.speedtest.net/result/c/efcc39ba-7eb4-4e2f-af32-5e02c935d8b1","1"
"Swish Fibre - London","34948","5.439","0.171","0","99013962","13864476","806737344","54100066","https://www.speedtest.net/result/c/7f260ae2-2aeb-4f26-ada0-6028cc9c59f9","1"
"Zen Internet - London","40788","3.661","1.49","0","53812288","14002436","491281977","53224033","https://www.speedtest.net/result/c/fd734c91-ca97-414e-ae8c-445b2bcbdd14","1"
"Swish Fibre - London","34948","5.6","0.454","0","98470021","13895677","771162496","55767351","https://www.speedtest.net/result/c/258b5cb1-17a0-4061-b503-67cc91a31ef7","1"
"Zen Internet - London","40788","3.7","2.998","0","51966402","13885148","472226511","55571975","https://www.speedtest.net/result/c/dea0b8ad-4545-4ebd-b4ce-8159312d72eb","1"
"Swish Fibre - London","34948","5.587","0.289","0","99053292","13887430","831103584","50154188","https://www.speedtest.net/result/c/7973b69a-d612-434e-9d7c-3e318b52effa","1"
05/07/2022
21:37
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 7 ms 12 ms 5 ms lo0-0.bng1.ixn-lon.zen.net.uk [51.148.77.128]
Trace complete.
"Zen Internet - London","40788","5.429","0.136","0","57918593","13983691","551134964","51625836","https://www.speedtest.net/result/c/e9267029-6324-437e-adbe-32dd68f5a523","1"
"Swish Fibre - London","34948","5.517","0.378","0","98936624","13787725","904702688","49846707","https://www.speedtest.net/result/c/c4f41cb6-7dc2-4e08-a636-d85e0d59908e","1"
"Zen Internet - London","40788","3.738","1.399","0","63206681","13894645","559164665","56992035","https://www.speedtest.net/result/c/daabe73b-f825-4ab5-8d26-0b909e162f0b","1"
"Swish Fibre - London","34948","5.376","0.263","0","98715178","13887327","755412512","51516040","https://www.speedtest.net/result/c/a0f04bbd-a2b5-4b23-a903-2855a2a58f32","1"
"Zen Internet - London","40788","3.708","1.462","0","69858651","13893512","985722616","52927613","https://www.speedtest.net/result/c/5262a0bd-79e5-4ea8-b4c7-81eb62cd9b89","1"
"Swish Fibre - London","34948","5.736","1.564","0","99335284","13882712","721576800","55670036","https://www.speedtest.net/result/c/975c7bb1-4439-470e-a008-ebf72d1b0212","1"
05/07/2022
21:39
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 17 ms 26 ms 5 ms lo0-0.bng5.thn-lon.zen.net.uk [51.148.77.133]
Trace complete.
"Zen Internet - London","40788","4.283","1.203","0","68676575","13896897","516143631","50286073","https://www.speedtest.net/result/c/648f8692-33e9-40ea-a19c-18b9eecaf139","1"
"Swish Fibre - London","34948","5.679","0.249","0","105845896","13858116","828174753","52596314","https://www.speedtest.net/result/c/2f013c73-1c5e-4b1a-8ce4-fe5b3c81997c","1"
"Zen Internet - London","40788","4.186","1.252","0","68555996","13891406","487690671","55813571","https://www.speedtest.net/result/c/f4070b96-fa81-4494-b072-00b9f33c4268","1"
"Swish Fibre - London","34948","5.633","0.229","0","107162561","13874109","752517024","54059224","https://www.speedtest.net/result/c/8afd01aa-eef3-4e2d-a38c-15e00edd62ca","1"
"Zen Internet - London","40788","4.25","1.084","0","68392925","13883426","455194543","54416641","https://www.speedtest.net/result/c/11b7ba35-7b5c-450d-a9a6-3f6abc90a68b","1"
"Swish Fibre - London","34948","5.834","0.094","0","106503894","13863322","727589143","49983933","https://www.speedtest.net/result/c/e6f6a6bb-a76c-468d-ba7f-3fef7b12c114","1"
05/07/2022
21:40
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 21 ms 8 ms 9 ms lo0-0.bng5.thn-lon.zen.net.uk [51.148.77.133]
Trace complete.
"Zen Internet - London","40788","4.646","0.766","0","71290848","13899984","785933554","51608947","https://www.speedtest.net/result/c/fe134e28-376a-4d60-bd5a-9ef0d9f4eb46","1"
"Swish Fibre - London","34948","5.761","0.843","0","108801199","13884050","866039282","50122521","https://www.speedtest.net/result/c/c255fc65-5e9d-4d51-96b6-394866301b65","1"
"Zen Internet - London","40788","4.217","1.263","0","70581883","13910243","681197871","53086667","https://www.speedtest.net/result/c/4d6bd2f8-be7d-4796-a7d2-53ad37fd0b1d","1"
"Swish Fibre - London","34948","5.609","0.27","0","107997526","13871060","670574474","52991427","https://www.speedtest.net/result/c/06cffc99-f490-4b4e-b7df-7b2c39587b52","1"
"Zen Internet - London","40788","4.421","0.852","0","69064960","13890237","657998031","52952788","https://www.speedtest.net/result/c/5529d2d5-ea8a-4efd-9d42-ae500061069c","1"
"Swish Fibre - London","34948","5.606","0.329","0","104942618","13879570","527348688","51640256","https://www.speedtest.net/result/c/2265c54c-be59-4601-9b1d-fb4298330c73","1"
05/07/2022
21:42
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 7 ms 8 ms 12 ms lo0-0.bng1.ixn-lon.zen.net.uk [51.148.77.128]
Trace complete.
"Zen Internet - London","40788","5.312","0.393","0","70321623","13897874","647453684","77877920","https://www.speedtest.net/result/c/7b910ffd-41b9-4368-816a-aab3a5cca122","1"
"Swish Fibre - London","34948","5.404","0.184","0","98547335","13882529","834543008","50130049","https://www.speedtest.net/result/c/6dcfcdb3-b360-41aa-b7d4-8744cb57978c","1"
"Zen Internet - London","40788","3.674","1.949","0","72695904","13894664","582265551","54513663","https://www.speedtest.net/result/c/63601f8d-1b31-4238-a437-315312c1f80d","1"
"Swish Fibre - London","34948","5.613","0.436","0","99200635","13878086","732540608","50128769","https://www.speedtest.net/result/c/282d2209-b3f0-4679-afd5-669e501b357e","1"
"Zen Internet - London","40788","3.716","0.805","0","72280817","13896396","581677679","54519445","https://www.speedtest.net/result/c/2ccdfabd-a162-45a3-a23d-7dfd33cc6530","1"
"Swish Fibre - London","34948","5.383","0.082","0","99981900","13897094","852696000","57331539","https://www.speedtest.net/result/c/5e362411-d178-4f13-aecc-4658e694c70a","1"
05/07/2022
21:43
Tracing route to dns.google [8.8.8.8]
over a maximum of 1 hops:
1 31 ms 12 ms 11 ms lo0-0.bng4.wh-man.zen.net.uk [51.148.77.131]
Trace complete.
"Zen Internet - London","40788","15.758","0.932","0","69166885","13887168","806530592","95933531","https://www.speedtest.net/result/c/c3ca52b4-2b14-465e-a443-d77f5fbcbd69","1"
"Swish Fibre - London","34948","17.187","0.575","0","113256879","13892656","1201986720","98421177","https://www.speedtest.net/result/c/ac3ef884-a955-439a-b3c6-77a4a9743ea4","1"
"Zen Internet - London","40788","15.831","0.087","0","40749442","13898229","531534144","99670918","https://www.speedtest.net/result/c/c67b3949-2372-43c2-9941-286858ac360f","1"
"Swish Fibre - London","34948","16.125","0.863","0","112307449","13892446","1368812785","92918779","https://www.speedtest.net/result/c/ec6d7ebd-05ca-48b6-bc28-b7c56d086b56","1"
"Zen Internet - London","40788","20.28","1.143","0","83495631","13882158","1075532000","90356131","https://www.speedtest.net/result/c/9f444837-b5d8-474b-a73c-696f2d954a05","1"
"Swish Fibre - London","34948","17.603","0.459","0","115553761","13891691","1062119376","94161931","https://www.speedtest.net/result/c/fa4b9207-3b06-4d73-a3eb-bd78859f6724","1"
This is also worth a read if you use cloudflare speedtest:
https://speed.cloudflare.com/about/
If you look at the detail you can see clear patterns emerge.
Latency is always around +10ms if I end up on a London gateway.I don't see this in your results, clicking on a sample of your results I see mainly 4-6ms.
Yes, I see a clear pattern that you will get different results depending on time, network condition, gateway selection, path to server, etc.It was perhaps worded clumsily. Manchester gateways are +10ms on top of the London gateway results.
I don't see this in your results, clicking on a sample of your results I see mainly 4-6ms.
Zen have 1Gb GEA cablelinks in some Headend exchanges but no 10Gb cablelinks so anything above 330Mb/s would need to be over BTW or TTB.Do you know this to be true? I'm on a 900 service from Zen and have apparently been migrated onto Zen's own backhaul from BTW, with subsequent performance issues :(
Zens issues aren't related to them only having 1Gb cablelinks.
If you are on 900Mb and they have migrated to to their own backhaul then they have 10Gb GEA cablelinks in place at your Head-End.
There's no chance their issues are down to trying to squeeze hundreds of Gb customers on to a Gb cablelink.
they are routinely continuing to migrate people on to this infrastructure seemingly ignorant.
We're not able to provide you with the information on the backhaul used as we're not privy to this information. We do have different suppliers we use, and how the service is provided to the property is managed and maintained by OpenReach which is a subsidiary of the BT Group.
Zens issues aren't related to them only having 1Gb cablelinks.Just to be clear, does that mean the info posted that I originally quoted from you must be incorrect or out of date? You posted with some authority, which lead me to believe that I might be on an exchange with one or several Gb links but not a 10Gb link. As I guess I could either believe that this might be what is supposed to happen (no migration unless 10Gb link) but actually what has happened is different.
If you are on 900Mb and they have migrated to to their own backhaul then they have 10Gb GEA cablelinks in place at your Head-End.
There's no chance their issues are down to trying to squeeze hundreds of Gb customers on to a Gb cablelink.Well, whatever it is about their own GEA setup, it seems a bit sub-par at least in my case in some respect. The before and after migration ping charts don't tell me a story of being migrated to a brilliant portion of their network. I'm intrigued to know where and how it is going wrong though, as I get faster speedtests to non-Zen servers than to Zen's own... All seems a bit barmy.
Touchwood so far I have not had any issues with zen after 2 years on fttc and now on fttp (900). It would be nice to know wether I am on zen's own gea or not but probably will never know unless an order goes through. I am following the various posts with interest.Your TBB doesn't seem to have the same furry characteristic as mine does, with the min/avg/max latency all being very distinctly visible as lines... And your speedtests seem good too. I don't think there is much point worrying about it unless one or both of those change. It's not like you can pre-emptively do much about it. The only thing I would say would be of benefit would be if possible to have some history of speed test results to fall back on if it changes.
I'm confused. What exactly is the specific 'performance' issue some users are experiencing? Is it ping times to a specific IP address? Is it speedtest results on a specific speedtest site? Is it issues with TBB ping results?My speed tests aren't what they should be, Zen's own server never hits line rate, whereas it has almost always, prior to my current situation.
Just to be clear, does that mean the info posted that I originally quoted from you must be incorrect or out of date? You posted with some authority, which lead me to believe that I might be on an exchange with one or several Gb links but not a 10Gb link.
Zen have 1Gb GEA cablelinks in some Headend exchanges but no 10Gb cablelinks so anything above 330Mb/s would need to be over BTW or TTB.
The info I posted is correct. They have exchanges where they only have 1Gb GEA cablelinks.OK, I see what you meant now, the order of your sentence I think left it ambiguous, hence my question. I read it that "Zen have no 10G cablelinks".
On those exchanges with 1Gb cablelinks any customers with over 330Mb/s need provisioned on BTW or TTB.
They do not put 1Gb customers on 1Gb links.
If Zen have moved your to their own GEA platform and you have gigabit then Zen have 10Gb GEA cablelinks in place.
That's what I said previously.
My speed tests aren't what they should be, Zen's own server never hits line rate, whereas it has almost always, prior to my current situation.OK, lets ignore the pings for the moment. Can you be more specific about the speed tests? My experience is they are at best a 'guesstimate' on the higher speed products, use 4 different speed test sites/servers you will get 4 different results. (see my post a few days ago in this thread)
The ping results changed markedly at the point of the GEA migration to Zen's service. These seem unlikely to be unrelated coincidences.
To be clear; I don't care about the variation in ping times, I merely think it gives some insight into what may be going on leading to poor throughput performance.
I'm also of an opinion that there is some marketing 'fluff' of the products, eg the Zen 900 residential package says '900Mbs Average Download' I think that is open to interpretation, my interpretation is that is a maximum speed of the product.
OK, lets ignore the pings for the moment. Can you be more specific about the speed tests? My experience is they are at best a 'guesstimate' on the higher speed products, use 4 different speed test sites/servers you will get 4 different results. (see my post a few days ago in this thread)The speed tests are done from the Speedtest.net CLI tool in the case reported above, to Zen's own and Swish Fibre's servers. These are the same test servers as you can access at www.speedtest.net, and they do perform as you would expect in the presence of an unencumbered internet connection and a home network that is so capable. I used the command line version as it was easy to script and run repeated tests over and over, but you get basically the same results from the web browser. @skyeci has already shown his connection now is performing how mine was prior to the Zen GEA migration event (ie achieving over 900mbps to Zen and Swish fibre servers). I am unable to get those kind of results since I was taken off BT's backhaul from our exchange to Zen, and put onto Zen's own backhaul from our exchange to Zen. The change is absolutely coincident with the very characteristic change in the ping graphs.
I'm also of an opinion that there is some marketing 'fluff' of the products, eg the Zen 900 residential package says '900Mbs Average Download' I think that is open to interpretation, my interpretation is that is a maximum speed of the product.
Besides that, given a PON is 2.4Gbit down, I'd expect it to take three heavy users before PON congestion would even kick in at all.Precisely.
Expecting a customer to just "put up with it" while they investigate is really unacceptable.I don't want to over-egg it, as clearly my connection does seem to be functioning better I dare say than many. I think from reading back in that TBB thread that the two main people there having issues they noted were much worse impacted than I am, which may mean they're not the same issue, or that this is just a less extreme version of what they see. Either way, it looked like their services were well below the Zen minimum in many conditions, whereas mine for much / most of the time is above it. Though as I said to the last person I spoke to on the phone; I don't think the minimum is a target above which all's good. I don't think it's unreasonable to expect you should hit line rate sometimes. Though I have some sympathy with the frontline staff as it sounds that unless I can show results consistently under the guarantee I get the feeling they're limited in what escalation they can do; it seems maybe if it was more broken I might have a greater chance of a timely resolution.
Any thoughts folk?
You You BQM says UDM, I assume Ubiquiti? This is running PPPoE on Linux, check what firewall settings you have, as it may not perform at full line speed using PPPoE, especially if you are running IPS etc.Thanks for the post & link. The connection is currently on a Ubiquiti UDM Pro SE that I just acquired; prior it was on a Ubiquiti USG4Pro with IDS and smart queues disabled (that definitely does need those disabled to hit line rates). The issues started with the previous router in place which was passing 900Mbps reliably prior to Zen's change - you seem to keep missing that point. If you read a bit further above you can see I've done plenty of tests with Zen's own router, which were different but similar.
https://community.ui.com/questions/ETA-on-bugfix-for-UDM-Pro-bad-PPPoE-performance/9119aa98-412f-41c7-9188-a30036c2e4c2
Maybe turn of some of the services on the UDM briefly and run speed test(s) again.
I think every single report is from people on Openreach FTTP products not CF FTTP?I think that is the case. Certainly for folk having had a Zen GEA Migration - I don't know if there have been any similar migrations from CF FTTP (I assume all the Zen CF FTTP start out as native Zen backhaul, but maybe not?).
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 8.77 MBytes 73.5 Mbits/sec 0 160 KBytes
[ 4] 1.00-2.00 sec 21.8 MBytes 183 Mbits/sec 3 259 KBytes
[ 4] 2.00-3.00 sec 23.2 MBytes 195 Mbits/sec 0 284 KBytes
[ 4] 3.00-4.00 sec 25.2 MBytes 212 Mbits/sec 0 311 KBytes
[ 4] 4.00-5.00 sec 29.7 MBytes 249 Mbits/sec 0 335 KBytes
[ 4] 5.00-6.00 sec 32.4 MBytes 272 Mbits/sec 0 355 KBytes
[ 4] 6.00-7.00 sec 33.6 MBytes 282 Mbits/sec 0 373 KBytes
[ 4] 7.00-8.00 sec 34.7 MBytes 291 Mbits/sec 0 392 KBytes
[ 4] 8.00-9.00 sec 43.5 MBytes 365 Mbits/sec 0 447 KBytes
[ 4] 9.00-10.00 sec 52.5 MBytes 440 Mbits/sec 0 556 KBytes
[ 4] 10.00-11.00 sec 57.9 MBytes 486 Mbits/sec 29 571 KBytes
[ 4] 11.00-12.00 sec 50.8 MBytes 426 Mbits/sec 0 583 KBytes
[ 4] 12.00-13.00 sec 66.1 MBytes 554 Mbits/sec 0 600 KBytes
[ 4] 13.00-14.00 sec 57.8 MBytes 485 Mbits/sec 0 614 KBytes
[ 4] 14.00-15.00 sec 59.4 MBytes 498 Mbits/sec 0 626 KBytes
[ 4] 15.00-16.00 sec 63.8 MBytes 536 Mbits/sec 0 666 KBytes
[ 4] 16.00-17.00 sec 69.4 MBytes 582 Mbits/sec 0 723 KBytes
[ 4] 17.00-18.00 sec 69.6 MBytes 584 Mbits/sec 93 573 KBytes
[ 4] 18.00-19.00 sec 58.9 MBytes 494 Mbits/sec 0 639 KBytes
[ 4] 19.00-20.00 sec 59.3 MBytes 497 Mbits/sec 0 718 KBytes
[ 4] 20.00-21.00 sec 78.2 MBytes 656 Mbits/sec 0 752 KBytes
[ 4] 21.00-22.00 sec 68.4 MBytes 574 Mbits/sec 0 782 KBytes
[ 4] 22.00-23.00 sec 68.2 MBytes 572 Mbits/sec 0 793 KBytes
[ 4] 23.00-24.00 sec 66.7 MBytes 559 Mbits/sec 0 802 KBytes
[ 4] 24.00-25.00 sec 74.6 MBytes 626 Mbits/sec 0 802 KBytes
[ 4] 25.00-26.00 sec 80.1 MBytes 672 Mbits/sec 0 802 KBytes
[ 4] 26.00-27.00 sec 82.8 MBytes 695 Mbits/sec 0 805 KBytes
[ 4] 27.00-28.00 sec 66.7 MBytes 560 Mbits/sec 0 812 KBytes
[ 4] 28.00-29.00 sec 71.8 MBytes 602 Mbits/sec 0 827 KBytes
[ 4] 29.00-30.00 sec 81.0 MBytes 680 Mbits/sec 0 858 KBytes
[ 4] 30.00-31.00 sec 82.3 MBytes 690 Mbits/sec 0 901 KBytes
[ 4] 31.00-32.00 sec 90.5 MBytes 759 Mbits/sec 0 963 KBytes
[ 4] 32.00-33.00 sec 88.6 MBytes 743 Mbits/sec 0 1.00 MBytes
[ 4] 33.00-34.00 sec 106 MBytes 891 Mbits/sec 0 1.00 MBytes
[ 4] 34.00-35.00 sec 81.8 MBytes 687 Mbits/sec 57 795 KBytes
[ 4] 35.00-36.00 sec 87.9 MBytes 737 Mbits/sec 0 881 KBytes
[ 4] 36.00-37.00 sec 81.3 MBytes 682 Mbits/sec 0 977 KBytes
[ 4] 37.00-38.00 sec 96.0 MBytes 805 Mbits/sec 0 1024 KBytes
[ 4] 38.00-39.00 sec 82.6 MBytes 693 Mbits/sec 0 1.01 MBytes
[ 4] 39.00-40.00 sec 80.7 MBytes 677 Mbits/sec 0 1.01 MBytes
[ 4] 40.00-41.00 sec 70.2 MBytes 589 Mbits/sec 5 758 KBytes
[ 4] 41.00-42.00 sec 60.6 MBytes 509 Mbits/sec 1 564 KBytes
[ 4] 42.00-43.00 sec 57.8 MBytes 485 Mbits/sec 0 608 KBytes
[ 4] 43.00-44.00 sec 57.2 MBytes 480 Mbits/sec 0 645 KBytes
[ 4] 44.00-45.00 sec 56.9 MBytes 477 Mbits/sec 0 663 KBytes
[ 4] 45.00-46.00 sec 54.0 MBytes 453 Mbits/sec 0 672 KBytes
[ 4] 46.00-47.00 sec 58.7 MBytes 493 Mbits/sec 0 675 KBytes
[ 4] 47.00-48.00 sec 60.8 MBytes 510 Mbits/sec 0 675 KBytes
[ 4] 48.00-49.00 sec 66.8 MBytes 560 Mbits/sec 0 675 KBytes
[ 4] 49.00-50.00 sec 73.2 MBytes 614 Mbits/sec 0 680 KBytes
[ 4] 50.00-51.00 sec 74.6 MBytes 625 Mbits/sec 0 693 KBytes
^C[ 4] 51.00-51.47 sec 29.1 MBytes 521 Mbits/sec 0 701 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-51.47 sec 3.18 GBytes 530 Mbits/sec 188 sender
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 4] 0.00-1.00 sec 93.7 MBytes 786 Mbits/sec 67877
[ 4] 1.00-2.00 sec 92.8 MBytes 778 Mbits/sec 67200
[ 4] 2.00-3.00 sec 109 MBytes 918 Mbits/sec 79286
[ 4] 3.00-4.00 sec 107 MBytes 895 Mbits/sec 77293
[ 4] 4.00-5.00 sec 109 MBytes 917 Mbits/sec 79182
[ 4] 5.00-6.00 sec 99.3 MBytes 833 Mbits/sec 71920
[ 4] 6.00-7.00 sec 99.1 MBytes 832 Mbits/sec 71788
[ 4] 7.00-8.00 sec 97.5 MBytes 818 Mbits/sec 70620
[ 4] 8.00-9.00 sec 109 MBytes 915 Mbits/sec 79003
[ 4] 9.00-10.00 sec 106 MBytes 892 Mbits/sec 77029
[ 4] 10.00-11.00 sec 92.1 MBytes 772 Mbits/sec 66660
[ 4] 11.00-12.00 sec 109 MBytes 911 Mbits/sec 78677
[ 4] 12.00-13.00 sec 109 MBytes 913 Mbits/sec 78850
[ 4] 13.00-14.00 sec 98.5 MBytes 827 Mbits/sec 71355
[ 4] 14.00-15.00 sec 94.8 MBytes 795 Mbits/sec 68670
[ 4] 15.00-16.00 sec 108 MBytes 904 Mbits/sec 78050
[ 4] 16.00-17.00 sec 109 MBytes 915 Mbits/sec 79020
[ 4] 17.00-18.00 sec 110 MBytes 920 Mbits/sec 79381
[ 4] 18.00-19.00 sec 108 MBytes 907 Mbits/sec 78289
[ 4] 19.00-20.00 sec 92.2 MBytes 774 Mbits/sec 66783
[ 4] 20.00-21.00 sec 96.7 MBytes 811 Mbits/sec 70011
[ 4] 21.00-22.00 sec 102 MBytes 853 Mbits/sec 73665
[ 4] 22.00-23.00 sec 97.4 MBytes 817 Mbits/sec 70557
[ 4] 23.00-24.00 sec 105 MBytes 878 Mbits/sec 75753
[ 4] 24.00-25.00 sec 99.9 MBytes 838 Mbits/sec 72345
[ 4] 25.00-26.00 sec 109 MBytes 916 Mbits/sec 79082
[ 4] 26.00-27.00 sec 109 MBytes 918 Mbits/sec 79211
[ 4] 27.00-28.00 sec 105 MBytes 885 Mbits/sec 76365
[ 4] 28.00-29.00 sec 103 MBytes 864 Mbits/sec 74621
[ 4] 29.00-30.00 sec 109 MBytes 911 Mbits/sec 78627
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-30.00 sec 3.02 GBytes 864 Mbits/sec 0.017 ms 2128/2237169 (0.095%) [ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 10.2 MBytes 85.7 Mbits/sec
[ 5] 1.00-2.00 sec 28.5 MBytes 239 Mbits/sec
[ 5] 2.00-3.00 sec 41.5 MBytes 348 Mbits/sec
[ 5] 3.00-4.00 sec 52.3 MBytes 438 Mbits/sec
[ 5] 4.00-5.00 sec 60.5 MBytes 508 Mbits/sec
[ 5] 5.00-6.00 sec 62.1 MBytes 521 Mbits/sec
[ 5] 6.00-7.00 sec 67.1 MBytes 563 Mbits/sec
[ 5] 7.00-8.00 sec 71.1 MBytes 597 Mbits/sec
[ 5] 8.00-9.00 sec 51.2 MBytes 429 Mbits/sec
[ 5] 9.00-10.00 sec 54.4 MBytes 457 Mbits/sec
[ 5] 10.00-11.00 sec 58.4 MBytes 490 Mbits/sec
[ 5] 11.00-12.00 sec 64.5 MBytes 541 Mbits/sec
[ 5] 12.00-13.00 sec 67.9 MBytes 570 Mbits/sec
[ 5] 13.00-14.00 sec 68.8 MBytes 577 Mbits/sec
[ 5] 14.00-15.00 sec 70.9 MBytes 595 Mbits/sec
[ 5] 15.00-16.00 sec 72.6 MBytes 609 Mbits/sec
[ 5] 16.00-17.00 sec 72.2 MBytes 605 Mbits/sec
[ 5] 17.00-18.00 sec 75.0 MBytes 629 Mbits/sec
[ 5] 18.00-19.00 sec 74.1 MBytes 621 Mbits/sec
[ 5] 19.00-20.00 sec 69.2 MBytes 580 Mbits/sec
[ 5] 20.00-21.00 sec 74.6 MBytes 626 Mbits/sec
[ 5] 21.00-22.00 sec 76.1 MBytes 638 Mbits/sec
[ 5] 22.00-23.00 sec 76.6 MBytes 642 Mbits/sec
[ 5] 23.00-24.00 sec 79.4 MBytes 666 Mbits/sec
[ 5] 24.00-25.00 sec 77.3 MBytes 648 Mbits/sec
[ 5] 25.00-26.00 sec 90.2 MBytes 756 Mbits/sec
[ 5] 26.00-27.00 sec 86.5 MBytes 726 Mbits/sec
[ 5] 27.00-28.00 sec 91.6 MBytes 769 Mbits/sec
[ 5] 28.00-29.00 sec 86.8 MBytes 729 Mbits/sec
[ 5] 29.00-30.00 sec 91.8 MBytes 770 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-30.00 sec 1.98 GBytes 566 Mbits/sec 1 sender
[ 5] 0.00-30.00 sec 1.98 GBytes 566 Mbits/sec receiver[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 92.1 MBytes 772 Mbits/sec 0.018 ms 314/66445 (0.47%)
[ 5] 1.00-2.00 sec 102 MBytes 858 Mbits/sec 0.023 ms 0/73474 (0%)
[ 5] 2.00-3.00 sec 93.9 MBytes 788 Mbits/sec 0.026 ms 0/67443 (0%)
[ 5] 3.00-4.00 sec 96.0 MBytes 805 Mbits/sec 0.029 ms 0/68942 (0%)
[ 5] 4.00-5.00 sec 89.9 MBytes 754 Mbits/sec 0.017 ms 0/64534 (0%)
[ 5] 5.00-6.00 sec 103 MBytes 861 Mbits/sec 0.017 ms 98/73818 (0.13%)
[ 5] 6.00-7.00 sec 102 MBytes 854 Mbits/sec 0.019 ms 0/73076 (0%)
[ 5] 7.00-8.00 sec 104 MBytes 870 Mbits/sec 0.017 ms 0/74447 (0%)
[ 5] 8.00-9.00 sec 103 MBytes 865 Mbits/sec 0.020 ms 0/74061 (0%)
[ 5] 9.00-10.00 sec 103 MBytes 868 Mbits/sec 0.019 ms 0/74278 (0%)
[ 5] 10.00-11.00 sec 100 MBytes 842 Mbits/sec 0.017 ms 0/72064 (0%)
[ 5] 11.00-12.00 sec 99.7 MBytes 836 Mbits/sec 0.018 ms 2384/73954 (3.2%)
[ 5] 12.00-13.00 sec 102 MBytes 852 Mbits/sec 0.019 ms 0/72966 (0%)
[ 5] 13.00-14.00 sec 99.7 MBytes 837 Mbits/sec 0.018 ms 0/71631 (0%)
[ 5] 14.00-15.00 sec 104 MBytes 872 Mbits/sec 0.018 ms 0/74636 (0%)
[ 5] 15.00-16.00 sec 103 MBytes 866 Mbits/sec 0.019 ms 0/74182 (0%)
[ 5] 16.00-17.00 sec 102 MBytes 854 Mbits/sec 0.018 ms 0/73126 (0%)
[ 5] 17.00-18.00 sec 105 MBytes 877 Mbits/sec 0.018 ms 0/75121 (0%)
[ 5] 18.00-19.00 sec 104 MBytes 870 Mbits/sec 0.019 ms 0/74527 (0%)
[ 5] 19.00-20.00 sec 101 MBytes 846 Mbits/sec 0.024 ms 0/72416 (0%)
[ 5] 20.00-21.00 sec 92.9 MBytes 779 Mbits/sec 0.030 ms 0/66695 (0%)
[ 5] 21.00-22.00 sec 101 MBytes 848 Mbits/sec 0.020 ms 0/72573 (0%)
[ 5] 22.00-23.00 sec 103 MBytes 864 Mbits/sec 0.018 ms 0/73978 (0%)
[ 5] 23.00-24.00 sec 101 MBytes 850 Mbits/sec 0.024 ms 0/72748 (0%)
[ 5] 24.00-25.00 sec 104 MBytes 870 Mbits/sec 0.017 ms 0/74512 (0%)
[ 5] 25.00-26.00 sec 103 MBytes 864 Mbits/sec 0.022 ms 0/73977 (0%)
[ 5] 26.00-27.00 sec 101 MBytes 843 Mbits/sec 0.019 ms 0/72205 (0%)
[ 5] 27.00-28.00 sec 102 MBytes 854 Mbits/sec 0.023 ms 0/73145 (0%)
[ 5] 28.00-29.00 sec 93.0 MBytes 780 Mbits/sec 0.018 ms 0/66816 (0%)
[ 5] 29.00-30.00 sec 104 MBytes 872 Mbits/sec 0.018 ms 0/74668 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-30.00 sec 2.95 GBytes 844 Mbits/sec 0.000 ms 0/2166458 (0%) sender
[SUM] 0.0-30.0 sec 3047 datagrams received out-of-order
[ 5] 0.00-30.00 sec 2.94 GBytes 842 Mbits/sec 0.018 ms 2796/2166458 (0.13%) receiver
The previous tests were from my main Linux box, the second two were using iperf3 on my VPS in server mode connecting using iperf3 on pfSense, so that I could remove NAT as any possible issue.What VPS service are you using?
I have PMed more details but bear in mind that VPS is IPv6 aware so you might need to tell iperf3 which protocol to use. I forgot to check if iperf3 supports IPv6 and test that separately.
but I can't shake the feeling that due to the coincidental timing one is exasperating the other.
Can I give you a little shake please? If so read on :)The sentiment isn't lost on me, and to be frank I've got better things to do with my time than wait for Openreach engineers to tell me what I already know... However the 739 was just an example making a point to Alex on a particular speed test server in London (not the Zen one). Others may vary depending on the gateway, with the Zen one coming in below their guarantee quite a lot of the time.
I checked the spec of my car, it says it can do a maximum speed of 'X', but I don't take it to a race track every day to verify the maximum speed, when I just need to pop to the shops.
I think you maybe chasing your tail. Forget the marketing fluff on these products. My expectation of a '900' residential type product is it should perform at 450-900 most of the time. If you are expecting an 'average of 900', I think your expectations maybe too high.
I know it can be frustrating sometimes when there is an expectation of something, and that expectation is not met, but its sometimes worth revaluating the original expectation, putting the potential 'issue' into context. If there is an issue with Zen network infrastructure, I am sure their clever network engineers will fix at some point.
Looking at your speed test results above, 739-925 down, consistent 111 up, if I was getting that, I would be happy :) You found a gateway that works for you, keep the PPPoE up on that gateway and enjoy. :)
My expectation of a '900' residential type product is it should perform at 450-900 most of the time. If you are expecting an 'average of 900', I think your expectations maybe too high.
It's a known issue that Zen have acknowledged and are investigating.
Speedtest by Ookla
Server: Zen Internet - London (id = 40788)
ISP: Amazon.com
Latency: 1.70 ms (0.06 ms jitter)
Download: 9373.96 Mbps (data used: 4.7 GB )
Upload: 4780.34 Mbps (data used: 2.2 GB )
Packet Loss: 0.0%
Result URL: https://www.speedtest.net/result/c/a70cd9eb-b2e9-4dc8-a35f-29d85520dcc2
Speedtest by Ookla
Server: Swish Fibre - London (id = 34948)
ISP: Amazon.com
Latency: 2.67 ms (0.03 ms jitter)
Download: 8420.16 Mbps (data used: 7.6 GB )
Upload: 4776.85 Mbps (data used: 2.3 GB )
Packet Loss: 0.0%
Result URL: https://www.speedtest.net/result/c/1ae21155-ce84-4b65-bc16-db3a4796fca5
I'm sorry but I must also disagree craigski. Someone performing a speed test at 3am should honestly be able to expect to see close to or above 900 megabits. This is a problem that rightfully needs reporting to the ISP and needs to be pushed on until there's a solution. I can understand potentially seeing slower speed during peak time but not in the early hours of the morning when a lot of people are likely asleep. Unfortunately this isn't the first time in the last couple of years that Zen has had this kind of issue reported by some customers. It was pretty much the reason why I moved away from Zen quite a few years ago.I think furthermore, given that the line from me to the exchange seems quite capable of the 900, if there are backend issues that make attaining the higher throughput levels problematic, I almost might as well just have a 500 connection - which I'd imagine would seem to be able to be saturated most of the time (single threaded issues not withstanding).
The GEA problem sure, but if its there is a single-threaded performance problem on other backhauls, that changes things a bit.I've not really looked into the single threaded performance much, but I should I guess. Unfortunately the command line Ookla speed tester doesn't seem to have a single threaded option, so I'll have to see about trying something perhaps with iperf on that Amazon server. It only cost me about 25p to play around with a couple of EC2 25G capable servers for a couple of hours yesterday, so I might give that a go this evening again.
It is quite literally sold as having an average of 900Mb/s.
Zen have Samknows boxes in the homes of random, geographically diverse users and they must meet an average of 900Mb/s to be able to advertise the service as it is.
Unfortunately the command line Ookla speed tester doesn't seem to have a single threaded option, so I'll have to see about trying something perhaps with iperf on that Amazon server. It only cost me about 25p to play around with a couple of EC2 25G capable servers for a couple of hours yesterday, so I might give that a go this evening again.
I did my own experiment. I went to speedtest.net in Chrome browser, I selected 'Zen Internet London', I opened developer tools to try to find out what speedtest.net is doing under the covers, as I'm curious.
It appears to be downloading lots of random 25MB files using port 8080 and timing those downloads. What is interesting is they are not all from Zen IP address space. It looks like its downloading from 4 servers:
51.148.82.21 (this is 5 hops away, Zens server)
5.61.120.37
45.92.46.21 (goes via twelve99.net, telia.net, 12 hops)
217.138.226.208 (this looks like m247 that is 15 hops away on traceroute)
The dev tools in Chrome will show you the times for each of the 25MB downloads, vastly different depending on what server its downloading from.
London
Lightning Fibre Ltd
Additional servers used for download test
Eastbourne
CloudConnX
Eastbourne
M-Tech Systems
Maidstone
Trooli
Hosted by Kubbur (London) [4.20 km]: 29.846 ms
Testing download speed................................................................................
Download: 136.04 Mbit/sHosted by Zen Internet (London) [5.75 km]: 33.774 ms
Testing download speed................................................................................
Download: 203.08 Mbit/sHosted by Telxius (London) [5.75 km]: 78.502 ms
Testing download speed................................................................................
Download: 146.17 Mbit/sI have speedtest-cli on my router and tried a few different servers it shows as the nearest. Some of those were really bad (double digit speeds) so certainly if its using some of those the result is going to be WAY off.
This is my point, the speedtest.net is at best an estimate of your speed. Its random what servers its using, even if you select what you think is a good one, any of the other 3 may not be so good at the specific time you are doing the test.The web based speed tester does spread to multiple servers sometimes. I don't know what logic it employs to do that - I think it might be based on it detecting an issue between you and the server selected, but it usually tells you about it (or at least it used to). It would say something like "zen and 3 others"
fast.com, this is similar uses 4 server on the netflix network, but uses 26.2MB files, and port 443/HTTPS
BTw test uses a different approach, a single IPv4 address on akamai, looks like it runs 4 threads to same server downloading 1GB file, for 5 seconds, and calculates how much its downloaded of the 1GB in 5 seconds, again port 443/https
I go back to my original point I just don't trust the numbers on speedtest.net site.
Worth noting, the Zen server immediately after this became unavailable which perhaps means its overloaded so speedtest.net dynamically removed it from the list?The Zen and Swish fibre servers both work from speedtest command line application from speedtest.net, but speedtest-cli doesn't work with either.
I will confirm however that multi-threaded speeds are absolutely fine especially real-world.
@bogoff has already confirmed that there is more than 1G connection from AWS to Zen speedtest server. I want to see the results from AWS to @bogoff, I happy to pay the 25p :)I don't have any useful way to do an apples-to-apples comparison as I can't set up an Ookla server on AWS without it costing me an arm and a leg, and Zen don't have any public iperf3 server.
EDIT- Zen came back to me again confirming I am on BTW..
I was interested to find out if I was a gea or BTW for my recent new service.. zen report I am neither as I am City fibre apparently which I was not aware of... have they got this right as my upgrade was direct with zen...somewhat confused.
Latency Jitter Download Upload
4.90 ms 0.90 ms 946 Mbps 939 Mbps
Hopefully it will result in a similar slow speed that the Samknows speed test website does for you so Zen can't potentially try to say something like "well the Samknows device say it's working as expected so that's that".Thanks for running the test, always useful to have other data points. I am hoping for a result to be as you say as it does give the clearest path forward.
For what it is worth, the web-based Samknows speed test seems to consistently test well below line rate here (https://speedtest.samknows.com/)Another day, another speedtest. I looked, this one starts 8 downloads from your browser, to same server IP 80.71.21.229, measures the total amount of data downloaded over 12 seconds, and ESTIMATES the speed. I know I keep banging on about this, but take any speedtest website with a pinch of salt, its just an estimate guide of your speed. *IF* they could accurately measure the speed they would all give the same result.
Just wanted to confirm that despite single-threaded speedtest.net results off Zen server still maxing out at 300Mbit I was able to download in Firefox at 100MB/s earlier which would rather confirm everything is fine under real-world scenarios.
Another day, another speedtest. I looked, this one starts 8 downloads from your browser, to same server IP 80.71.21.229, measures the total amount of data downloaded over 12 seconds, and ESTIMATES the speed. I know I keep banging on about this, but take any speedtest website with a pinch of salt, its just an estimate guide of your speed. *IF* they could accurately measure the speed they would all give the same result.I was only interested to see what the Samknows web based test looked like as potential indication of how the servers they use for the Samknows box might behave on my connection. Most of my testing is done using native binaries under Linux and not browser based (you seem to be the one referring back to browser stuff all the time).
A far more accurate way to confirm what you speed is is to look at the interface on your firewall router when you are loading the network, using multiple devices on your LAN. I honestly believe you are chasing your tail trying to get the results you want from a speed test website with a fast FTTP connection.
I would also suspect that speedtest traffic from websites is given a low priority on the various networks. It makes sense ISPs would not want speedtest traffic run by a very few people to impact the many majority that are streaming music/video and making VoIP calls.Traffic management of speed test traffic is an interesting one; not least because the Zen server is one of the worst behaving on Zen's own network; according to traceroute the traffic never leaves the Zen network (or they all at least have Zen IP range 51.148.0.0/16 and Zen reverse DNS), and Zen make a point of saying they do not do any traffic shaping on their network. So I'm sorry but I can't square that as being a cause. And remember, I know their server is capable as it tests at 10G from a 3rd party network, and I know their server previously tested at line rate any time of the day pretty much for me. It's not possible to look at that result with a logical mind and come to any other conclusion than in recent time something is getting in the way of throughput to that Zen server; and when that is a new situation it raises a valid question of what that is.
Traffic management
Many ISPs advertise a certain feature, such as a particular speed or download limit, but tend to provide varying degrees of traffic management.
For example, an ISP might slow down your connection speed at certain times of the day, or when you’ve downloaded a certain amount of data in a month.
As well as throttling speeds, an ISP might put restrictions on certain types of traffic passing through their network.
At Zen, you won’t be victim to this type of management – we believe in providing you with what you’ve paid for. This means that, if you’re downloading lots of games, or you like to play even during the busiest periods of the day, you shouldn’t experience any degradation in performance.
I honestly believe you are chasing your tail trying to get the results you want from a speed test website with a fast FTTP connection.
I would also suspect that speedtest traffic from websites is given a low priority on the various networks. It makes sense ISPs would not want speedtest traffic run by a very few people to impact the many majority that are streaming music/video and making VoIP calls.
I think @Alex as sort of confirmed this:
I have personally found some gateways on Zen perform better than others, find one that works for you, and stick with it. If PPPoE drops, redial until you get back on the one that performs best for you.
This is true, but on the flip side I get full speed on SamKnows. So if I can get full speed and someone else on the same ISP even during off peak hours does not, I'd call that an issue. The particular background behind the change being all the more concerning.I did drag the gigabit LAN laptop downstairs yesterday and briefly check - samknows webchecker appears to be middle of the ground via the Zen supplied router, testing at between 700-750Mbps (while at the same time Zen's test server comes in around 600mbps and Voicehost still close to 900Mbps, when pre-migration nearly everywhere was >900Mbps). So I'll be interested to see what the Samknows dedicated testbox clocks in at. If the range holds when moving to the dedicated box then that range would put this connection below the average of the worst 20% performing 900mbps FTTP lines Zen had in their test pool it would seem. Though I note from the chart in that article that they have rounded all their numbers to the nearest 25Mbps - as 950Mbps for FTTP is a little bit of a stretch... ;)
This is a case of Zen backhaul performing worse than BTW, all parties want that fixed as it would cost Zen a fortune to revert everyone back, plus we have no idea if the problem might get worse as more users come online at the exchange.
Zen told me I am on a btw circuit and following a random reboot of my opnsense box a few days ago I notice my latency shot up to almost double what it would normally be.. It now seems when ipv6 is enabled and I end up on lo0-0.bng4.wh-man.zen.net.uk I get around 14ms yet when I disable ipv6 and I seem to go via vt1.cor1.lond2.ptn.zen.net.uk using just ipv4 my latency goes back to 6/7ms..I think this is fairly standard for Zen. They have gateways in London and Manchester (3x lon and 2x man I believe that I've been able to count recently). Additionally I think two of the London ones are at one physical location, and one is at a different one. I didn't think there was an IPV6 trigger as to which you end up on - but I don't have IPV6 enabled. Most of the time I tend to end up on a London gateway, but I will occasionally get put onto a Manchester one, with a resulting worsening of latency. I think the Manchester gateways can be of benefit if you are closer to Manchester than London, but I'm not sure as that would depend on exactly how and where traffic flows at various points, and that's not really discussed openly anywhere.
Dont wish to hi-jack the thread but this just seems odd and has never been an issue before.
any thoughts?, why is the man gateway giving me a much greater latency?
Ok thanks. Seems that's a new thing to me as historically I have always connected to London which is closer than Manchester for me. Have messed about with it and 99% of the time I'm London with v4. Still can't get back on London with v6 as it keeps reverting to man which is annoying... :-\I believe Zen have said elsewhere that whichever gateway answers first gets it; and that usually geography takes over so you will end up on the one closest; perhaps there is an issue with the London IPV6 connectivity at the moment, or something else.
Tracing route to google.co.uk [2a00:1450:4009:823::2003]
over a maximum of 30 hops:
1 9 ms 3 ms 3 ms fritz.box [2a02:8010:xxxx:0:7642:7fff:fe20:1e6c]
2 18 ms 23 ms 59 ms lo0-0.bng2.ixn-lon.zen.net.uk [2a02:8010::154]
3 13 ms 12 ms 13 ms 2a02:8010::14a
4 13 ms 13 ms 12 ms ae-2.p1.thn-lon.zen.net.uk [2a02:8010:0:b00::3a]
5 13 ms 12 ms 14 ms 2001:4860:0:135d::1
6 12 ms 13 ms 13 ms 2001:4860:0:1::1a51
7 13 ms 13 ms 12 ms lhr48s30-in-x03.1e100.net [2a00:1450:4009:823::2003]
hmmIndeed, and you'd have to hope that the Samknows box tests will represent their very best effort speed test results, as those results are what are used for the purposes of generating the advertised averages for the VCoP, so in a way it will be very interesting to see the Samknows box performance chart over time and across gateways.
If Zen were marking speedtests as low priority, the obvious question would be why are they doing it? My second issue with it is if anything an ISP would prioritise speedtests which is what they tended to do in the era when shaping was common in the UK.
I do agree of course though that just using a single speedtest as a means of diagnosing a connection isnt adequate, however the guys on the TBB thread including bogof in here have been doing their testing in sound ways and have repeatable behaviour when switching gateways, and switching between BTW and Zen backhaul of large performance differences, that kind of rules out typical internet backhaul or end user equipment causes.
If Zen were marking speedtests as low priority, the obvious question would be why are they doing it?
I'm only guessing here, but it makes sense to prioritise time sensitive traffic, such as video and VoIP. I would certainly hope that Netflix/BBC/Apple/Spotify/etc traffic had a higher priority on the Internet, than some potential DoS speedtest service.
While I agree in principle, in practice there are a lot of problems so the likes of AAISP and Zen DO NOT apply this as their networks are strictly complying to net neutrality.
Works well with A&A broadband
Our own broadband services have constant quality monitoring and will give priority to VoIP traffic by default, ensuring good quality calls.
Net Neutrality is something totally different. I'm talking about prioritisation of specific time sensitive traffic, streaming, video calls, VoIP etc
Net Neutrality is something totally different. I'm talking about prioritisation of specific time sensitive traffic, streaming, video calls, VoIP etc, eg:You'd expect though that (taking that example):
https://www.aa.net.uk/voice-and-mobile/voip-information/
If the above is true, then A&A will drop 'speedtest' packets in favour of VoIP if there is congestion . I would assume other ISPs do similar.
any thoughts?, why is the man gateway giving me a much greater latency?
I'm only guessing here, but it makes sense to prioritise time sensitive traffic, such as video and VoIP. I would certainly hope that Netflix/BBC/Apple/Spotify/etc traffic had a higher priority on the Internet, than some potential DoS speedtest service.
I read the Samknows blog after J0hn posted, and their methodology is better. They are in control of when the speedtests run, and what servers, rather than ad-hoc tests on random servers that may or may not be busy.
The pings are also probably irrelevant. Responding to a ping should always be a lower priority than processing traffic.
My own experience, if I create network traffic from multiple devices (real world traffic, like downloading ISOs from say Microsoft) , I can nearly always see a higher utilisation on the WAN interface on my firewall vs trying to run a speedtest.
Has anyone tried my 2 minute experiment I posted on this thread a few pages back, ie try to prove your connection is faster that the speedtest result.
However, we do recommend using IP Precedence to ensure your traffic is prioritized in general.
DAC (direct attach copper) connections are lower latency than Fibre, and both of those are an order of magnitude faster than 10GbaseT
Small update; Zen are going to be sending out a Samknows speed testing device to install on the network.We digressed slightly with prioritisation etc past few days. Has boxed arrived?
We digressed slightly with prioritisation etc past few days. Has boxed arrived?Sam may, but I know little...
Does Sam Know anything yet?
Sam is getting to Know a bit more then :)Zen have not asked me to do any such testing, I've provided them with plenty of data showing differences between gateways.
Did Zen ask you to do anything other than connect up the SamKnows box? Still using Fritz or gone back UDM? Do you know if Zen/SamKnows have tested against the different gateways you can connect to when you dial PPPoE?
I have found there is one Zen gateway that gives me consistent performance. If my PPPoE connection drops (this seems to happen in the early hours, and more frequently, I used to maintain connection for 100's of hours, now its 10's of hours), and I get connected to another, a 'speedtest' estimate can vary by upto 30%. My simple work around is to redial the PPPoE a few times until I get connected to my 'favourite' best performing gateway.
I'll be interested to hear the final results of effort this in a few weeks.
And on BTW backhaul I'm not really seeing ANY major difference from gateway hopping. In fact, the only thing to suggest my gateway changes at all is there can be a 1-2ms difference in latency.That was my experience too, no throughput difference, occasional small step in latency from gateway change.
I noted from a thread on TBB that I think the GEA migrated and unmigrated lines may connect to different sets of gateways.
I have been GEA migrated, but I can consistently max out connection (looking at WAN port on firewall) if I connect to the fastest (for me) gateway.Is that on FTTP900, and is that even to Zen's speedtest.net server?
Yep, I'm thinking same:
51.148.77.xxx is Zen backhaul
51.148.72.xxx is BTw backhaul
Maybe @Alex can confirm this, as he is still on BTw
I am still on btw with zen 900. My gateway currently gives 51.148.72.22 vt1.cor2.lond1.ptn.zen.net.uk
Is that on FTTP900, and is that even to Zen's speedtest.net server?Yes, 900. No, not 'speedtest' estimates. You know my thoughts on those, you get your chosen server + 3 random servers, results are unreliable. I look at WAN port utilisation when downloading files, and/or use fast.com. The results on fast.com seem to align with WAN port utilisation.
Which of the gateways is fastest for you?
Okay, nobody else online here right now so let's see if I can get some different gateways:
vt1.cor1.lond1.ptn.zen.net.uk (51.148.72.21) 7.827 ms 7.711 ms 7.968 ms
vt1.cor1.lond2.ptn.zen.net.uk (51.148.72.23) 6.990 ms 7.215 ms 7.339 ms
vt1.cor2.lond2.ptn.zen.net.uk (51.148.72.24) 8.027 ms 8.263 ms 8.226 ms
lo0-0.bng3.wh-man.zen.net.uk (51.148.77.130) 12.737 ms 12.720 ms 12.714 ms
lo0-0.bng4.wh-man.zen.net.uk (51.148.77.131) 3.260 ms 3.249 ms 3.242 ms
Yes, 900. No, not 'speedtest' estimates. You know my thoughts on those, you get your chosen server + 3 random servers, results are unreliable. I look at WAN port utilisation when downloading files, and/or use fast.com. The results on fast.com seem to align with WAN port utilisation.If you use the command line speedtest application it doesn't behave as you mention - it only uses the server you request. You should try it and we can stop having this conversation... :) The speedtest website also tends to only do that 3 servers thing if you let it do what it wants when you open the site. If you manually select a server it tends to only use that one. But I digress, just use the command line tool. You can download it https://www.speedtest.net/apps/cli (https://www.speedtest.net/apps/cli) for windows and many other OSes (I even have the arm Linux binaries running on my router).
lo0-0.bng2.ixn-lon.zen.net.uk [51.148.77.129] is fastest for me, but don't tell everyone
You should try it and we can stop having this conversation... :)The plot thickens....
Is my connection sub par, or are the 'speedtests' sub par? Thoughts?If you try some of the other servers I'm sure you'll find there's nothing "wrong" per se with the speedtest CLI tool itself, and that you can hit close to line rate with other servers...
The GEA problem sure, but if its there is a single-threaded performance problem on other backhauls, that changes things a bit.I do wonder still about whether some of the differences seen on various speed testers and servers ends up significantly related to single thread performance.
rm 512MB.zip ; time axel -U Chrome -n 40 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com/512MB.zip
rm bigfile ; time axel -n 1 -v -a -o bigfile https://releases.ubuntu.com/22.04.1/ubuntu-22.04.1-desktop-amd64.iso?_ga=2.63790191.1176441848.1661034061-697000926.1661034061
wget http://ipv4.download.thinkbroadband.com/512MB.zip
--2022-08-20 23:53:03-- http://ipv4.download.thinkbroadband.com/512MB.zip
Resolving ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)... 80.249.99.148
Connecting to ipv4.download.thinkbroadband.com (ipv4.download.thinkbroadband.com)|80.249.99.148|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 536870912 (512M) [application/zip]
Saving to: ‘512MB.zip.1’
512MB.zip.1 100%[================================================================================================================>] 512.00M 109MB/s in 4.8s
2022-08-20 23:53:08 (107 MB/s) - ‘512MB.zip.1’ saved [536870912/536870912]rm 512MB.zip ; time axel -U Chrome -n 1 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com/512MB.zip
Initializing download: http://ipv4.download.thinkbroadband.com/512MB.zip
File size: 512 Megabyte(s) (536870912 bytes)
Opening output file 512MB.zip
Starting download
[100%] [...] [ 104.7MB/s] [00:00]
Downloaded 512 Megabyte(s) in 4 second(s). (107259.86 KB/s)
real 0m4.904s
user 0m0.127s
sys 0m0.533srm 512MB.zip ; time axel -U Chrome -n 4 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com/512MB.zip
rm: cannot remove '512MB.zip': No such file or directory
Initializing download: http://ipv4.download.thinkbroadband.com/512MB.zip
File size: 512 Megabyte(s) (536870912 bytes)
Opening output file 512MB.zip
Starting download
[100%] [...] [ 105.1MB/s] [00:00]
Downloaded 512 Megabyte(s) in 4 second(s). (107589.71 KB/s)
real 0m5.195s
user 0m0.093s
sys 0m0.516straceroute to ipv4.download.thinkbroadband.com (80.249.99.148), 30 hops max, 60 byte packets
1 vt1.cor1.lond2.ptn.zen.net.uk (51.148.72.23) 7.118 ms 7.210 ms 6.982 ms
2 lag-8.p2.ixn-lon.zen.net.uk (51.148.73.206) 7.063 ms 7.057 ms 7.051 ms
3 lag-2.p2.thn-lon.zen.net.uk (51.148.73.138) 7.099 ms 7.170 ms lag-2.p1.thn-lon.zen.net.uk (51.148.73.132) 7.101 ms
4 lag-1.br1.thn-lon.zen.net.uk (51.148.73.153) 7.058 ms 7.052 ms 7.046 ms
5 netconnex-gw.zen.net.uk (82.71.254.2) 6.740 ms 6.734 ms 6.847 ms
6 ae11-11.edge-rt5.thdo.ncuk.net (80.249.97.21) 6.536 ms 6.681 ms 6.497 msMy experience seems to be very different, right now at least:Very interesting that our single thread performance is so different there to the same site... I'm only getting 34MB/s now for a single thread.Code: [Select]rm 512MB.zip ; time axel -U Chrome -n 1 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com/512MB.zip
Initializing download: http://ipv4.download.thinkbroadband.com/512MB.zip
File size: 512 Megabyte(s) (536870912 bytes)
Opening output file 512MB.zip
Starting download
[100%] [...] [ 98.9MB/s] [00:00]
Downloaded 512 Megabyte(s) in 5 second(s). (101227.50 KB/s)
real 0m4.950s
user 0m0.147s
sys 0m0.657sCode: [Select]rm 512MB.zip ; time axel -U Chrome -n 4 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com/512MB.zip
rm: cannot remove '512MB.zip': No such file or directory
Initializing download: http://ipv4.download.thinkbroadband.com/512MB.zip
File size: 512 Megabyte(s) (536870912 bytes)
Opening output file 512MB.zip
Starting download
[100%] [...] [ 105.1MB/s] [00:00]
Downloaded 512 Megabyte(s) in 4 second(s). (107589.71 KB/s)
real 0m5.195s
user 0m0.093s
sys 0m0.516sCode: [Select]traceroute to ipv4.download.thinkbroadband.com (80.249.99.148), 30 hops max, 60 byte packets
1 vt1.cor1.lond2.ptn.zen.net.uk (51.148.72.23) 7.118 ms 7.210 ms 6.982 ms
2 lag-8.p2.ixn-lon.zen.net.uk (51.148.73.206) 7.063 ms 7.057 ms 7.051 ms
3 lag-2.p2.thn-lon.zen.net.uk (51.148.73.138) 7.099 ms 7.170 ms lag-2.p1.thn-lon.zen.net.uk (51.148.73.132) 7.101 ms
4 lag-1.br1.thn-lon.zen.net.uk (51.148.73.153) 7.058 ms 7.052 ms 7.046 ms
5 netconnex-gw.zen.net.uk (82.71.254.2) 6.740 ms 6.734 ms 6.847 ms
6 ae11-11.edge-rt5.thdo.ncuk.net (80.249.97.21) 6.536 ms 6.681 ms 6.497 ms
(https://www.thinkbroadband.com/_assets/speedtest/button/1661035582796290555.png) (https://www.thinkbroadband.com/speedtest/results.html?test=1661035582796290555)
traceroute to ipv4.download.thinkbroadband.com (80.249.99.148), 30 hops max, 60 byte packets
1 192.168.178.1 (192.168.178.1) 0.378 ms 0.484 ms 0.492 ms
2 lo0-0.bng4.thn-lon.zen.net.uk (51.148.77.132) 6.691 ms 11.386 ms 11.384 ms
3 lag-14.p1.thn-lon.zen.net.uk (51.148.73.94) 8.993 ms lag-14.p2.thn-lon.zen.net.uk (51.148.73.96) 8.760 ms 8.753 ms
4 lag-2.br1.thn-lon.zen.net.uk (51.148.73.167) 8.684 ms lag-1.br1.thn-lon.zen.net.uk (51.148.73.153) 6.694 ms lag-2.br1.thn-lon.zen.net.uk (51.148.73.167) 6.464 ms
5 netconnex-gw.zen.net.uk (82.71.254.2) 6.499 ms 6.493 ms 6.486 ms
6 ae11-11.edge-rt5.thdo.ncuk.net (80.249.97.21) 6.650 ms 6.249 ms 6.156 ms
root@Home-Dream-Machine-SE:/ssd1/test# rm bigfile ; time axel -n 1 -v -a -o bigfile http://slackware.uk/slackware/slackware-iso/slackware64-15.0-iso/slackware64-15.0-install-dvd.iso
Initializing download: http://slackware.uk/slackware/slackware-iso/slackware64-15.0-iso/slackware64-15.0-install-dvd.iso
File size: 3780542464 bytes
State file found, but no downloaded data. Starting from scratch.
Opening output file bigfile
Starting download
[100%] [.............................................................................................] [ 100.8MB/s] [00:00]
Downloaded 3.5 Gigabyte in 35 seconds. (103227.12 KB/s)
real 0m35.782s
user 0m5.759s
sys 0m24.417s
1 192.168.178.1 (192.168.178.1) 0.388 ms 0.397 ms 0.465 ms
2 lo0-0.bng4.thn-lon.zen.net.uk (51.148.77.132) 20.160 ms 20.180 ms 20.173 ms
3 lag-14.p1.thn-lon.zen.net.uk (51.148.73.94) 7.286 ms lag-14.p2.thn-lon.zen.net.uk (51.148.73.96) 9.621 ms lag-14.p1hn-lon.zen.net.uk (51.148.73.94) 7.303 ms
4 lag-2.br1.thn-lon.zen.net.uk (51.148.73.167) 7.094 ms 7.021 ms 7.069 ms
5 linx-226.as13213.net (195.66.236.19) 7.172 ms 7.164 ms 9.638 ms
6 no-ptr.midphase.com (98.158.181.92) 9.479 ms 9.107 ms 9.053 ms
7 * 212.78.92.1 (212.78.92.1) 5.041 ms 5.019 ms
8 * * *
9 * * *
10 fry.opensourcerers.net (212.78.94.73) 5.067 ms 5.013 ms 5.037 ms
11 * * *
12 * * *
13 * * *
Our egress from Zen's network is the same place, which would imply the bottlenecks are within Zen's network...
Wish I could say I was surprised, but even though my connection is fine my opinion of Zen has taken a nose dive seeing how they do not care that people randomly end up via Manchester and in some cases doubling latency, plus this GEA issue. A well managed network should not have huge variances like this.Indeed.
traceroute to speedtest02a.web.zen.net.uk (51.148.82.21), 30 hops max, 60 byte packets
1 192.168.178.1 (192.168.178.1) 0.374 ms 0.475 ms 0.517 ms
2 lo0-0.bng4.thn-lon.zen.net.uk (51.148.77.132) 26.819 ms 26.850 ms 26.845 ms
3 lag-14.p1.thn-lon.zen.net.uk (51.148.73.94) 6.371 ms 6.328 ms 6.340 ms
4 lag-2.p1.ixn-lon.zen.net.uk (51.148.73.133) 6.192 ms lag-2.p2.ixn-lon.zen.net.uk (51.148.73.139) 8.857 ms 8.857 ms
5 lag-2.br1.ixn-lon.zen.net.uk (51.148.73.195) 8.634 ms 8.672 ms 8.628 ms
6 * * *
Have you tried picking up one of the lo0-0.bng gateways like I'm on (I guess ideally 4) and seeing how those single thread web download tests perform? That would be quite interesting to see if your single thread falls off the cliff then...
I tried reconnecting about 10 times or so and only got the gateways I mentioned earlier. Can't really sit reconnecting PPP all day, unless I move the FTTP to a PC and leave mum on the Three 5G - as every reconnect of PPP restarts the firewall which takes a minute or so to adjust. I only did the earlier tests as she was asleep.
Yes, thanks both for additional testing. You have re-confirmed to me that 'speedtests' can not be relied on.Thanks for your testing, too.
What's important (to me), is that I can do something useful like drive my car 70 miles in 1 hour (ie I know I've travelled 70mph) , rather than constant tinkering with multiple unreliable speedo's fitted to the car that all say something different (ie none of them say 70mph).
I'm going to take a back seat now in this car, but still genuinely interested from an academic point of view on how you get on in the front seat tinkering your speedo's.
Slackware image from UK server downloads at almost line rate single thread:Code: [Select]root@Home-Dream-Machine-SE:/ssd1/test# rm bigfile ; time axel -n 1 -v -a -o bigfile http://slackware.uk/slackware/slackware-iso/slackware64-15.0-iso/slackware64-15.0-install-dvd.iso
Initializing download: http://slackware.uk/slackware/slackware-iso/slackware64-15.0-iso/slackware64-15.0-install-dvd.iso
File size: 3780542464 bytes
State file found, but no downloaded data. Starting from scratch.
Opening output file bigfile
Starting download
[100%] [.............................................................................................] [ 100.8MB/s] [00:00]
Downloaded 3.5 Gigabyte in 35 seconds. (103227.12 KB/s)
real 0m35.782s
user 0m5.759s
sys 0m24.417s
That traceroute:Code: [Select]1 192.168.178.1 (192.168.178.1) 0.388 ms 0.397 ms 0.465 ms
2 lo0-0.bng4.thn-lon.zen.net.uk (51.148.77.132) 20.160 ms 20.180 ms 20.173 ms
3 lag-14.p1.thn-lon.zen.net.uk (51.148.73.94) 7.286 ms lag-14.p2.thn-lon.zen.net.uk (51.148.73.96) 9.621 ms lag-14.p1hn-lon.zen.net.uk (51.148.73.94) 7.303 ms
4 lag-2.br1.thn-lon.zen.net.uk (51.148.73.167) 7.094 ms 7.021 ms 7.069 ms
5 linx-226.as13213.net (195.66.236.19) 7.172 ms 7.164 ms 9.638 ms
6 no-ptr.midphase.com (98.158.181.92) 9.479 ms 9.107 ms 9.053 ms
7 * 212.78.92.1 (212.78.92.1) 5.041 ms 5.019 ms
8 * * *
9 * * *
10 fry.opensourcerers.net (212.78.94.73) 5.067 ms 5.013 ms 5.037 ms
11 * * *
12 * * *
13 * * *
So why are you able to download single thread from TBB at line rate while I can't, and yet I can clearly achieve line rate single thread to this Slackware server? Sounds like some bottlenecks somewhere in Zen for sure.
I ran all the same tests tonight on my zen 900 using the same gateway as Alex -all the tests are the same throughput wise apart from the single thread on tbb. I am somewhere between 550-680, can't get anywhere over that. I asked zen about this a few months ago as I noticed my single thread tests didn't seem that good when my other tests hit full speed etc. They told me it wasn't a problem but I'm not convinced. I am still on btw connection.Thanks. I wonder if Alex's is still performing that well. Of course I'm using the Zen 7530 still as a PPPoE client, I wonder how / if things will change with that swapped out.
I am somewhere between 550-680, can't get anywhere over that.
root@Home-Dream-Machine-SE:/ssd1/test# rm 512MB.zip ; time axel -U Chrome -n 1 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com:81/512MB.zip
Initializing download: http://ipv4.download.thinkbroadband.com:81/512MB.zip
File size: 536870912 bytes
State file found, but no downloaded data. Starting from scratch.
Opening output file 512MB.zip
Starting download
[100%] [...] [ 80.4MB/s] [00:00]
Downloaded 512.0 Megabyte in 6 seconds. (82306.84 KB/s)
https://forums.thinkbroadband.com/zen/t/4709246-slow-speed-after-gea-migration.html
Any thoughts folk?
I had completely forgotten about that TBB forum thread and so had another three pages of posts to read through. Phew! :o
My only thought is "currently avoid Zen".
Its certainly a long-term problem they need to solve as I was under the impression Zen were planning to ultimately cover all head-ends with their own infrastructure.Indeed, and in initial conversations Zen told me they wanted to fix it and not migrate me back as ultimately everything would migrate to their network where possible. But they really seemed to draw a blank in terms of things to fix it. There was a replacement of some SFP modules at our exchange in some bit of gear, but other than that I'm not aware of what else they really tried.
root@Home-Dream-Machine-SE:/ssd1/test# rm 512MB.zip ; time axel -U Chrome -n 1 -v -a -o 512MB.zip http://ipv4.download.thinkbroadband.com:81/512MB.zipLike a kid in a sweet shop again, rather than the sack of disappointment they delivered with the migration. Line rate here, line rate there, line rate every &&$**$ where! :)
Initializing download: http://ipv4.download.thinkbroadband.com:81/512MB.zip
File size: 536870912 bytes
Opening output file 512MB.zip
Starting download
[100%] [...] [ 101.0MB/s] [00:00]
Downloaded 512.0 Megabyte in 5 seconds. (103460.45 KB/s)
root@Home-Dream-Machine-SE:/mnt/data/speedtest# ./speedtest -s 40788
Speedtest by Ookla
Server: Zen Internet - London (id = 40788)
ISP: Zen Internet Ltd
Latency: 6.41 ms (0.08 ms jitter)
Download: 914.37 Mbps (data used: 893.9 MB )
Upload: 110.20 Mbps (data used: 49.7 MB )
Packet Loss: 0.0%
Result URL: https://www.speedtest.net/result/c/3ed9415c-b22d-4589-9c85-26c77a2f56bc
Its not clear what we looking at here, are these graphs for BTw or Zen backhaul?These are grabs of both simultaneously. I was on Zen GEA for the last two months until 31st August. The last stepchange on the graphs (ie the right hand side) is BTW, everything to the left of that stepchange in the land-of-rather-random is Zen GEA. The various stepchanges within are where you can see how different the gateways are to each other. You can see quite different jitter between two different London gateways on GEA, for instance (before August 24th and from August 24th to August 30th). On August 30th it switched to a Manchester gateway for good measure just to make it look really poor before the migration back...! Though actually not so bad for jitter.
Do you have grabs for both to show for comparison?
Incidentally I'd say no, it probably wouldn't be great to have this in routers as the tests its performing can potentially be quite invasive, at least the regular speed tests.The samknows setup does have solutions in place for both those issues. The test running is triggered centrally, so they control how many tests are running simultaneously. Otherwise the tests would be meaningless as they'd easily eat up all the test server capacity. And when a test is requested centrally, the Samknows agent has the option to abort the test if it detects significant other traffic. This seems to work well. On my unit there is a long list of tests the device refused to run or aborted because of user activity. Of course over time for watching long term trends it doesn't matter.
Nobody wants a random speed test performing in the middle of watching Netflix or worse, making a VoIP call, video conference, working from home or gaming. Plus it would be a huge waste of backhaul bandwidth, how would you schedule it so you don't suddenly have 100s or 1000s of customers all maxing out the lines at the same time? You'd jam up the network quite quickly.
Would CF not use CF backhaul?I think they're handing off at least some stuff through BTW or other providers links or facilities, as they are running fiber in Norwich into the BTW exchange.
...
Its concerning for me because when CF ever get their act together and activate my area, I feel like Zen would be where I would go (AAISP annoyed me recently with a response about outages, apparently its normal for outages to happen on a nightly basis for an always on connection), however this one issue is putting me off. No idea if there is an option for CF FTTP users to not use Zen's backhaul.
Would CF not use CF backhaul?
Ah, so presumably that is the reason why AAISP are England only?
It certainly will be interesting to see if Zen GEA on CF has the same issues as OR. If it doesn't, that would be extremely curious and give them some leverage to get OR to look into it deeper?I don't think it's even a given that Zen have this issue all over their GEA network, you'd think more folk would report it if it was widespread. It could be related to certain locations and / or equipment combinations.
You You BQM says UDM, I assume Ubiquiti? This is running PPPoE on Linux, check what firewall settings you have, as it may not perform at full line speed using PPPoE, especially if you are running IPS etc.Now the line is actually working well I did spend a bit of time looking into the Unifi UDM-Pro SE performance re: PPPoE. I am currently using the built-in 2.5G copper interface. It does PPPoE at line speed fine if you disable IPS/IDS and traffic / device identification. If you enable traffic / device identification it just about limps up to line rate (maybe -2 or -5Mbps on the downstream, still >900Mbps). If you enable moderate levels of IPS/IDS then performance drops to around 750Mbps. I've just read recent discoveries that this is much improved if you use an SFP module for the WAN instead of the built-in 2.5G copper interface; I've got a Ubiquiti SFP module on the way to try that out.
https://community.ui.com/questions/ETA-on-bugfix-for-UDM-Pro-bad-PPPoE-performance/9119aa98-412f-41c7-9188-a30036c2e4c2
Maybe turn of some of the services on the UDM briefly and run speed test(s) again.
I've just read recent discoveries that this is much improved if you use an SFP module for the WAN instead of the built-in 2.5G copper interface; I've got a Ubiquiti SFP module on the way to try that out.
Its my understanding any NIC offloading is moot on a router/gateway so how would this make a difference? You'd expect IPS/IDS to be an issue regardless as its a heavy CPU task.Reading between the lines of some slightly cryptic posts that may be from non-native speakers - it's a multicore CPU and there's suggestion that the core affinity of various things may be different when you use the SFP slot, which may mean the IPS / IDS on the SFP slot ends up running on a different core and not fighting PPPoE for resources (or similar). Or perhaps the route into the CPU is somehow less expensive computationally for the 10G SFP port than it is for the 2.5G copper port.
Alex meant that AA over CF is currently England-only. I’m of course not in England.
Ah, so presumably that is the reason why AAISP are England only?
I think it best if I just make a quick modification using my moderator's "override button".
Reading between the lines of some slightly cryptic posts that may be from non-native speakers - it's a multicore CPU and there's suggestion that the core affinity of various things may be different when you use the SFP slot, which may mean the IPS / IDS on the SFP slot ends up running on a different core and not fighting PPPoE for resources (or similar). Or perhaps the route into the CPU is somehow less expensive computationally for the 10G SFP port than it is for the 2.5G copper port.
Anyway, I don't know that I fully believe it, but it was worth the £25 that the SFP cost to try it out (instead of the 2.5G copper port). There are at least a couple of people on the thread on the Ubiquiti support site that have said moving their WAN to SFP->Copper converter allowed them to use IPS/IDS and hit full rate. I can easily test before and after and see if I see a similar difference in behaviour. Should have it by the weekend to play with.
I have an SE (on IDnet) and the £20 SPF module does improve the PPPoE situation for some reason, I even did a clean install to do testing and the 2.5G capped out around 850Mb/s at times (with no IDS) but the SPF allowed near enough full rates.Thanks for confirming. Will be interesting to see what my results are like.
This excludes IPv6 which will have issues and hit a problem getting full speed. The SPF allows near full speed with IDS on medium for me.
If you log in to ssh on and run top you should see a process ksoftirqd/0 taking large amounts of CPU and when it gets high it causes speeds to drop.
I have this extremely dodgy recollection that someone told me there was some CityFibre development in the Glasgow-Edinburgh ‘Central Belt’ of Scotland. (Or wherever the ‘Central Belt’ extends to.) Is there any truth in this or is it just nonsense? After all, J0hn will know much better than I do.
So I wonder what the "England-only" thing with AA is about ?
Cityfibres "national" product is England only, currently. I'm sure that will change over time.
AFAIK AAISP don't have anywhere "unbundled" and rely on other carriers to get across the country.
Cityfibre only launched their national access product fairly recently. Before that you needed your own kit in each of their FEX's that you wanted to offer services in.
That's why you have so many different ISP's available in different areas on Cityfibre.
So I wonder what the "England-only" thing with AA is about ?
For anyone interested, moving my WAN connection to the ONT from the built-in 2.5G copper WAN interface on the UDM-Pro SE to a UF-RJ45-1G SFP 1G SFP module in the 10G SFP+ slot took the performance with medium IDS/IPS settings from 750-800Mbps back up to full line rate - now hitting 918/110 even with high IPS/IDS settings.
Thank you for your latest update.I did do a bit of digging at the command line. The SFP connections are connected direct to the SOC from AnnaPurnaLabs - you can see they use the AnnaPurnaLabs driver. The 2.5G copper is a Realtek 8125 PCI express LAN chip. The internal 1G switch is another Realtek device, but connected directly to the SOC via MII or similar I think...
That is an interesting result, which shows there are at least two different "lanes" towards the internals of the UDM-Pro SE. One used by the 2.5Gbps metallic interface and a second used by the 10Gbps SFP+ interface.
Its looking really consistent here at 915, right now at least. I was even able to hit 855 over Wireguard.Ho hum, will have another look this evening. Do you know what gateway you're on at the moment?
I can't really comment on real-world as the only thing I know of that allows line rate are Steam downloads.
Ho hum, will have another look this evening. Do you know what gateway you're on at the moment?
Cheers
Sorry for jumping in :-)
This is a bit of a concern, as repeated nightly outages would be very unusual - I can take a look if you like? PM me (either via forum or on our IRC) if you like...
As for Zen's network, I'd not be surprised if they connect to CityFibre at multiple local POPs around the country and then run that traffic over their own national network to reach their core POPs. We're smaller, so rather than having multiple links we have a connection to CityFibre in London and CityFibre give us access to their (English) network via that link.
Well, this is the gift that keeps on giving!
After a couple of weeks of stellar performance from BTW connection, "something" has happened which appears to have spread the lacklustre performance onto the BTW connection now.
What's odd is that max performance achieved has simultaneously increased. So whereas I was pegged at 918Mbps previous, I'm now hitting up to 932Mbps to some particular segment or similar - that Norwich speedtest server 9060, for instance, but most other places are no-where near line rate.
Anyone else on Zen seeing similar?
vt1.cor2.lond2.ptn.zen.net.ukThanks. I was on that gateway at the time and was getting poor results.
Log in to Zen portal, and sure enough, I've just been migrated BACK to the GEA service from BTW.... A new unrequested migration order has appeared in my account.
Unbelievable.
If that had happened to me, it would now be a "Goodbye Zen and don't try to charge me" letter, sent to the company's registered address.The email's gone in to their cancellations team already. If the response isn't positive it's going "postal"...
Very disappointing (to put it mildly), it rather suggests when there is a problem they have no way to block the automatic migration from going through again. Your account should absolutely have been flagged for not making any changes without consulting you first, at the very least.Or maybe they do have such an ability and just forgot to enable it on my account. Who knows. It's all pretty ludicrous, though.
Reading this thread recently, it appears to me that Zen haven't really improved over the last few years sadly. It looks like you'll need to change provider.
If you're going down that route, which by emailing cancellations I'm assuming that you are, if they don't budge then leave them a detailed and critical Trustpilot review instead. I did this years ago when I had issues which at the time they weren't really interested in resolving. Someone got back to me from Zen within a few days of that review, regarding that review, and they decided to release me from the contract as well as refund some months of service on top. Trustpilot reviews can be a powerful tool for customers to use sometimes.
I will make a separate post later.
If AAISP isn't better, which I would be a little amazed if that were the case, I'm fairly certain that their support team would be interested in resolving the problem. Hopefully AAISP will be considerably better for you.I'm sure it will be, given Zen via BTW was also good for me. There's always just that small question of the unknown.
You know whats funny guys, TT were having some stability issues lately so I migrated to BTw, I assumed BTw congestion days were gone given all the nice reports about them on FTTP, but I have the exact same problems as you guys now, slow single threaded, line rate multi threaded on BT wholesale lol.Would love to understand the mechanism by which the backhaul affects single threaded performance. Given you're talking to some real people at AAISP instead of 'droids, maybe you can ask them what gives? Since it seems very odd when you'd consider that the PPPoE session I think terminates at AAISP. As a relative luddite it's hard to explain the mechanism.
BT wholesale are struggling to even do single threaded 20mbit/second, so I guess they still have congestion problems on their platform TBB visible congestion since around 5pm. Surely they must be using a different platform for FTTC and FTTP as FTTP rates that low would be really bad. I expect AAISP to give me a quick resolution though, it wouldnt be surprising if I run back to talk talk now with tail in between my legs.
Would love to understand the mechanism by which the backhaul affects single threaded performance. Given you're talking to some real people at AAISP instead of 'droids, maybe you can ask them what gives? Since it seems very odd when you'd consider that the PPPoE session I think terminates at AAISP. As a relative luddite it's hard to explain the mechanism.
See my post #202 above, although J0hn didn't agree.But surely, the PPPoE stream session looks like a single thread through the backhaul, so it's hard to understand how these mechanisms work. I'm going for there being some issue with the timing of the PPPoE arrival at the gateway (maybe it arrives in bursts) which causes the single connections through the PPPoE link to get backed off by TCP. If there is congestion on the route the PPPoE traffic gets more bursty, worsening the single threaded throughput.
Lets say you have 2 10G switches in a lab. You uplink the 2 switches using 2 x 1G ports using LAG, so the switches have 2G LAG interconnect.
You plug a 10G devices into each switch.
You then run a iperf3 single thread test between devices, will you see 1G or 2G?
Well, yesterday my connection dropped, and when it came back around I was on some odd gateway with it looks like dodgy / incomplete DNS settings.BTw?
BTw?I'm not sure what the point is that you're making. I'm not (as far as I can tell) migrated back back back back to BTW (lol) so I think this just represents some other bank of gateways hitherto unseen, as indicated by the DNS, and the IP address not being in the same range as any of the others (although it's a 72 like the BTW one). What is odd is that it is the only gateway I've seen while on GEA (assuming it hasn't migrated me back) that is performant. Maybe I was the only person on it...
Scroll back to reply #117
https://forum.kitz.co.uk/index.php/topic,27091.msg456733.html#msg456733
But surely, the PPPoE stream session looks like a single thread through the backhaul
I'm not sure what the point is that you're making.Based on what we saw around post #116, it looks like you were possibly using a gateway on the BTw backhaul when you did the traceroute on your post today.
Would love to understand the mechanism by which the backhaul affects single threaded performance. Given you're talking to some real people at AAISP instead of 'droids, maybe you can ask them what gives? Since it seems very odd when you'd consider that the PPPoE session I think terminates at AAISP. As a relative luddite it's hard to explain the mechanism.
Based on what we saw around post #116, it looks like you were possibly using a gateway on the BTw backhaul when you did the traceroute on your post today.I don't think that's possible to happen on a one-off basis as I think the switchover to BTW from Zen is a significant event (it has a 14 day leadtime) and involves someone at BT reconfiguring equipment in the exchange.
By ‘single thread’ I think people are trying to say ‘single TCP connection’, as opposed to a speed test that attempts to more fully load up the pipe full of traffic, with no ‘bubbles’, by using multiple TCP connections active simultaneously. Is that correct? (Using the water pipe analogy where water is the data and bubbles represent time where the link is idle at some point along the path. Max throughput can only be achieved when the whole pipe is full of water and no bubbles, ie full of data with no idle time at any point along the length. The water pipe analogy is not entirely correct, because water is incompressible and the max total water capacity is fixed, whereas queuing in routers means that in theory you may possibly have more data in flight by having ingress queues in middleboxes that are part filled and so that gives more room for data than is needed to just fill the pipe and get max throughput. That isn’t an entirely realistic situation everywhere though, because TCP aims to avoid that situation, as it would mean queuing for no reason. But as we know, the phenomenon of buffer bloat exists, in ingress queues at the start of a path.)
I would expect that a badly set up TCP implementation at one or t’ other end would be slow because of higher round trip time relative to the max throughput capability. Multiple TCP connections’ effect on a speed test would be like dividing the rtt by n.
Does that sound correct?
RTTs should be quite low for many of you, unless you live in far off wilderness regions like me, but the rtt is not allowed to be much at the kinds of ≥1 Gbps speeds we’re talking about.
That is interesting, as Zen does seem to top-out around 915Mbit which is lower than a lot of FTTP ISPs. It does make you wonder why this is.I'm not sure what it is. There was a period of a couple of days when I had some connections to Zen that would speedtest above that level, but it went, just like it arrived.
I've been wanting to test direct into the ONT since I got the service, but my network is never idle so unplugging the router is not really practical (and also could account for 10Mbit less on the speed test).
That is interesting, as Zen does seem to top-out around 915Mbit which is lower than a lot of FTTP ISPs. It does make you wonder why this is.
Perhaps I'm lucky. I generally get 934 (zen/btw).
It's a separate issue.Ah, Ok, that's a shame (not that it matters to me anymore)
The issue affecting Zen customers on the 900Mb/s service on their GEA network is still unresolved.
Zen acknowledged the GEA issue long before 2 weeks ago, but they claim it is affecting a tiny proportion of users.