Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Chrysalis]

Don't PN peer with TBB..  or at least they used to.   
In which case then there shouldnt be any external transit problems when doing speedtests with TBB

Plusnet have LINX peering yeah, of course peering can get congested its not immune to it.

Some of plusnet's recent comments have made things less conclusive from them, they commented that they checked the end points again and is sufficient capacity, but on the same page they also said there was some imbalance and some endpoints were congested.  So I take that to mean in a situation where traffic is within plusnet's expectations "and" things are well balanced then there should be no congestion PN side, of course that is an if.

[ejs]

I think that recent comment about Plusnet's network being out of balance was referring to a couple of recent events which won't have helped matters, but can't be responsible for the ongoing issues.

The graphs have looked a total mess recently, but it was still possible to spot that all the users got dropped off pcl-bng01 on Wed 11 March, and on Fri 13 March it looks like there were very few users on pcl-ag02 (and there were some service status messages yesterday).

[tommy45]

I think that recent comment about Plusnet's network being out of balance was referring to a couple of recent events which won't have helped matters, but can't be responsible for the ongoing issues.

The graphs have looked a total mess recently, but it was still possible to spot that all the users got dropped off pcl-bng01 on Wed 11 March, and on Fri 13 March it looks like there were very few users on pcl-ag02 (and there were some service status messages yesterday).

I have been on PCL bng01 since the 11th @10:30pm , The said gateway has endpoints for me that are better for latency than some of the others, both this gateway and PCL BNG 02  always seem to end up with more customers connected than the other bng's maybe lower latency is a factor for everyone ? but there are some endpoints to PCL BNG 01 that have really poor performance at peak times , i do seem to see low level packet loss more so on the ping monitor, in particular when streaming something on youtube

usage last night was lower than it was on the 12 and other nights this week , even so  single stream throughput was still affected ,but not as severely as previous nights, there also was the absence of the useual peak time increase in latency ,though there was a small increase in the level of jitter, but overall a better night that the rest of this week,
Those plusnet graphs don't show the full picture IMO, so probably serve to confuse things further maybe ?

[kitz]

Don't PN peer with TBB..  or at least they used to.   
In which case then there shouldnt be any external transit problems when doing speedtests with TBB

Plusnet have LINX peering yeah, of course peering can get congested its not immune to it.

As you say it's possible, but tbh I dont think its that.   Otherwise how could some people do perfectly fine speedtests and others get bad.  Also if there was a problem with the linx peering, then that wouldnt explain why those that who do have issues also see problems to other sites that dont use linx peering. If you get hit by the problem it seems to be all routes out affected so I think its more likely to be before or at PN than peer or transit. 

Some of plusnet's recent comments have made things less conclusive from them, they commented that they checked the end points again and is sufficient capacity, but on the same page they also said there was some imbalance and some endpoints were congested.  So I take that to mean in a situation where traffic is within plusnet's expectations "and" things are well balanced then there should be no congestion PN side, of course that is an if.

That explanation is fine by me, it can and does happen to all BTw based ISPs.   It's because of the way BTw load the pipes using round robin then this problem will continue to crop up from time to time.   As more people leave their routers on 24/7 things can become unbalanced particularly if there's been any sort of unusual activity.   

Several years ago Zen started to every night drop their users off all pipes at just gone midnight for a while in an attempt to keep their centrals balanced.    Obviously that proved to be highly unpopular.  If pipes become unbalanced the ISP has an option to drop all users on that pipe or leave things unbalanced.    When I say drop all users on the pipe, its a little more complicated than that, because they actually instruct their edge router to drop all sessions.  Depending upon the ISP network and their equipment and the fact that an edge router can be feeding several links, then you could end up dropping more users than you want.  For example in the case of Zen (and PN of old), you could have several centrals attached to the same Redback/Juniper... and hence why in the case of Zen why they were dropping all of their customers every night.    I dont keep up with what network equipment PN has these days - they are much bigger now and will have much more equipment...  but that doesn't make them immune from imbalanced pipes.

[kitz]

I have been on PCL bng01 since the 11th @10:30pm ,

Ive been on central12.pcl-bng02 [195.166.130.153] since

Code: [Select]

25 2015 Jan 23 21:19:46 PPPoE notice PPPoE ACK, ifName:ppp1.1, assigned ip=81.174.xxx.xxx gateway=195.166.130.153 nameserver=212.159.6.10,212.159.6.9 lastConnectionError=ERROR_NONE servicename=
As you will recall I'd been suffering for few weeks with the same problem you guys are seeing.  Then one evening when my speeds had dropped to circa 20Mb across the board, I went pipe hopping.   I immediately went from 20Mb straight back up to 76Mb.   Since the 23rd Jan Ive not seen any problems my speeds are fine and so are my TBB BQM.   Ive remained constantly connected to PCL-BNG2



So I think we can definitely rule out peering or transit.

If Plusnet are saying that their network is fine..  and if we know that the SVLANs/VPs are fine...   then I'm beginning to wonder about the MSILs.   How on earth they are monitored by BTw Ive no idea.


------------

ETA
Reason I say MSIL's, its because its possible that you may end up using different routing and going through different MSILs each time you drop PPP session.    For example your BTw based routing is going to be different if you connect to an endpoint connected say a PTW gateway than a PCL gateway.  Because of L2TP we dont see any of the actual routing when we do a tracert.

[Chrysalis]

I think that recent comment about Plusnet's network being out of balance was referring to a couple of recent events which won't have helped matters, but can't be responsible for the ongoing issues.

The graphs have looked a total mess recently, but it was still possible to spot that all the users got dropped off pcl-bng01 on Wed 11 March, and on Fri 13 March it looks like there were very few users on pcl-ag02 (and there were some service status messages yesterday).

remember thats just per gateway but is multiple endpoints per gateway, in other words that data isnt very useful.

[Chrysalis]

Don't PN peer with TBB..  or at least they used to.   
In which case then there shouldnt be any external transit problems when doing speedtests with TBB

Plusnet have LINX peering yeah, of course peering can get congested its not immune to it.

As you say it's possible, but tbh I dont think its that.   Otherwise how could some people do perfectly fine speedtests and others get bad.  Also if there was a problem with the linx peering, then that wouldnt explain why those that who do have issues also see problems to other sites that dont use linx peering. If you get hit by the problem it seems to be all routes out affected so I think its more likely to be before or at PN than peer or transit. 

Some of plusnet's recent comments have made things less conclusive from them, they commented that they checked the end points again and is sufficient capacity, but on the same page they also said there was some imbalance and some endpoints were congested.  So I take that to mean in a situation where traffic is within plusnet's expectations "and" things are well balanced then there should be no congestion PN side, of course that is an if.

That explanation is fine by me, it can and does happen to all BTw based ISPs.   It's because of the way BTw load the pipes using round robin then this problem will continue to crop up from time to time.   As more people leave their routers on 24/7 things can become unbalanced particularly if there's been any sort of unusual activity.   

Several years ago Zen started to every night drop their users off all pipes at just gone midnight for a while in an attempt to keep their centrals balanced.    Obviously that proved to be highly unpopular.  If pipes become unbalanced the ISP has an option to drop all users on that pipe or leave things unbalanced.    When I say drop all users on the pipe, its a little more complicated than that, because they actually instruct their edge router to drop all sessions.  Depending upon the ISP network and their equipment and the fact that an edge router can be feeding several links, then you could end up dropping more users than you want.  For example in the case of Zen (and PN of old), you could have several centrals attached to the same Redback/Juniper... and hence why in the case of Zen why they were dropping all of their customers every night.    I dont keep up with what network equipment PN has these days - they are much bigger now and will have much more equipment...  but that doesn't make them immune from imbalanced pipes.

I agree its not the peering. 

To me its either something within BTw close to PN or PN endpoint's.

If you think it might be MSIL's perhaps you can post in the PN thread to tell them to check?

[tommy45]

 This the second week that saturdays have been affected, so 7 days a week congestion

[boost]

Seems OK here:




Trace to speedtest4 = ptn-ag02 > ptn-gw01:

Code: [Select]

Tracing route to speedtest4.thinkbroadband.com [80.249.107.221]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  192.168.1.254
  2    10 ms    16 ms    12 ms  lo0-central10.ptn-ag02.plus.net [195.166.128.191
]
  3    10 ms    10 ms     9 ms  link-a-central10.ptn-gw01.plus.net [212.159.2.13
2]
  4     9 ms     9 ms     9 ms  xe-1-2-0.ptw-cr01.plus.net [212.159.0.112]
  5    10 ms    10 ms    17 ms  195.99.126.138
  6    18 ms    17 ms    14 ms  194.72.31.138
  7    11 ms    11 ms    10 ms  transit2-xe-9-1-0.ilford.ukcore.bt.net [62.172.1
03.167]
  8    14 ms    33 ms    11 ms  t2c4-xe-10-3-0-0.uk-ilf.eu.bt.net [166.49.168.10
9]
  9    11 ms    11 ms    11 ms  xe-10-3-1.edge3.London1.Level3.net [195.50.124.2
1]
 10    13 ms    14 ms    11 ms  4.68.70.78
 11    12 ms    11 ms    11 ms  ae10.mpr2.lhr2.uk.zip.zayo.com [64.125.31.194]
 12    18 ms    20 ms    19 ms  ae5.mpr1.lhr15.uk.zip.zayo.com [64.125.21.10]
 13    10 ms    11 ms    10 ms  94.31.40.154.IPYX-080002-001-ZYO.above.net [94.3
1.40.154]
 14    11 ms    10 ms    10 ms  te2-1-9.star10g.bdr-rt1.thn.ncuk.net [80.249.97.
18]

Trace to bbc = ptn-ag02 > ptn-gw02:

Code: [Select]

Tracing route to www.bbc.net.uk [212.58.246.90]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  192.168.1.254
  2    11 ms    10 ms    10 ms  lo0-central10.ptn-ag02.plus.net [195.166.128.191
]
  3    10 ms    10 ms     9 ms  link-b-central10.ptn-gw02.plus.net [212.159.2.13
4]
  4    13 ms     9 ms    10 ms  xe-1-2-0.ptw-cr02.plus.net [212.159.0.114]
  5    17 ms     9 ms     9 ms  ae2.ptw-cr01.plus.net [195.166.129.4]
  6    10 ms     9 ms     9 ms  kingston-gw.thdo.bbc.co.uk [212.58.239.6]
  7     *        *        *     Request timed out.
  8    11 ms    10 ms    10 ms  ae0.er01.cwwtf.bbc.co.uk [132.185.254.93]
  9    11 ms    11 ms    11 ms  132.185.255.165
 10    12 ms    10 ms    11 ms  bbc-vip011.cwwtf.bbc.co.uk [212.58.246.90]

Trace complete.


[tommy45]

It's ok  here too, now that i have hopped gateways & endpoints again although it was approaching 10pm , and the upstream is't that great, a single threaded upload( FTP) wasn't maxing out  around 1mbps shy  So until until this endpoint becomes saturated as well, then it's put up with congestion or hop yet again,

[boost]

Which doodar are you on? Whereabouts? etc?

[tommy45]

I'm currently on , gateway central10-ptn-bng01 exchange is LCSOU It's GEA parent is LCBIR

[kitz]

If you think it might be MSIL's perhaps you can post in the PN thread to tell them to check?

The problem is BTw are very quiet about their MSIL's, and I dont know how they monitor these and when they increase capacity. Information about the BTw MSIL's is exceedingly sparse, but they are likely to have many MSILs at each core location.  I don't have sufficient information or knowledge to be able to categorically point my finger at the MSILs.

However MSILs are a known major point of congestion.  This can be evidenced by any WBC ISPs (such as Enta) who purchase and maintain their own APs and MSILs.

Anyhow, as requested Ive made a post here.

[boost]

I stumbled across this earlier. I guess it's donkeys old but still interesting:



From that, I assume the following from the gateway names:

ptn = Telehouse North
ptw = Telehouse West?
pcl = City Lifeline

AG and BNG are presumably two different types of router. I assume both are MX, with BNG being a newer, shinier version? Or maybe ERX and MX.

Also found this forum post:

http://forums.juniper.net/t5/Routing/VPLS-Multihoming-with-Multiple-sites/td-p/184069

Where the same 3 char abbreviations are used. The topic is VPLS (bridging ethernet to remote sites) and the instance name is 'PROCERA'.

Procera Networks is a networking equipment company based in Fremont, California, United States, that designs and sells deep packet inspection (DPI), policy charging and rules function, data analytics and reporting hardware, software and services.

Because I love leaping to conclusions, my extrapolation is BNG = DPI = Reduced throughput and increased latency once load is greater than n.


Going back to the peering talk earlier, we missed something perhaps; the PN xconnects between PTN/PTW/PCL for any site specific transit. Going by the above image, Telehouse North was their first site to get a 10Gb MSIL to BT.

[Chrysalis]

If you think it might be MSIL's perhaps you can post in the PN thread to tell them to check?

The problem is BTw are very quiet about their MSIL's, and I dont know how they monitor these and when they increase capacity. Information about the BTw MSIL's is exceedingly sparse, but they are likely to have many MSILs at each core location.  I don't have sufficient information or knowledge to be able to categorically point my finger at the MSILs.

However MSILs are a known major point of congestion.  This can be evidenced by any WBC ISPs (such as Enta) who purchase and maintain their own APs and MSILs.

Anyhow, as requested Ive made a post here.

wow yeah thats a good post, thanks.

lets see what they come up with, like yourself my performance has been ok for weeks now as well.