Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[boost]

My VPs have been checked:

Right. There are 8 VPs I can see at your exchange from the report on the 4th Feb.

5 AMBER, 1 GREEN and 2 BLUE.

You're on the GREEN one, so in the date range used to compile the report there was no capacity issues.

I'm going to ask someone more faulty than me to have a look and see what they suggest.

Chris Parr
Plusnet Customer Relations Team

1 month old results mean nothing. Where does this elusive VP spreadsheet exist? What time of day are these VPs polled? etc etc

[tommy45]

Asking mr kennard at AAISP  may yield the info , as i doubt that you would get that from the likes of PN support. BTW you got similar results to me ,I'm on a green VP for what it's worth, thing is though with FTTC i thought it was the SVLAN status that was relevant  and not the Exchange VP's?

[kitz]

As a quick of a post as I can, because its very late... long days and also having to do lots of backend site stuff when I do get home:(...  so Im just going to blurb.


@boost - Theres an explanation and more information on such things as MSILs, APs etc on this page
http://www.kitz.co.uk/adsl/wbc_wbmc.htm

The main Core network is highly unlikely to be congested..ever. Would take a real catatrophic problem to see issues there.  Its highly resiliant and the way its meshed means that all major routes could be re-routed.   
It's unlikely to see congestion on any of the core nodes (think of this as entry point to the core), although this also could become a problem in a major outage (Say for eg when there was the Manchester fire near to / at the RAS many years ago).

Usual points of congestion are either the VP (for 20CN) or SVLAN (for FTTC) OR the ISP Host links (WBMC/IPsC).

OR Another point of congestion could be the MSILs/APs, but since PN is supposedly WBMC - Shared, then its BTw's responsibililty to maintain sufficient bandwidth on the WBC Interconnects. Ive no idea how BTw monitor these and what their proceedure is for upgrading them.


Re the SVLANs - Im not 100% certain on these.

CVLAN is the customer tag (inner), whilst SVLAN is the Service tag (outer).   I think the SVLAN is basically kind of the same as the old VPs and the amount of bandwidth available on that particular tunnel's 'backhaul', ie between the DSLAM and the RAS.

The new MSE bRAS may change things somewhat, because now many of the bRAS are closer to the dslam than they used to be.
It is possible to change SVLANs with much persausion (ie like BT would change you to a different VP if you moaned enough).  AFAIK you cant change your CVLAN, but you can (and will be) be grouped with other CVLANs in to an SVLAN.  There's 2 distinct different types of SVLANs 1)Copper-21CN 2)Fibre- FTTC.  Whatever way, I'm pretty sure that the SVLAN is the backhaul to the bRAS. 

Im unsure on FTTC if its from the DSLAM to RAS, or perhaps because of the fact DSLAM is BTor, then its possible it could be from the OLT to the bRAS.  Thinking about it further probably most likely to be OLT because this is the point where GEA is handed over... and in which case we also have a possible BT Openreach congestion point between the dslam and the OLT. 


Only your ISP or BTw has access to which SVLAN you are on.   The SVLANS are constantly monitored to make sure they dont run hot and are supposed to be upgraded whenever they reach a certain threshold.   The SVLAN status reports are updated twice weekly, but you need to know which SVLAN youre on to be able to check the reports.

[boost]

kitz: Interesting about the BTW managed interconnects. That might make a bit more sense with regard to people harping on about SVLAN issues.

jelv: You're wiltshire based? I wonder if Sheffield appears in your routing?

[jelv]

I know I go through Reading - just done a gateway hop and this is in the router log:

Code: [Select]

<132> Mar 13 09:59:27 PPP link down (Internet) [80.229.*.*]
 
<38> Mar 13 09:59:42 PPP CHAP Receive challenge from rhost ERX18.Reading (Internet)
<38> Mar 13 09:59:43 PPP CHAP Receive challenge from rhost PCL-AG05 (Internet)
<38> Mar 13 09:59:43 PPP CHAP Receive success (Internet)
<132> Mar 13 09:59:43 PPP link up (Internet) [80.229.*.*]

[boost]

I know I go through Reading - just done a gateway hop and this is in the router log:

Code: [Select]

<132> Mar 13 09:59:27 PPP link down (Internet) [80.229.7.240]
 
<38> Mar 13 09:59:42 PPP CHAP Receive challenge from rhost ERX18.Reading (Internet)
<38> Mar 13 09:59:43 PPP CHAP Receive challenge from rhost PCL-AG05 (Internet)
<38> Mar 13 09:59:43 PPP CHAP Receive success (Internet)
<132> Mar 13 09:59:43 PPP link up (Internet) [80.229.*.*]

You, Sir, are on the ball!

What does a trace route to bbc look like out of interest?

[boost]

I was looking back at the IPStream stuff earlier and came across a document, from AAISP I think, that said it was possible to move IPStream traffic over to 21CN backhaul, as maintaining the two is expensive. This makes perfect sense. This reminded me of the red flag I saw in one of the BT SINs regarding MTU across the network, specifically for the L2TP tunnels:

I'm making the assumption that PN is L2TP passthrough from the BT LAC. The more I think about it, the less sense it makes. More encapsulation, more latency but perhaps more control. I'm missing several pieces of this jigsaw puzzle, I digress.

A quick reminder of which bit of the network we're talking about - BT BRAS (LAC) to CP BRAS LNS:

http://www.sinet.bt.com/sinet/sins/pdf/472v2p6.pdf



http://www.sinet.bt.com/sinet/sins/pdf/472v2p6.pdf

TLDR; the PPP MRU must be equal both sides.

4. WBC End User Handover
4.1 Introduction
WBC can be run in two customer handover modes from the AP:
• PPP Termination and Aggregation (PTA), whereby the End User PPP Session is terminated
on the WBC BRAS and Routed IP is used to handover the End User traffic to the CP;
• Layer 2 Tunnelling Protocol (L2TP) handover, whereby the End User PPP session is
tunnelled to a CP-owned L2TP Network Server (LNS) where the PPP session is terminated.
4.2 L2TP handover
The L2TP handover solution uses IPv4 source & destination addressing.
It should be noted that if a CP opts to use L2TP handover for its end users’ sessions the L2TP control
traffic (“keep alives”) must be prioritised by the CP’s LNS. If this traffic is not prioritised and the link is
congested to the point where some of the control traffic is dropped then the CP runs the risk that the
L2TP tunnel will be dropped. This will result in the users’ sessions on that tunnel being terminated and
the users having to start a new PPP session.
4.3 PPPoE and WBC L2TP Pass through (for information)
If a WBC Customer’s End Users are using PPPoE then they should be aware of the following
behaviour with L2TP Pass through.
PPPoE clients should behave as per RFC2516[15] and RFC1661[9] – if so:-
• PPPoE End User client should send an MRU of 1492 Bytes or less to the BT LAC in the
BRAS.
• The BT LAC (in the BRAS) will send an MRU of 1492 Bytes to the PPPoE client.
• The PPPoE client and BT LAC will agree on the lower value MRU.
• The BT LAC will then proxy on this value towards the WBC Customer’s LNS.
If the PPPoE client does not obey RFC2516[15] and RFC1661[9] (which occurs regularly), one of the
following circumstances will occur:
• If the PPPoE client and BT LAC do not agree a valid MRU then the BT LAC will proxy a value
of 1500 Bytes towards the WBC Customer’s LNS.
• If the PPPoE client does not send an MRU, but agrees the BT LAC MRU of 1492 Bytes then
the BT LAC will proxy that value towards the WBC Customer’s LNS for that End User to the
WBC Customer.
• If the PPPoE client and BT LAC do not agree a valid MRU then the BT LAC will default to
proxying a value of 1500 Bytes towards the WBC Customer’s LNS which will advertise this as
the IP MTU for that End User to the WBC Customer.
• If the End Users PC is behind the device raising the PPPoE session (for example a routed
LAN), then the IP MTU of the PC must be set to 1492 Bytes or lower.
The WBC Customer’s LNS should be set up with the correct MTU depending upon the service they
wish to offer.
• If they wish to offer just PPPoE then they could set up their LNS with a statically configured IP
MTU of 1492 Bytes or lower.
• If they wish to offer just PPPoA then they could set up their LNS with a statically configured IP
MTU of 1500 Bytes. However, if an End User chose to use a PPPoE client then packets over
1492 Bytes will be carried over the WBC network but then dropped by the client.
If the WBC Customer wishes to offer a mixed PPPoE and PPPoA service then their LNS should be
able to accept the MRU proxied on by the BT LAC and assign that on a per End User basis. However,
this will not always be accurate if the PPPoE client does not obey RFC2516[15] and RFC1661[9] (as
detailed in the information above).

Juniper document detailing typical behaviour:
http://www.juniper.net/documentation/en_US/junos14.2/topics/concept/pppoe-subscriber-access-mru-mtu-overview.html

PPP MTU and MRU for Tunneled Subscribers on LNS

For PPP subscribers on L2TP network server (LNS), the configured MTU can be either the explicit MTU size specified using the mtu size statement or the derived MTU using the mtu use-lower-layer statement.

If the PPP MTU is configured as use-lower-layer, the PPP local MTU is determined as:
interface MTU – 58 bytes.
   
Note: 58 bytes is the PPP overhead payload, which is calculated as the sum of the IP, UDP, L2TP, HDLC, and PPP header payloads.
If the PPP MTU is configured using the mtu size statement, the PPP local MTU is the lesser of the configured MTU and the (interface MTU – 58 bytes) value.
When you configure an explicit MRU value by using the mru size statement, Junos OS determines the PPP local MRU value for PPP subscribers on LNS interfaces based on the following scenarios:

If the MRU value is not configured for PPP subscribers on the LNS and if the proxy LCP options are received from the L2TP access concentrator (LAC), the PPP local MRU value sent for LCP negotiation is the lesser of the local MRU (determined based on the local MTU as explained in the PPP MTU and MRU for PPPoE Subscribers section) and the proxy MRU value. If the LCP options are not received, the local MRU is sent during LCP negotiation.
If, however, the MRU value is configured for the PPP subscribers on the LNS, the local MRU is the lesser of the configured MRU and the local MTU value. Further, if the proxy LCP options are received from the LAC, the MRU value sent during LCP negotiation is the lesser of the local MRU and the proxy MRU value. If the settings are absent, the local MRU is sent as the offered MRU during LCP negotiation.

I then found this other Juniper based document which makes it seem far less black and white, although it's an older document it seems like it may be applicable:

https://www.juniper.net/techpubs/en_US/junose10.3/information-products/topic-collections/broadband-access/l2tp-packet-fragmentation.html

acket Fragmentation

The E Series router supports the reassembly of IP-fragmented L2TP packets. (For more information, see the IP Reassembly for Tunnels chapter in JUNOSe IP Services Configuration Guide.) However, it is preferable to prevent fragmentation within L2TP tunnels because of the effects of fragmentation and reassembly on performance.

To prevent fragmentation, PPP LCP negotiation of the maximum receive unit (MRU) may be used to determine a proper maximum transmission unit (MTU). However, the normal automatic method of determining the proper MRU to negotiate (by evaluating the MRU of all lower layers in the interface stack) is not adequate for L2TP. The initial LCP negotiation between PPP in the client and the LAC is inadequate because it does not cover the entire extent of the eventual PPP session that travels all the way from the client to the LNS. Furthermore, even if PPP in the LNS chooses to renegotiate the MRU, it has no way to determine the proper MRU, since it does not know the minimum MRU on all of the intervening links between it and the LAC.

To overcome the inadequacy of normal determination of the MRU under such circumstances, you can configure the PPP MRU size by using the ppp mru command in Profile Configuration mode, Interface Configuration mode, or Subinterface Configuration mode. Use Profile Configuration mode for dynamic PPP interfaces, and Interface Configuration mode or Subinterface Configuration mode for static PPP interfaces.

When you specify the size, you need to take into account the MRU for all possible links between the LAC and the LNS. You must also take into account the L2TP encapsulation that is added to all packets entering the tunnel.

For example, if the link between the LAC and LNS with the lowest MRU were an Ethernet link, the following calculation applies:

Minimum link MRU

L2TP encapsulating IP header

L2TP encapsulating UDP header

Maximum L2TP header (assumes a maximum of 16 bytes of Offset Pad)

1500

   -20

     -8

   -30

MRU size to specify

1442

If the smallest intervening link is an Ethernet link, specifying ppp mru 1442 at either the LAC or LNS guarantees that no fragmentation will occur within the L2TP tunnel.

So, what's my point?

If Router 1 establishes a tunnel to Router 2 with unknowingly mismatched MRU either side, when the end user tries to send/receive packets that exceed the lower MRU (any reasonably heavy download, video stream, bandwidth test or similar) they are going to get blackholed/dropped in the case of UDP or be constantly dropped/rewindowed (up and down) if TCP. Some tunnels are clever and will resize your packets regardless. Whether this operation can be done at line rate or is passed back to 'software' I don't know. Worst case on both is reduced throughput and increased latency.

How likely is any of this? Pretty unlikely I would have thought. There is no escaping the fact that BTW have deployed 500 odd new MSE BRAS. What is the chance of a misconfiguration or bug on one/several of these devices? Low but not impossible?

Why does the problem only present at peak times? Who knows This does provide another answer to the 'why does re-PPP seem to sort my connection out' though possibly.

That's enough straw clutching from me for now

[jelv]

Plusnet have been using IP Stream Connect for 20CN for a long time.

[boost]

Good to know

Max Premium still has a preferential weighting over WBC; AF31/AF21/AF11 I think so in the case of actual congestion, you are one of the few that should truly benefit.
Although I do seem to remember there being something about the CP having to buy special bandwidth for the Premium users? Can't remember now, will try and find it again.

[guest]

IIRC Plusnet's external (paid) routing is exceptionally limited? ISTR its basically BT public internet service & that's it.

So basically most traffic (60% +)  to/from Plusnet goes via AS2856 (BT public internet service - http://bgp.he.net/AS2856#_asinfo ) and then to AS5400 (BT plc - http://bgp.he.net/AS5400#_asinfo )

Also IIRC Plusnet only have L3 peering with them on AS2856 which caused significant issues last year when the BGP routing table exceeded 512k entries and lots of L3 kit fell over in a heap.

Frankly its pretty goddamn awful connectivity & that's where Plusnet save the pennies IMHO. Were it me I think I'd be looking at the interfaces between AS6871 (Plusnet) and AS2856 (BT Public Internet Service)...

Edit - There are differences in routing between IPv4 & IPv6 traffic from what I can see. If anyone can do a speedtest with both protocols (one after the other) from Plusnet then it'd be interesting to see if there is a difference.....

[Chrysalis]

ipv6 seems to exclusively use one provider trying to remember who they are but forgot the name, the traceroute entries have no rdns.

I agree the lack of transit diversity is an issue, they seem to have the usual peering to things like youtube, netflix, bbc etc.  But when is no peering its nearly always level3.

root@TomatoAC66:/tmp/home/root# traceroute6 google.com
traceroute to google.com (2a00:1450:400b:c02::8a), 30 hops max, 16 byte packets
 1  2a02:16c8:0:1::15 (2a02:16c8:0:1::15)  20.028 ms  39.774 ms  16.588 ms
 2  2a02:16c8:1:2::19 (2a02:16c8:1:2::19)  15.415 ms  15.382 ms  15.430 ms
 3  2a02:16c8::d (2a02:16c8::d)  15.371 ms  15.382 ms  15.017 ms
 4  2a02:16c8:1:2::8 (2a02:16c8:1:2::8)  15.795 ms  15.383 ms  15.805 ms
 5  2001:4860:1:1:0:1ad7:: (2001:4860:1:1:0:1ad7::)  15.818 ms  15.436 ms  15.770 ms
 6  2001:4860::1:0:15f (2001:4860::1:0:15f)  16.207 ms  15.775 ms  15.810 ms
 7  2001:4860::8:0:5bb9 (2001:4860::8:0:5bb9)  15.404 ms  15.394 ms  15.471 ms
 8  2001:4860::1:0:70c5 (2001:4860::1:0:70c5)  24.552 ms  24.225 ms  24.983 ms
 9  2001:4860::2:0:3de7 (2001:4860::2:0:3de7)  25.395 ms  25.379 ms  24.593 ms
10  *  *  *
11  de-in-x8a.1e100.net (2a00:1450:400b:c02::8a)  24.353 ms  24.847 ms  25.026 ms

I think I just remembered, might be hurricane electric.

[tommy45]

In that trace it would appear that Plusnet peers direct with Google,Well according to the reverse look up of the IPV6 , IPV4 appear to be the same routing to google.com  they would appear to of dropped the use of Level 3 peering by a lot recently , replaced by  BT peering now  , as if this will be a permanent thing is anyone's guess

[kitz]

Don't PN peer with TBB..  or at least they used to.   
In which case then there shouldnt be any external transit problems when doing speedtests with TBB

[kitz]

I'm making the assumption that PN is L2TP passthrough

Yep afaik practically all of the BTw based ISPs (aside from BTr) use l2tp.

A quick reminder of which bit of the network we're talking about - BT BRAS to CP BRAS:

Only BT has bRAS.    The bit you highlighted in red (aside from the MSILs) is mostly CORE transit.
SVLAN & VP is the bit before the bRAS. On 20CN, the VPs are definitely MSAN to BRAS location ie backhaul.

[tommy45]

Don't PN peer with TBB..  or at least they used to.   
In which case then there shouldnt be any external transit problems when doing speedtests with TBB

they peer via linx-gw1.thn.ncuk.net