Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[craigski]

I'm wondering if someone can help me understand a strange issue I am facing with my TalkTalk Fibre Large service. It has been working well for 1 month, about ~40Mb/s down and ~8Mb/s up.

Since monday, it is been performing very poorly. When I run a speedtest it fails 50% of time with latency fail. when it does work it shows 40Mb/s down but only 0.01Mb/s up.

The router shows the sync is good and there are no errors visible in the diagnostic screen.

I do a trace route to 8.8.8.8 and find the next hop IP address.

If I ping this next hop IP address directly from the router, I get a 25-50% failure, i.e. packet loss.

I have restarted the router and got a totally different IP network, but again when I do a trace route (again directly from the router web interface) and find the next hop IP address, I still get lots of packet loss.

What I don't understand is if the line sync is OK and stable with no errors (according to the TalkTalk Super Router, there is no BT Modem), why is there packet loss?

Where is the 'next hop' device physically located? i.e. when I ping the next hop is it in the cabinet or in the exchange or somewhere else? Is there anything else I can do to try and diagnose the fault? I have opened a ticket with TalkTalk, but their tests show everything is fine. It has been in this state for 24 hours.

By coincidence I checked the BT Retail status page for outages, and my exchange showed an outage yesterday (when issue started) for BT infinity customers. I explained this to Talk Talk, and they advised me that TT use separate equipment, so it is unrelated.

Here is the one stats, it has only been up 1 hour because I restarted router again hoping for issue to be resolved after 24 hours:

DSL synchronization status:
Up
Connection status:
Showtime
Upstream line rate (kbit/s):
11668
Downstream line rate (kbit/s):
50231
Maximum upstream rate (kbit/s):
11635
Maximum downstream rate (kbit/s):
61285
Upstream noise safety coefficient (dB):
6.2
Downstream noise safety coefficient (dB):
6.5
Upstream interleave depth:
0
Downstream interleave depth:
0
Line standard:
VDSL
Upstream line attenuation (dB):
34.4
Downstream line attenuation (dB):
16
Upstream output power (dBm):
7
Downstream output power (dBm):
12.4
Channel type:
None
DSL up-time:
0 days 1 hour 18 minutes 45 seconds



Interface                                   Byte         Packet       Err    Disc  Byte          Packet      Err    Disc   
INTERNET_TR069_R_VID_101   99525516   73882   0   0   1803685   19995   0   0
INTERNET_TR069_R_ETH1   0   0   0   0   0   0   0   0
INTERNET_TR069_R_0_38   0   0   0   0   0   0   0   0
Other_B_0_65   0   0   0   0   0   0   0   0

[bufferoverflow]

The speediest results are odd and show that you have issues (you have tried different speediest did you?). But it is not uncommon for routers along the path not to respond to ICMP eco requests (commonly called pings ). I believe traceroute "sort of cheats" by sending the ICMP echo request to 8.8.8.8 but artificially reduces the number of allowed hops to small increasing numbers, so the first hop responds to let you know that it can not pass on the ICMP request further up as the hop budget for the packet is already spent, so it responds on behalf of the traceroute target. Ping however wants a reply from the 1 hop router directly, which many routers are configured to simply drop...

Hope that helps (and is correct enough)

[kitz]

Your stats look fine.

To answer tp the question about next hop on a tracert.  That will be your ISPs gateway.

Although TT adsl2+ uses their own equipment.  FTTC does rely on BTs equipment more.  You wil be using a BT Openreach dslam (not the TT MSAN) and a ONT that belongs to bt and the bt DLM.  From the BT ONT you will then go on to your ISP backhaul, which is highly likely be separate from the adsl2+ traffic.  So in theory yes if there is a problem with the bt infinity equipment then it could also affect your line.  It depends where the problem is.

It may be worth running a test from visualware. Unfortunately I can't give you the link atm as I'm on a mobile device.


Edited to attempt to add a link

http://myspeed.visualware.com/index.php

[craigski]

Thanks Kitz, useful info in your post

Here is the visualware stats:


Speed test statistics
---------------------
Download speed: 30638 kbps
Upload speed: 2178 kbps
Download consistency of service: 95 %
Upload consistency of service: 56 %
Download test type: socket
Upload test type: socket
Maximum TCP delay: 16 ms
Average download pause: 1 ms
Minimum round trip time to server: 16 ms
Average round trip time to server: 19 ms
Estimated download bandwidth: 34148 kbps
Route concurrency: 1.1145552
Download TCP forced idle: 21 %
Maximum route speed: 32767 kbps
TCP MTU: 1442
Pkts Norder: 774
Bytes Norder  : 1084911
Pkts XWindow: 0
Bytes XWindow: 0
Pkts Dup: 30
Bytes Dup: 42060
Pkts Part Dup: 1
Bytes Part Dup: 668
Pkts CRC errors: 0
Pkts bad offset: 0
Pkts too short: 0
Wnd probes recvd: 0
Zero Wnd updates: 0
Bytes lost: 167572
Retransmit timeouts: 0
Fast Retransmits: 0
Packets retransmitted: 0
Bytes retransmitted: 0
Send Wnd Closed: 0
Pure Wnd Updates: 0
Acks for unsent: 0
Dup acks: 4
Wnd probes sent: 0
Persist timeouts: 0


General information
-------------------
IP address: 2.103.xxx.xxx
Local time: 27-Aug-2014 09:34:52
Test server: http://qualitytestlhr.visualware.com/

[craigski]

I ran another test, this time it was unable to complete the upload test 


Speed test statistics
---------------------
Download speed: 29686 kbps
Upload speed: -- kbps
Download consistency of service: 97 %
Upload consistency of service: -- %
Download test type: socket
Upload test type: --
Maximum TCP delay: 18 ms
Average download pause: 1 ms
Minimum round trip time to server: 16 ms
Average round trip time to server: 615 ms
Estimated download bandwidth: 34439 kbps
Route concurrency: 1.1601145
Download TCP forced idle: 97 %
Maximum route speed: 32767 kbps


General information
-------------------
IP address: 2.103.xxx.xxx
Local time: 27-Aug-2014 10:03:07
Test server: http://mcslhr.visualware.com/

[Chrysalis]

sadly we missing the all important ES/CRC stats.

Your router is also the modem built in? so when you rebooted the router you forced a new dsl sync right?

[craigski]

Hi, yes my router is the TalkTalk 'SuperRouter' that has Modem built in, I am unable to get the CRC stats.

Yes, I have rebooted the router several times. It now has a totally different public IP address.

My understanding is if there are CRC errors on the line the BT equipment will resync at a low speed until the line becomes stable ?

The stats I can see from router show it has been up for over 24 hours since last reboot. I dont think its a 'line' issue, if there were errors the line would be unstable, and reconnect?

DSL synchronization status:
Up
Connection status:
Showtime
Upstream line rate (kbit/s):
11668
Downstream line rate (kbit/s):
50231
Maximum upstream rate (kbit/s):
11873
Maximum downstream rate (kbit/s):
61358
Upstream noise safety coefficient (dB):
6.4
Downstream noise safety coefficient (dB):
6.6
Upstream interleave depth:
0
Downstream interleave depth:
0
Line standard:
VDSL
Upstream line attenuation (dB):
34.4
Downstream line attenuation (dB):
16
Upstream output power (dBm):
7.1
Downstream output power (dBm):
12.4
Channel type:
None
DSL up-time:
1 day 2 hours 36 minutes 47 seconds



I guess I will just have to wait for the TT / BT engineers to resolve the issue. It has been 48 hours so far since I reported the problem, with no ETA for a fix yet 

My fault started on 25th Aug. What I think is a *big* coincidence is BT Retail reported an issue at my exchange 'THEY' EAST HORSLEY on 25th around 10:00am on their status page affecting BT infinity customers, this was cleared at 16:00 on 25th. I also found today that a neighboring exchange is also having issues LEATHERHEAD since 25th, this is still showing on BT status page. Also there is a fault affecting Zen users with my dialing code '01483' since 25th Sep:

http://status.zen.co.uk/broadband/outages.aspx?number=01483

However, the TalkTalk status page does not show any issues with either of these exchanges.

I am now wondering if the BT & Zen issues are related? Or could Engineers working on resolving one fault be causing other issues with other lines?

[Chrysalis]

sometimes I have seen what I consider silent noise.

There will be interfeence on the line that causes packet loss and errors, but it doesnt affect the snrm, meaning if the modem is resynced it stays at the same speed and keeps the errors (unless you deliberatly force a higher srnm).

[kitz]

Re the visualware test, there's an option somewhere to do an advanced quality test.  I cant check atm as I'm on iOS which won't run java.

The test runs a thorough check on top ip tcp/ip packets and sometimes assists to identify hops,  any packet shaping, mtu issues & other packet loss issues which may be causing slow throughput.  Once the test is run (it's not the speedtest one) it gives to links to click to find further info on any identified problem areas.

---

Edit
damn auto-correct.

[burakkucat]

TalkTalk were having general broadband connection related issues, starting late on the 25th and continuing on to the 26th August. 

26/08/2014 16:50    Open    -   Broadband status   Telephone status   P1 - Some customers are reporting broadband connection related issues.    1    TalkTalk    In Progress

[kitz]

thanks b*cat for that info.

@craigski, you may not be the only one - see here

[craigski]

My service was back to normal was restored around 15:30 on 27/8. I didnt post yesterday, as I didn't was to tempt fate 
I will never know the exact issue, but I have learnt a little more in the process how FTTC works

This is what I experienced (documenting for my records):

Starting 25/8 I was seeing packet loss when pining next hop directly from my TalkTalk Router.
Line Sync was fine and stable.
Reported to TT, they said everything was fine (I think they just look at connection time, connection errors etc)
I spotted BT infinity was having issues at my exchange 'East Horsley':
Dialling Codes Affected:01483
Impacting service since 25/08/2014 09:00
We're really sorry but we've got a problem at the moment in the East Horsley area, which
means that some of our customers are having trouble getting online.

The above was cleared around 16:00 25/8
My connection still having packet loss.
TT advised still no issue, but would report to BT.
Tried to visualware test on Kitz advise, this showed the packet loss, duplicates etc.
Noticed a Neighbouring exchange was also having broadband issues (4 faults) affecting my dialling code on the Zen website. I monitored the open faults.

Incident Headline: 21CN FTTC : LEATHERHEAD : DEGRADED SERVICE
Incident Details: Some of your 21CN services may have experienced a degraded service
Area Codes: 01372 01483

26/08/2014 18:19 1 Fault diagnosis is ongoing. NE 609451 port 1/2/6 2/2/4 DEGRADED service. Next update will be issued upon service restoration or sooner should there be a change in impact. 

27/08/2014 19:42 FINAL Service fully restored at 16:40. Hardware on the Openreach (OR) network has been changed at Leatherhead. Fibres cleaned and re-seated on the TSO and OR networks at Leatherhead. Service has been monitored and remained stable. BT regrets any inconvenience this may have caused.

My service was restored around 15:45 on 27/8/2014. As my service failed and restored around same time as other faults with other ISPs on my exchange and neighbouring exchanges, this is too much of a coincidence to be unrelated.

What is annoying is TalkTalk were unaware of these issues, and they claimed my service was fine because they could see my connection was syncing fine (ie router online). They would looking at my specific connection rather than the 'bigger picture' of what was going on with other equipment. It seems that smaller 'niche' ISPs eg Zen are more aware of what is going on the BT networks than the larger ISPs, but I guess that is what you pay for at end of day 


Just for my further understanding of the FTTC architecture, I'm assuming the FTTC connection to my exchange is tunnelled to another larger exchange, so the 'next hop' IP address could be in a neighbouring exchange? If another ISP is having issues at the exchange my FTTC is tunnelled through, the issues could also affect other ISPs because of shared BT equipment for all ISPs?

[kitz]

My service was back to normal was restored around 15:30 on 27/8.

Thats good news

Some of your 21CN services may have experienced a degraded service

If it says 21CN, then I wouldnt expect it to affect Sky/TT FTTC as the 21CN specifically relates to the BTw backhaul which the LLU providers dont use.
However.....   "FTTC" regardless of LLU or BTw 21CN based providers will still use a BT OLT and also the BToR DLM..  which is linked to RAMBO and the BT bRAS via the 21CN backhaul.   How this would impact the GEA FTTC providers such as Sky and TT Ive no idea.
I guess it depends which bit is broken.

Fibres cleaned and re-seated on the TSO and OR networks at Leatherhead.

I dont know what they mean by TSO - to me it stands for "Technology, Services and Operations"...  but in the context of that sentence it could imply both the BToR network and others?  Im really not sure.

Just for my further understanding of the FTTC architecture, I'm assuming the FTTC connection to my exchange is tunnelled to another larger exchange, so the 'next hop' IP address could be in a neighbouring exchange?

No - certainly not that what would show up on a tracert.   The BT based part such as exchange is hidden anyhow using L2TP tunnelling even for BTw users.

For GEA FTTC such as Sky or TT, then traffic will go BToR dslam ( @ cab) -> OLT/Switch ( @ exchange) -> TT backhaul