Kitz Forum

Broadband Related => FTTC and FTTP Issues => Topic started by: 7cfm on April 03, 2019, 05:54:18 AM

Title: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 05:54:18 AM
Screenshots attached from samknows and thinkbroadband BQM for the same period, the BQM looks much worse that sam knows for some reason. Browsing the internet doesn't feel too bad and we are able to watch iplayer, netflix etc without buffering, there are no gamers in the household. My biggest problem was disconnections 2 or 3 times a day but since I started using a TP-Link TD-W9970 instead of a Zyxel VMG1312-B10A as a modem and the SNR raised to 9 these seem to have settled down and I've been up about 3 weeks now (or at least that's what the modem is reporting). I'm not sure whether to report this to BT or just leave well alone?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: PhilipD on April 03, 2019, 09:38:14 AM
Hi

Those charts are very bad, showing a large amount of packet loss.

What are your modem stats, i.e. lets work from the bottom upwards.  However given you've had no disconnections and no problems browsing, it might just be a switch somewhere is quite busy and is on purpose dropping the ICMP packets.  Usually with that sort of reported packet loss you would be having issues when using the connection if it was dropping all packets and not just pings.

What were the charts like on the ZyXel?

It could simply by the TP-Link is not responding to the packets all the time (which is why it would be useful to know if the problem was also present on the ZyXel), because it's too busy doing something else (TP-Link budget kit is cheap for a reason).

Regards

Phil
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 02:25:12 PM
Hi

Packet loss on the on zyxel was broadly the same but it would lose connection 2 to 3 times a day, not a problem at 3:00am but very annoying in the middle of watching a film on netflix in the evening. The TP-Link seems to hold onto the line like glue, I know it’s a budget chipset but it seems to get good reviews over at thinkbroadband which is why I thought I’d try it. I got a lot of downstream CRC errors and errored seconds until the SNR was raised to 9 and I now get very few, I still get a lot on the upsteam but no webserver or gamers so it matters little.

I’ve attached the full output of the router, I’ve also got dslstats running 24/7 on a rpi with Kitzs custom interface and also a samknows white box so lots of stats and pretty graphs I can share if any of them would shed any light? 

BTW the modem is plugged directly into the master socket so no nasty internal wiring to interfere with it.

Cheers

Ken
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: j0hn on April 03, 2019, 02:54:55 PM
The packet loss doesn't look like it's being caused by the xDSL link as the errors look perfectly fine.
You only have 10 ES in 24 days. You have less FEC in that time than I get in 1 hour, and I consider my line to be pretty perfect.

You should be getting around 27Mb but DLM has capped the line at 22.4Mb which is why the SNRM is up at 9dB.

The ISP needs to investigate the packet loss. Don't let them send OpenReach as I don't believe their part of the link is responsible.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 03:12:24 PM
I'm not sure what you mean, doesn't openreach look after the infrastructure? What else might it be? The ISP is BT so there will be poor customer service training and language barrier to overcome first so I'm going to need to be pretty well prepped before I attempt that one.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: jelv on April 03, 2019, 03:53:42 PM
Do you see the BQM showing the same level of loss if you turn off WiFi on the router and disconnect all all your PC's etc. (including the RPi) for a period (say 20 minutes)?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 04:44:19 PM
I'll disconnect the router from the modem tonight so he modem is isolated from the LAN and see, what do you suspect is happening?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: j0hn on April 03, 2019, 05:07:10 PM
OpenReach control the pair from your home to the DSLAM/cabinet.
This isn't the issue.
They also control the fibre between the DSLAM and the exchange, but this is also unlikely to be the issue.

Packet loss is more likely to be between the exchange and the wider internet.
That's the responsibility of your ISP and/or their backhaul provider, such as BT Wholesale.

I've never seen an OpenReach engineer fix packet loss, unless the xDSL link had huge errors.

I'd be asking for BT to request BT Wholesale check if you are on a congested VLAN.

You also state you have changed modems. Have you tried a different router?.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 06:34:38 PM
Yes, when it first went in I had the BT Smarthub that they provided on the line, I initially though that was what was causing the issue.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Ronski on April 03, 2019, 08:47:04 PM
I'll disconnect the router from the modem tonight so he modem is isolated from the LAN and see, what do you suspect is happening?

If you disconnect the router from the modem, then there will be nothing to respond to the pings, that is presuming the modem is in bridge mode
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 03, 2019, 09:07:26 PM
Doh! It's been a long day :-) 

I've got an old BT Home Hub somewhere with an Ethernet port, I'll pop that on at the weekend to eliminate a router issue.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: jelv on April 04, 2019, 11:11:15 AM
I'll disconnect the router from the modem tonight so he modem is isolated from the LAN and see, what do you suspect is happening?

As Ronski posted you need something to respond to the pings.

My thought was to do an absolute belt and braces check that it is nothing internal causing the issue. You'd then be able to confirm that you had checked that when you complain to your ISP.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Chrysalis on April 06, 2019, 05:34:03 AM
i think the router might be rate limiting icmp, what router is it?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 06, 2019, 07:22:59 AM
It's an ASUS RT-AC66U_B1 with the merlin firmwire. I wondered that as well, particularly as the packet loss isn't as bad on the samknows box, presumably that is pinging outwards to a  server rather than pinging in? I'm not really sure how it works though. I'm hoping to find some time today to swap the router out and get an answer one way or another, I'm also wondering whether it's worth the effort of swapping out the modem as well, also will I still get the dropouts with the  Zyxel VMG1312-B10A  now the SNR has been raised. Also, although the TP-Link TD-W9970 is showing as not having any resynchs I'm beginning to suspect that isn't the case as I've noticed that the external iP has changed several times and the router occasionally drops connection to the modem.

So many unknowns atm.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 06, 2019, 09:12:10 AM
OK, the results are in, I've now replaced TP-Link TD-W9970 (modem) to ASUS RT-AC66U_B1 (router) with Zyxel VMG1312-B10A (modem) to BT Home Hub 5 (router) and the result is....... dramatic drum roll......100% packet loss, the BT HH5 doesn't support ping requests and is non configurable, one HH5 dropped in the bin.

Now where did I stash that travel router?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Weaver on April 06, 2019, 10:16:23 AM
It would be worth doing a speed test using the test download files of various sizes that are available on the thinkbroadband website under speed tests. If you pick one file of a certain size download it and time it then if the resulting speed is way off then we can suspect there are errors, that or congestion or alien traffic from other things going on through your internet connection or too much load on the test server or congestion on the link that goes into it. Should do several tests to see if the resulting numbers are varying, because if they are then it could well be alien traffic variations or congestion or server load variations.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 06, 2019, 07:21:15 PM
Well the proper results are in (attached) and they are pretty conclusive, it's the router. I've spent the rest of the afternoon setting up my travel router as the router and converting the Asus to a WAP as the wifi on the travel router isn't brilliant. So I've now got Zyxel VMG1312-B10A (modem) - GL-AR750S  (router) - ASUS RT-AC66U_B1 (WAP).

Thanks to every-one who chipped in with their suggestions, much appreciated
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Weaver on April 06, 2019, 09:44:14 PM
The 1312-B10A is a pretty superb piece of kit.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 07, 2019, 08:29:57 PM
The new router is setup and the packet loss is sorted. In the attached screenshot you can see when the router was running yesterday with nothing connected there was hardly any latency but today once I'd connected my LAN the latency has significantly increased. Should I be concerned about this? If so is there much I can do about it?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 08, 2019, 07:10:47 AM
Just a quick update, I disabled 2.4 wifi this morning for an hour just leaving 5 GHz wifi and wired connections still active, the attached chart is very telling, for that hour latency was fine. I guess the next step is to disable the 2.4 GHz wireless devices one by one to see what's causing the issue, unless I'm barking up the wrong tree?

The thin red line at 07:00am is where the line dropped while I was online and I had to manually update the IP address in BQM because I'm back on the Zyxel modem, I had hoped these would stop now my SNR has been raised.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: burakkucat on April 08, 2019, 09:14:29 PM
Hmm . . . Both puzzling and interesting.  :-\
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 09, 2019, 06:16:01 AM
Yes very puzzling indeed. Now I've started this thread I'm going to continue to share the results in case it helps some-one else with a similar problem in the future.

Yesterday evening I tried disabling 3 wireless devices at a time to see if I could find what which device is responsible but the results were less than conclusive. I switched off 2.4 GHz wifi overnight again (attached) and the result was that the BQM graph was very healthy. The next step is to leave the wifi on overnight and disable all but 3 devices at a time to try and isolate which one is causing the issue, unless I get lucky it could take a few days to find the culprit and I will update when i have more.

Feel free to let me know if my logic is flawed or you think something else is going on, maybe it's 2 devices 'chatting' that's causing it and it's therefore a certain combination of devices? (I guess you can tell from the way I describe things I'm not technical  :) )
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: jelv on April 09, 2019, 10:24:34 AM
Possibilities:

Are any of your devices using file sharing (Google drive, Onedrive, Dropbox etc.)?

Do any of your devices have P2P/BitTorrent software installed?

Is there an email client constantly checking for new mail?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: PhilipD on April 09, 2019, 11:32:41 AM
Hi

What is responding to the pings from the Thinkbroadband site?

Do you have with your ISP any pages that show how much data you are using, this may identify a lot of background usage from a device.  Any devices streaming all the time without your knowledge such as smart speakers left on but the volume turned down rather than streaming stopped?  Things like video players (Chromecast etc) that have wallpapers constantly changing can use quite a bit of data constantly downloading a new wall paper every few seconds (as I found!).  Any security cameras uploading to the cloud?

Don't forget if your network is being used then ping times are often higher, that isn't a problem as they are given a low priority usually.  Your issue isn't ping times, but something constantly using your network, it may just be a combination of every device doing something.  You might be able to add a QoS rule for pings so your router treats them with a high priority so the chart then excludes your own network/router busyness.

Regards

Phil
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 09, 2019, 11:55:30 AM
Thanks for the suggestions, no bit torrents or file sharing etc on that wireless connection, from memory there is only mobile phones, a youview box, a coupe of chrome sticks an amazon firestick, my dns server (pihole) and an amazon echo dot. The idea of giving pings QoS is an interesting idea and yes it crossed my mind that it might be a combination of devices which could be why taking 3 out at a time was inconclusive.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: PhilipD on April 09, 2019, 03:17:30 PM
Hi

Things like Amazon firesticks and YouView boxes can be quite chatty.

What is responding to the pings? 

Regards

Phil
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 22over7 on April 09, 2019, 04:59:01 PM
You said you've a pihole. I've found its logs (that you can look at via the admin web pages) can give you a hint about the chattiest things on your lan.
Rather appalling really.  (I now keep my amazon firestick and tablet powered off till actual use).

I guess its your ASUS wifi router that responds to pings. I wonder if failure to respond in a timely way to pings, or dropping PPPOE several
times a day might be one symptom exhibited by soon-to-fail elderly router. 

Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 09, 2019, 05:21:29 PM
I've taken to Asus off and replaced it with a gl.inet AR750S as the Asus was also dropping packets, it's the AR750S that's responding to pings now.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Ronski on April 09, 2019, 08:01:54 PM
Any chance someone else could be using your wi-fi, perhaps a password change would be a good idea - preferably something random and at least 14 characters long. Also make sure you're using WPA2.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 10, 2019, 07:00:49 AM
It's a good suggestion but I'm pretty sure that's not it, I've already got a decent WPA2 key and where I live is very rural which makes it unlikely.

The results from last night are narrowing things down, I got lucky with the 3 clients I disabled, 2 Chromecasts and an amazon firestick, you can see overnight the latency was fine, the high latency in the evening was caused by watching BBC iplayer, I'll investigate further tonight.

I thought it would be a good idea to monitor hourly data usage as well to see what correlation there was with latency and if any of my devices are using silly amounts of data and I'm going to use dslstats to do this. There are two interfaces I can monitor and I'm not sure which I should be looking at, ptm0 shows RX : 3.1 GB / TX : 1.3 GB and ptm0.1 shows RX : 3.4 GB / TX 1.1 GB, I'm using a Zyxel VMG1312-B10A?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 22over7 on April 10, 2019, 08:26:24 PM
I'm very interested in the outcome of your investigations.

For traffic, I use pmt0.1 (VMG1312-B10a). (Just to see what that blind choice looked like. )

dslstats shows daily histograms, that that more or less conform with my expectations.
There must be some kind of wrap-around, that dslatats copes with by Dark Magic. Or by Understanding.

Wow. Do you know how to ask dslstats to monitor hourly data? How?

Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 11, 2019, 06:04:59 AM
ptm0.1 is the one I went for as well. I've attached a pic of the setting you need to change in DSLStats and the graph it produces, I've also attached the results from last night. The Chromecasts are causing quite a few spikes, next I'm going to disable them and enable to firesticks to see what happens.

My prediction is that they will also cause spikes and the combination of 2 chromecasts and 2 firesticks (I found another one) is causing the latency, but we'll see.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 11, 2019, 07:15:54 AM
Also I had a look at pihole this morning, it must be the way I've got it setup but it doesn't show individual clients, it just lumps everything into my router IP address of 192.168.2.10 so it doesn't give any clues
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: jelv on April 11, 2019, 08:35:17 AM
Do you know how to ask dslstats to monitor hourly data? How?

You can't get hourly but on the traffic tab you can select Traffic / minute which gives the graph others have posted.
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 22over7 on April 11, 2019, 10:40:43 AM
Also I had a look at pihole this morning, it must be the way I've got it setup but it doesn't show individual clients, it just lumps everything into my router IP address of 192.168.2.10 so it doesn't give any clues

What I do is: go to "query log" under "long term data", click on "date and time range", then "today".  Under "recent queries", specify all (instead of last 10).
You should see a huge list of blocked and unblocked queries.
Then scroll down to some time of interest -- when you're fast asleep for example, and your main machines are off. 
In my case I see masses of lookups related to microsoft, google, bitdefender, etc etc ... and peculiar stuff I've difficulty in explaining.
(Most of this emanated from my wife's windows laptop, which is supposedly asleep.)

I find all this enlightening, and slightly horrifying.  Now and then I see something that definitely needs to be blocked.
Or some poll interval (for ntp or whatever) that can safely be whacked up.

(I'm sorry, I've not really mastered the chops to post screen shots very reliably.) 
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 11, 2019, 02:15:49 PM
It still only shows my routers IP address but you're right, all of those DNS queries when everything is supposedly shut is down is a bit of an eye opener   
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 12, 2019, 09:08:06 AM
I think this post concludes my investigation. Yesterday afternoon I disabled all chromecasts and firesticks, there are still a lot of yellow spikes on the graph but it quietens things down a lot with fewer spikes but they are still there. It would have been nice to have correlated this with the DSLStats traffic graph but I was greeted with the attached error message this morning when I logged into my Pi.

My very unscientific conclusion is that any traffic, no matter how small, will cause a spike on the BQM graph for maximum latency but subjectively this makes little real world difference with the caveat that there are no gamers in the household.

Equipment Zyxel 1312-B10A in bridge mode, GL.iNet AR750S router and DSLStats on a Pi 3B+
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Chrysalis on April 12, 2019, 11:18:34 AM
curious how fast is your sync speed?
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: 7cfm on April 12, 2019, 01:04:27 PM
Stats attached, not brilliant for FTTC but way better than we had with adsl
Title: Re: Should I be concerned with this level of packet loss and latency?
Post by: Chrysalis on April 12, 2019, 01:48:43 PM
yeah essentially the lower your burst speed, the higher and easier it is to spike your latency under load.

I mean you not in adsl territory but with that speed I can deffo see you would have issues without any QoS.

You have got rid of the packet loss, which in itself was the real problem.