Kitz Forum

Broadband Related => FTTC and FTTP Issues => Topic started by: Ixel on May 26, 2018, 10:51:01 PM

Title: Weather related question
Post by: Ixel on May 26, 2018, 10:51:01 PM
Hi all,
This evening I've got a moderate amount of rain here currently and it looks like shortly after the rain started I began getting lots more errored seconds than usual on both lines. I'm not 100% certain but I thought I heard a crackle sound every so often (maybe once or twice a minute) when testing the phone. I ran a PSTN test from AAISP's control panel on one of my lines but it came back 'pass'. Could it just be a coincidence or is perhaps rain getting in somewhere it shouldn't and therefore causing the spike in error seconds on both lines? If it's due to the rain, which I assume it is, I'm not sure how I can safely report this as a fault as the PSTN test passed on one of the lines I ran a test on and even if I did hear crackles every so often on the phone call test I did then an engineer could turn up on a day where the weather's improved and the problem source has dried out leaving no traceable fault.

In the last 15 minutes I've had 258 downstream errored seconds on one line and 277 downstream errored seconds on the other. Downstream SNRM on both lines appears to be stable. Upstream remains normal all round.

Any suggestions appreciated. I can't provide a Hlog or QLN graph since I'm using Lantiq devices (which historically have worked better for me compared to Broadcom devices). Worst case scenario I could always switch over to a Broadcom device on one of the lines just so I can get those if need be. I'm on FTTC, unfortunately connected to an ECI DSLAM :P.
Title: Re: Weather related question
Post by: grahamb on May 26, 2018, 11:24:47 PM
Thunderstorms about to turn up where I am. My error count is on the up.  :ouch:
Title: Re: Weather related question
Post by: grahamb on May 26, 2018, 11:41:12 PM
Aaand... there goes my internet.   :bye:
Title: Re: Weather related question
Post by: g3uiss on May 27, 2018, 12:27:39 AM
Hi all,
This evening I've got a moderate amount of rain here currently and it looks like shortly after the rain started I began getting lots more errored seconds than usual on both lines. I'm not 100% certain but I thought I heard a crackle sound every so often (maybe once or twice a minute) when testing the phone. I ran a PSTN test from AAISP's control panel on one of my lines but it came back 'pass'. Could it just be a coincidence or is perhaps rain getting in somewhere it shouldn't and therefore causing the spike in error seconds on both lines? If it's due to the rain, which I assume it is, I'm not sure how I can safely report this as a fault as the PSTN test passed on one of the lines I ran a test on and even if I did hear crackles every so often on the phone call test I did then an engineer could turn up on a day where the weather's improved and the problem source has dried out leaving no traceable fault.

In the last 15 minutes I've had 258 downstream errored seconds on one line and 277 downstream errored seconds on the other. Downstream SNRM on both lines appears to be stable. Upstream remains normal all round.

Any suggestions appreciated. I can't provide a Hlog or QLN graph since I'm using Lantiq devices (which historically have worked better for me compared to Broadcom devices). Worst case scenario I could always switch over to a Broadcom device on one of the lines just so I can get those if need be. I'm on FTTC, unfortunately connected to an ECI DSLAM :P.

Same here yesterday just managed to get away with less than 2880 ES and escapes DLM, today it’s been worse and going to get interleaved early in the morning. Strange as no thunderstorms in my area just rain.
Title: Re: Weather related question
Post by: Ixel on May 27, 2018, 05:29:43 AM
Well I've never seen a thunderstorm like yesterday, at least not in person. Constant flashing pretty much in the sky, it was practically daylight at times. Went on for quite some time too. I ended up turning off both modems although they got close to 2880 ES (no SES) on the downstream. SNRM changes, which my DLM programs did to both modems due to the high amount of errors, didn't make much difference. DLM hasn't intervened this morning yet and assuming I did exceed the MTBE red threshold then my guess is that it may have flagged this as an 'area wide event'... at least I hope so anyway. I'm guessing it was the rather unusual lightning, even from a distance, which caused the huge amount of ES as previously rain alone hasn't caused a spike in ES.
Title: Re: Weather related question
Post by: johnson on May 27, 2018, 07:18:03 AM
Was a similarly weird storm here. No thunder and the lightning wasnt visible for half of it, but the error counts surged. I took preemptive action after the first few 100 errored seconds and forced a resync to half speed, it stopped the error count rising much more without having to turn the modem off. Can almost plot the course of the storm through the FEC graph though, a clean curve as it approached, went over head then passed, crazy.
Title: Re: Weather related question
Post by: broadstairs on May 27, 2018, 07:43:24 AM
My experience is that storms within 50 miles of you can affect your error rate, we had no storms last night but watching my FECs (I'm interleaved) go up considerably although overall my ES rate did not change as it was only FECs. When there is a threat of a storm or I see the error rate start to climb I always take a look at Blitzortung http://en.blitzortung.org/live_lightning_maps.php?map=12 (http://en.blitzortung.org/live_lightning_maps.php?map=12) and watch the strikes. Last night the storms were way to the west of us well over 50 miles but still affected my FEC rate.

Stuart
Title: Re: Weather related question
Post by: Ixel on May 27, 2018, 09:45:29 AM
I see.

Well, although work in progress and sadly doesn't show 're-syncs' really as yet (or even when the connection went down, though it's not hard to tell when I shutdown both modems in the end) see below for the graphs from both my programs (for the DrayTek and LEDE).

http://754.dsl.diagnostics.fr89.uk
http://608.dsl.diagnostics.fr89.uk

The dates and times shown are UTC.

I've reset both lines back to an offset SNRM of 0 dB on the downstream (which makes it 6 dB again). I'm wondering if banding will kick in tomorrow on one or both of the lines due to the significant SNRM changes in a short time, hopefully not though. I'll let the '754' line work its way towards getting a lower SNRM offset again since that line is far more reliable (way less errors) than my other line '608'. The 'Downstream ES + SES (Past 24h)' graph is rolling 24 hours unless I reset the data, you'll also see green, amber, red and purple bands there representing the thresholds I configured :). Green, if the number of 'minutes' for being green is met as well as SNRM not being too low then it will decrease the downstream SNRM target offset by 0.5 dB. Amber is obviously do nothing but no 'minutes' are accumulated. Red will increase the offset by 1 dB overnight. Finally purple will increase the offset by 3 dB and take immediate action.
Title: Re: Weather related question
Post by: g3uiss on May 27, 2018, 10:03:06 AM
I got interleaved last night as I was away and couldn’t turn off modems.

So much for the ide area stuff seemed pretty wide to me 😰
Title: Re: Weather related question
Post by: Ixel on May 27, 2018, 10:09:52 AM
I got interleaved last night as I was away and couldn’t turn off modems.

So much for the ide area stuff seemed pretty wide to me 😰

That's unfortunate :(. Hopefully it won't take too long to get rid of interleaving. That reminds me, I should implement some 'admin' options in the programs so that I can manage the modems remotely (with a password) so that I can either shutdown their connection, restart the connection or such.
Title: Re: Weather related question
Post by: g3uiss on May 27, 2018, 12:06:58 PM
Likely quite easy to shut down over s VPN but not as easy to restart !

Perhaps a remote power socket the answer although all my gear runs of a couple of UPS.

Title: Re: Weather related question
Post by: Ronski on May 27, 2018, 12:35:54 PM
Didn't Kitz use a TP Link Smart plug, you could set the time for it to come back on before turning off.
Title: Re: Weather related question
Post by: Ixel on May 27, 2018, 01:56:53 PM
Yeah, a smart plug is a good option for remotely controlling power to a device. I can remotely control my UPS so could easily shut everything down if I wanted to. I'll add in some functionality today however to shutdown or restart the DSL connection on either the DrayTek or LEDE device. The LEDE should be '/etc/init.d/dsl_control stop' I believe to shutdown the DSL connection. DrayTek is possibly 'vdsl idle on' to shutdown the DSL connection, or at least put it into a 'quiet' state, I hope. To achieve this remotely I'll most likely just add in support for using websockets on both programs I'm currently developing for monitoring and interacting with both the DrayTek and LEDE device. Obviously to stop someone gaining unauthorised access I'll add a password requirement to use 'admin' functionality.
Title: Re: Weather related question
Post by: GaryW on May 29, 2018, 10:07:31 AM
I've had 3 VERY red days (6400, 7500 and 5400 ES with up to 2000 SES per day) without DLM intervening - never seen anything like it, although I've only been doing detailed monitoring since September last year.  I assume it's weather-related, and must be being treated as a wide-area event as DLM is ignoring it (so far).  The CRC graphs look like gothic arches!  The joys of long overhead lines....

Title: Re: Weather related question
Post by: Ixel on May 29, 2018, 02:22:16 PM
I've had 3 VERY red days (6400, 7500 and 5400 ES with up to 2000 SES per day) without DLM intervening - never seen anything like it, although I've only been doing detailed monitoring since September last year.  I assume it's weather-related, and must be being treated as a wide-area event as DLM is ignoring it (so far).  The CRC graphs look like gothic arches!  The joys of long overhead lines....

Ouch, well at least DLM appears to be considering that as a 'wide area event' so isn't intervening unnecessarily. I imagine if I had left the modems on that night I was getting insane error seconds then I may have been safe too, however I decided to remain cautious :P.
Title: Re: Weather related question
Post by: GaryW on May 30, 2018, 10:54:52 AM
Looks like I spoke too soon...  After 4 very red days due to electrical storms, DLM finally kicked in with a vengeance  :(

At 3am it forced a resync to impose high interleaving (but still banded at 15000), then at 10am it forced a resync with banding reduced to 13096 but interleaving back to low.  Tedious, as if it had done its usual thing of imposing high interleaving immediately the error rates would have been low enough to avoid the banding being reduced.
Title: Re: Weather related question
Post by: Ixel on May 30, 2018, 03:51:49 PM
Damn, that's unfortunate :(.