Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on September 21, 2017, 11:47:49 AM

Title: DSLstats pre-release version 6.0.11
Post by: roseway on September 21, 2017, 11:47:49 AM
Changes compared to previous pre-release v6.0.9:

- the resync detection code was rewritten to simplify it
- the email alert function now supports both Implicit SSL and StartTLS
- fixed incorrect attenuation log details with ADSL systems
- added option to choose which types of alert to send emails for

Full list of changes at: http://dslstats.me.uk/changelog.html

There are some changes to the configuration of email alerts. Previous settings should be transferred over automatically, but please check that they are correct. Hover the mouse over each item for a description of its function.

I intend to make this the basis of the next full release.

Downloads from: http://dslstats.me.uk/downloads.html
Title: Re: DSLstats pre-release version 6.0.11
Post by: Dray on September 21, 2017, 01:07:14 PM
Up and running :)
Thanks
Title: Re: DSLstats pre-release version 6.0.11
Post by: Dray on September 21, 2017, 03:29:01 PM
I notice the "Email from:" has to be formatted as an email address - "dslstats@example.com" rather than "dslstats" for the email to be accepted. But that may be because I'm using Gmail.
Title: Re: DSLstats pre-release version 6.0.11
Post by: tbailey2 on September 21, 2017, 04:27:17 PM
All okay here as well  ;)
Title: Re: DSLstats pre-release version 6.0.11
Post by: licquorice on September 21, 2017, 04:31:31 PM
All ok on RPi here.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 21, 2017, 04:34:18 PM
Thanks chaps.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 21, 2017, 04:37:50 PM
I notice the "Email from:" has to be formatted as an email address - "dslstats@example.com" rather than "dslstats" for the email to be accepted. But that may be because I'm using Gmail.

I think that the networking library demands that the From address is in that format. I'll have a look into it, but I probably won't be able to change it.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Dray on September 21, 2017, 04:41:17 PM
It's not a problem, I'm just noting I had to make the change for alerts to work.
Title: Re: DSLstats pre-release version 6.0.11
Post by: jelv on September 21, 2017, 05:18:03 PM
Don't some mail servers refuse to accept mail without a valid return address?
Title: Re: DSLstats pre-release version 6.0.11
Post by: burakkucat on September 21, 2017, 11:04:13 PM
I've only just downloaded this latest pre-release version and will give it a test tomorrow.
Title: Re: DSLstats pre-release version 6.0.11
Post by: broadstairs on September 22, 2017, 08:18:50 AM
Windows version running fine here Eric. Email alerts work fine (using test button). All alerts carried over fine. I have set up for email alrts for MDWS failure and re-sync but not others so I suspect I will not see many emails  ;)

Stuart
Title: Re: DSLstats pre-release version 6.0.11
Post by: Chrysalis on September 22, 2017, 08:37:50 AM
The from box was added yes because various email servers wont accept email without a valid from address.
Title: Re: DSLstats pre-release version 6.0.11
Post by: burakkucat on September 22, 2017, 09:22:27 PM
I have been running the utility for eight hours without any problems. Everything appears to be recorded correctly . . . including the LATN & SATN.  :thumbs:
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 22, 2017, 10:46:55 PM
Thanks all. :)
Title: Re: DSLstats pre-release version 6.0.11
Post by: broadstairs on September 24, 2017, 09:55:33 PM
Eric today I had a re-sync (which was correctly detected) however the alert email failed to be sent. Test emails still work fine. I suspect it tried to send the email before the connection was fully up, at least that's my theory.

Extract from Event Log:-
Code: [Select]
24 Sep 2017 12:38:07 Alert: Downstream CRC/min rose to 5871.00
24 Sep 2017 12:38:08 DSL connection restored. SNRMup = 6.4 dB, SNRMdown = 6.6 dB
24 Sep 2017 12:38:21 Failed to send email alert
24 Sep 2017 12:38:21 Alert: Modem/router re-sync

Stuart
Title: Re: DSLstats pre-release version 6.0.11
Post by: banger on September 24, 2017, 10:00:17 PM
I had this the other day. Could some sort of delay be inserted to ensure the connection has fully restored?
Title: Re: DSLstats pre-release version 6.0.11
Post by: j0hn on September 24, 2017, 10:04:55 PM
perhaps some kind of delay can be added to resync emails to give the line a chance to pick up a ppp session.

edit: badger beat me to it
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 24, 2017, 11:01:00 PM
I think that j0hn has probably got it right. DSLstats can only detect a resync after the DSL connection is restored (because the sample collection would fail before the connection is restored). So the likely explanation is that sometimes the PPP session won't yet have been restored. Rather than a fixed delay, it might be better to detect when the PPP session is up, if I can do a simple test for that.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Dray on September 24, 2017, 11:16:01 PM
Ping google DNS?
Title: Re: DSLstats pre-release version 6.0.11
Post by: Chrysalis on September 25, 2017, 03:01:13 AM
i assume a mdws online check would work given you already coded for that?
Title: Re: DSLstats pre-release version 6.0.11
Post by: Dray on September 25, 2017, 03:12:15 AM
DSLstats already detects the IP address so that would work?
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 25, 2017, 07:18:21 AM
Thanks for the suggestions. There are plenty of ways to do it, but they all depend on a timeout before returning a 'fail' response. I'm thinking that the best way will probably be to attempt to send the email, and if it fails, try again after (say) 15 seconds. Or something like that.

Title: Re: DSLstats pre-release version 6.0.11
Post by: d2d4j on September 25, 2017, 09:16:48 AM
Hi

@roseway - kudos to you, that's the way to do this.

The only thing I may suggest, is a 2 minute delay between attempts and an expiry after say 5 attempts.

Many thanks

John
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on September 25, 2017, 10:10:31 AM
Thanks John.
Title: Re: DSLstats pre-release version 6.0.11
Post by: ssbertilson on October 12, 2017, 02:44:18 PM
New to DSLstats, amazing program.

Have been running it for a few weeks against an Actiontec Q1000.  Has been working great, but a few times now it seems to get itself wedged and then finally just gives up:
06 Oct 2017 09:11:43   IP address is now X.X.X.X
06 Oct 2017 16:47:06   Implausible down traffic value reported by modem/router
06 Oct 2017 16:49:36   Implausible down traffic value reported by modem/router
06 Oct 2017 17:23:07   Implausible down traffic value reported by modem/router
... (didn't notice it for quite some time)
11 Oct 2017 22:06:36   Implausible down traffic value reported by modem/router
11 Oct 2017 22:32:37   Implausible down traffic value reported by modem/router
11 Oct 2017 23:07:07   Recording paused
11 Oct 2017 23:07:07   Unable to login to modem/router

I have had a packet capture running and with wireshark I get a nice view of the conversation (Analyze->Follow->TCP Stream) and, looking at the exact time when it reported the problem I can't see anything that looks awry.  Would it be possible for this log message to include a snip of what it doesn't like?  The packet capture and DSLstats are running on the same host, so I'd think the timestamps would line up as well as they could, but according to wireshark the first error didn't even happen at the time it was pulling the counters using ifconfig.

It works fine again if I just click the "go" button after it gets into "Unable to login to modem/router" state.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 12, 2017, 03:10:36 PM
The "Implausible..." message isn't related to any actual error, it's just an informational message based on an assessment of how the latest sample relates to the previous one. To be honest, it probably doesn't serve any useful purpose, and it may be a bad assessment anyway. It doesn't relate to the subsequent "Unable to login..." message.

I presume that you paused recording manually because of the "Implausible..." messages? If so, the issue here is that, after a pause, it fails to login the first time. I can't think of a reasonable explanation for this, other than some quirk of the relationship between the program and the modem.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Chrysalis on October 12, 2017, 09:11:46 PM
eric my resync today I only got an email from MDWS,. but not dslstats, however I am running 6.0.8 not 6.0.11, is there any info you want from me?

I will update to 6.0.11 in the mean time.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 12, 2017, 10:47:03 PM
Resync alerts are still work in progress, but at least with v6.0.11 DSLstats will try. The issue now seems to be that establishing the PPP connection can take some time after the resync, and so there needs to be a delay after the resync before trying to send an email alert. I'm working on it.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Chrysalis on October 12, 2017, 10:57:50 PM
ok thanks eric.
Title: Re: DSLstats pre-release version 6.0.11
Post by: ssbertilson on October 13, 2017, 01:14:24 AM
I didn't stop it...I wasn't paying that much attention to it for multiple days then happened to look at it again and found that it had stopped.

Can the paused message originate from any other source than a manual pause?
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 13, 2017, 09:22:42 AM
I don't think I've ever had a report of DSLstats pausing without it being done manually, and it's difficult to see how it could happen (apart from a gross programming error of course). I think I'd like to put that one aside, unless it happens again.

Thinking again about all those "Implausible..." messages, is it possible that you haven't configured the traffic monitoring properly? This has always been problematical because of the many different ways modems report it. There's some general advice here: http://dslstats.me.uk/trafficconfig.html (http://dslstats.me.uk/trafficconfig.html)

If the configuration isn't right it's possible that this would cause DSLstats to misbehave in some way, although this is rather unlikely. It would certainly explain the "Implausible..." messages.
Title: Re: DSLstats pre-release version 6.0.11
Post by: LordSven on October 13, 2017, 01:40:47 PM
Good day,
@ r'way: just to thank you for v6.0.11, which is working a treat on win 7 x64. An amazingly useful program. All of your hard work is much appreciated.
Cheers!
LS
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 13, 2017, 01:42:58 PM
Thank you LS :)
Title: Re: DSLstats pre-release version 6.0.11
Post by: ssbertilson on October 17, 2017, 04:16:33 PM
I don't think I've ever had a report of DSLstats pausing without it being done manually, and it's difficult to see how it could happen (apart from a gross programming error of course). I think I'd like to put that one aside, unless it happens again.
My modem took a hit last night and rebooted.  At that time it announced a pause and then apparently was unable to reconnect.  I rebooted after installing an update this morning, so had to restart DSLstats.  Before the restart the Event Log was showing the event, but now that I've restarted it, it is gone...only going back to when I restarted it.  Shouldn't ~/.dslstats/Eventlog0.txt retain past events?

Thinking again about all those "Implausible..." messages, is it possible that you haven't configured the traffic monitoring properly? This has always been problematical because of the many different ways modems report it. There's some general advice here: http://dslstats.me.uk/trafficconfig.html (http://dslstats.me.uk/trafficconfig.html)
They only come up periodically (10s of them per day) and the graphs all look great.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 17, 2017, 06:25:55 PM
Have you got "Save event log on exit" ticked, in the Misc section of Configuration?
Title: Re: DSLstats pre-release version 6.0.11
Post by: NewtronStar on October 23, 2017, 06:56:21 PM
The Bitloading on D2 has turned blue this happened before on DSLstats and fixed by turning off the 8924-B10A for 30 minutes it hasn't not worked this time, must be something to do my sync at 40000 Kbps it should be 39999 Kbps on 40/10 service.

Did see someone years ago having the same odd thing happen to them but can't find how they cured it.

Sorry you can move this into DSLstats 6.0.11 thread my mistake - Done - roseway
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 23, 2017, 07:25:53 PM
Are the other tone-related graphs coloured correctly?
Title: Re: DSLstats pre-release version 6.0.11
Post by: NewtronStar on October 23, 2017, 07:36:36 PM
I think so U0 green D1 red U1 green D2 blue also the HLOG shows as blue in that band and QLN
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 23, 2017, 10:43:22 PM
DSLstats gets its information for the different bands from the pbParams output. Can you copy the pbParams output here please (from Telnet Data --> pbParams).
Title: Re: DSLstats pre-release version 6.0.11
Post by: NewtronStar on October 23, 2017, 11:27:19 PM
Here is a copy of the pbParams

Code: [Select]
xdslctl info --pbParams
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:    0
Last initialization procedure status:   0
Max:    Upstream rate = 6191 Kbps, Downstream rate = 44174 Kbps
Bearer: 0, Upstream rate = 6191 Kbps, Downstream rate = 40000 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Discovery Phase (Initial) Band Plan
US: (7,32) (871,1205) (1972,2782)
DS: (33,859) (1216,1961) (2793,3970)
Medley Phase (Final) Band Plan
US: (7,32) (871,1205)
DS: (33,859) (1216,1961) (2793,3099)
                  VDSL Port Details               Upstream                Downstream
Attainable Net Data Rate:            6191 kbps              44174 kbps
Actual Aggregate Tx Power:         -   0.1 dBm               11.7 dBm
====================================================================================
  VDSL Band Status U0 U1 U2 U3 U4 D1 D2 D3
  Line Attenuation(dB): 8.6 42.7 N/A N/A N/A 19.8 52.4 81.0
Signal Attenuation(dB): 8.6 42.3 N/A N/A N/A 22.0 52.0 100.3
        SNR Margin(dB): 6.0 6.0 N/A N/A N/A 3.9 4.1 4.0
         TX Power(dBm): -11.4 -0.4 N/A N/A N/A 8.7 7.9 0.6
 >
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 24, 2017, 07:01:10 AM
Thank you. I see there's no U2 band but there is a D3 band after it, but with very little data in it. It's possible that you've uncovered a bug here, and of course it might go away the next time you resync. I'll get working on it.
Title: Re: DSLstats pre-release version 6.0.11
Post by: NewtronStar on October 24, 2017, 05:32:48 PM
No worry's this has only started to appeared since this line went Huawei XdB and when a retrain happens same with the RPI will update the RPI to 6.0.11 this weekend when BST ends and the clock goes back by 1 hour.
Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on October 24, 2017, 06:24:33 PM
Thanks. I've got a theory, but it needs some more thought.
Title: Re: DSLstats pre-release version 6.0.11
Post by: ssbertilson on October 29, 2017, 06:41:15 PM
Have you got "Save event log on exit" ticked, in the Misc section of Configuration?

Thanks, didn't, but do now.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Korben_dallas on November 08, 2017, 12:48:52 PM
Hi,

Over the last few days I have been getting error messages on DSLstats 6.0.11 running on my raspberry pi3 in relation to MyDSLWebStats.

On looking at DSLstats it says 'Second Stage Logon Failed' there is a reference to having selected requiring command to enter shell but that the only place I see this is for logon onto the HG612.

I have stopped/started DSLStats and rebooted the pi but still get the error.

As I have email alerting I get an email for MyDSLWebStats Upload Fail and then for it being ok.  The issue seems to be occurring nearly every few minutes.

Any ideas ?

Title: Re: DSLstats pre-release version 6.0.11
Post by: roseway on November 08, 2017, 12:59:44 PM
@Korben_Dallas: Did version 6.0.11 work properly for some time before this started happening? If so, it could be an issue with the HG612, and it would be worth rebooting that to check it out. Otherwise, is the Pi directly connected to the HG612 by ethernet, or by WiFi? If it's WiFi, there can be intermittent problems with that.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Korben_dallas on November 08, 2017, 02:36:44 PM
Hi Roseway,

The version worked for some time without any issues, the HG612 connected via a ethernet directly into the router and the pi itself is connected by ethernet into the netgear switch.

The HG612 itself had been online for over 4 months until lunchtime when it mysteriously rebooted itself.

I have re-enabled MyDSLWebStats and am going to monitor.

Any thoughts on the Billion 8800NL R2, I was thinking of replacing the HG612 which I have had for over 3 years?

Cheers
Title: Re: DSLstats pre-release version 6.0.11
Post by: tiffy on November 08, 2017, 02:59:46 PM
Apologies for jumping in, another factor to bear in mind is that micro SD cards have a limited life, there are records on the forum of failures of cards in RPi, DSLStats service, usually quite drastic, program won't boot or such, just a thought ?
Running DSLStats to MDWS involves quite a heavy shedule of reading from & writing to the SD card which can reduce it's life span considerably.

Ref. the Billion 8800NL R2, I bought one S/H from a forum member for use on my daughters PN VDSL line to replace her PN Hub One which has very limited stat's provision capabilities, very pleased with it's performance, running DSLStats on a RPi Zero W to MDWS, her ID is "cruth211", not a very good line, lots of noise but can't fault the router.
Title: Re: DSLstats pre-release version 6.0.11
Post by: Korben_dallas on November 08, 2017, 03:41:57 PM
Thanks for the input,

Am using a PiDrive with the RPi and this handles the likes of DSLStats and the Weather Program.  I suffered issues with SD Cards previously so hence got the PiDrive.

Cheers
Title: Re: DSLstats pre-release version 6.0.11
Post by: tiffy on November 08, 2017, 05:56:03 PM
Ahh, thanks for the tip on the PiDrive, had never thought of this option, looks interesting, may consider for my RPi 3, I'am still using my very old RPi1B for DSLStats and two RPi Zero W's for the kids systems, if I get a couple of years out of a micro SD card on these I will be happy.

Other forum members have reported this sort of card life span on their RPi/DSLStats systems, time will tell ?