Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roseway on September 11, 2017, 07:14:22 PM
-
The main improvement in this pre-release is a complete rewrite of the email alert function, to work with more recent systems. TLS version 1.2 is supported, so long as openssl v1.0.1 or later is installed on the system. For the Windows version, a pair of DLLs providing openssl v1.0.2 is included in the download package. For the Linux and Raspberry Pi versions, openssl needs to be installed; in most systems it will be installed by default. In addition, for the Raspberry Pi and Debian-based Linux systems, the package libssl-dev needs to be installed. For other types of Linux system the equivalent requirement may apply, but I don't know if it does.
Changes since the last pre-release version 6.0.6:
v6.0.9
- added "Save and Clear Now" button to event log
v6.0.8
- added "Email from" box to the alerts setup (Login username can be different)
v6.0.7
- connection speed change is not now used as an indicator of a resync
- the email alert function was rewritten to work with newer systems
Full list of changes at: http://dslstats.me.uk/changelog.html (http://dslstats.me.uk/changelog.html)
Downloads at: http://dslstats.me.uk/downloads.html (http://dslstats.me.uk/downloads.html)
My thanks to Chrysalis for testing the email alert system and providing helpful advice.
-
As I don't need or use the email alert functions will this version still work fine on the RPI and no extra bloatware has been added to this version
-
Tested on Windows 10 and working fine. I think I may have solved the problem with crashing. Bitdefender decided to block dslstats, why I have no idea but excluded dslstats from Bitdefender once it had started blocking it. The only way to do that is if Bitdefender tells you it is blocking it you can exclude it. So all running well. Will test with my ISPs servers who thinks their Cpanel may have been updated to the latest version of TLS.
Doesn't like port 587 on Gmail but got it to work with port 465.
Thanks Eric. :)
-
Many thanks Eric
my pi3 is working again using gmail and 465. I have another line with a windows 10 which continues to work after changing the port from 587 to 465.
-
Working fine here on W7. Thanks Eric.
Stuart
-
Just a quick note to say that the Linux 64-bit version is operating correctly and the "Save and Clear Now" button, under the "Event Log" tab, works exactly as expected.
Later edit: I was a little fast in typing the above. Unfortunately a problem has surfaced in that every sample is now being noted in the "Event Log". A partial screen-scrape is attached, below.
-
As I don't need or use the email alert functions will this version still work fine on the RPI and no extra bloatware has been added to this version
No, there's no extra bloatware. In fact the code in v6.0.9 is rather less bloated than earlier versions.
-
Thanks for the reports. I should have mentioned that it seems to prefer port 465 rather than 587. I don't know why.
@b*cat: What sequence of events led up to your reported situation?
-
That sounds good as the old RPI B+ is struggling under the latest RASPBIAN OS upgrade STRETCH
-
A further problem has been noted with the data shown under the "Stats" tab. It has noted a significant number of non-existing "Resyncs". An example partial screen-scrape is attached, below.
@b*cat: What sequence of events led up to your reported situation?
The tarball was downloaded and unpacked. The "dslstats64L-6.0.9" directory was entered, a double left-click on the "dslstats" icon was given and the utility was started. I then stepped through each tab, checking for any oddities.
-
It's my bedtime now, so I'll look at this tomorrow. It doesn't do anything like that here, so I've no idea at the moment.
-
Okay here under Windows 8.1. reads 0, which is correct for the uptime.
-
Ok here on Win 10 reading 0 from start of sampling a couple of hours ago and the event log is clear since testing the button.
-
I do not see the issue with resyncs on either my W7 live or my test x86-64 Linux install.
Stuart
-
Thanks all.
I think I've found a possible cause of b*cat's issue, but it does require a real resync to kick it off. I'm aware that I've made resync detection more complicated than it generally needs to be (possibly in order to meet the needs of Tecnicolor modems, but I'm not sure about this at present). I'm going to look at stripping out the unnecessary code, but I may need to make special arrangements for Technicolor.
-
port 465 is implicit ssl instead of starttls so that can give a hint whats going on, for me it works fine using starttls
-
I see, thanks. There are some options in the code which may affect this, I'll check them out.
-
I am both pleased and puzzled to be able to say that upon starting the utility this afternoon it has behaved impeccably with no spurious events appearing in the "Event Log".
There does still appear to be some sort of confusion over the "Line Attenuation" and "Signal Attenuation" values as reported under the "Stats" tab, the "Telnet Data >> Attenuation Log" and that reported in a harvest of the data generated by an invocation of "xdslctl info --linediag" at the Busybox shell.
-
Can you pls give support for 35b vdsl?
Can't underestand why bitloading works but hlog and qln doesn't...
-
I've put it on the todo list, but it won't be very soon, I'm afraid.
-
port 465 is implicit ssl instead of starttls so that can give a hint whats going on, for me it works fine using starttls
I've found the options which control this and made some changes to support both varieties. On my system I can now use either port 465 or 587 for gmail and my own email accounts. At present I'm assuming that port 25 is unencrypted, port 465 is full SSL, and all other ports are StartTLS. Are these reasonable assumptions, or do I need to give the user the option to choose the SSL/TLS type explicitly?
-
Hi roseway
Sorry, that's what I was trying to let you know when I could not remember the term implicit and explicit @chrysalis thanks
The way our implicit mail server works is as follows
Client connects using port 25 no encryption
If starttls is passed to server, the connection is upgraded to TLS secure on 587
Once upgraded to secure, credentials/information are the sent
We do not use 465 on most mail platforms (which was dropped at same time as SSLv3), but it is upto each admin
I hope that helps a little
Many thanks
John
-
Hi Roseway
To show you what I mean, please see a test for TLS on one of our platforms.
Also, please see that port 25 has startTLS as an option or plain, but port 587 only has startTLS. On our shared platforms, we only use port 25 or 587, so clients could use either port, but if using port 25, then upgrading to TLS, it changes to port 587
I hope that helps a little
Many thanks
John
port 25
220 ns1.domain.url InterWorx-CP SMTP Server Ready ESMTP
rset
250 flushed
ehlo me
250-ns1.domain.url InterWorx-CP SMTP Server Ready
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-STARTTLS
250-SIZE 52428800
250-PIPELINING
250 8BITMIME
port 587
220 ns1.domain.url InterWorx-CP SMTP Server Ready ESMTP
rset
250 flushed
ehlo me
250-ns1.domain.url InterWorx-CP SMTP Server Ready
250-STARTTLS
250-SIZE 52428800
250-PIPELINING
250 8BITMIME
External test
Trying TLS on mail.domain.url[nnn.nnn.nnn.nnn] (10):
seconds test stage and result
[000.100] Connected to server
[002.266] <-- 220 ns1.domain.url InterWorx-CP SMTP Server Ready ESMTP
[002.266] We are allowed to connect
[002.266] --> EHLO domaincheck.url
[002.365] <-- 250-ns1.domain.url InterWorx-CP SMTP Server Ready
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-STARTTLS
250-SIZE 52428800
250-PIPELINING
250 8BITMIME
[002.365] We can use this server
[002.365] TLS is an option on this server
[002.366] --> STARTTLS
[002.465] <-- 220 ready for tls
[002.465] STARTTLS command works on this server
[002.794] SSLVersion in use: TLSv1.2
[002.794] Cipher in use: AES128-SHA256
[002.794] Connection converted to SSL
[002.797] Certificate 1 of 4 in chain:
serialNumber= ec:6a:6e:cf:17:06:ba:20:13:cb:54:31:a2:45:5f:d9
subject= /OU=PositiveSSL Wildcard/CN=*.domain.url
issuer= /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
[002.799] Certificate 2 of 4 in chain:
serialNumber= 2b:2e:6e:ea:d9:75:36:6c:14:8a:6e:db:a3:7c:8c:07
subject= /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Domain Validation Secure Server CA
issuer= /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
[002.801] Certificate 3 of 4 in chain:
serialNumber= 27:66:ee:56:eb:49:f3:8e:ab:d7:70:a2:fc:84:de:22
subject= /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
issuer= /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
[002.802] Certificate 4 of 4 in chain:
serialNumber= 1
subject= /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
issuer= /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
[002.802] Cert VALIDATED:
[002.802] Cert Hostname VERIFIED (mail.domain.url = *.domain.url | DNS:*.domain.url | DNS:domain.url)
[002.803] ~~> EHLO domaincheck.url
[002.902] <~~ 250-ns1.domain.url InterWorx-CP SMTP Server Ready
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-SIZE 52428800
250-PIPELINING
250 8BITMIME
[002.902] TLS successfully started on this server
[002.903] ~~> MAIL FROM:<test@domaincheck.url>
[003.079] <~~ 250 ok
[003.079] Sender is OK
[003.079] ~~> RCPT TO:<me@domain.url>
[003.178] <~~ 250 ok
[003.178] Recipient OK, email address proofed
[003.179] ~~> QUIT
[003.278] <~~ 221 ns1.domain.url InterWorx-CP SMTP Server Ready
-
Thank you John for that helpful explanation.
-
Hi roseway
Sorry, there maybe 1 point to note which has not been mentioned sorry
On our platforms, or most anyway, we implement a wait time between initial connection and response from server
This is to help stop spammers sending email to our servers
It works in most cases because spammers send mass mail immediately and do not care what response is given. If it is accepted or rejected, they do not care as some will be accepted
I am not sure over your implementation, but if possible, I would allow a period of seconds to elapse whilst waiting for a response before dropping the email
I hope that makes sense
Many thanks
John
-
I am not sure over your implementation, but if possible, I would allow a period of seconds to elapse whilst waiting for a response before dropping the email
My implementation has a built-in timeout of several seconds, after which it returns a failure response. If the login details are wrong it returns a failure response, either after the timeout period or when the server responds, whichever is earlier.
-
I've found the options which control this and made some changes to support both varieties. On my system I can now use either port 465 or 587 for gmail and my own email accounts. At present I'm assuming that port 25 is unencrypted, port 465 is full SSL, and all other ports are StartTLS. Are these reasonable assumptions, or do I need to give the user the option to choose the SSL/TLS type explicitly?
both port 25 and 587 are typically optional unencrypted or starttls. Please dont lock down 25 to no encryption only. The reason 587 exists is that some isp's block outgoing connections to port 25 to prevent spam, so port 587 is for those people so they have an alternate port. Port 465 is implicit ssl only and is considered obsolete but some providers still allow it for older clients.
Its also possible some providers enforce encryption in which case port 25 and 587 would be starttls only.
So
25 and 587 - plain and starttls
465 - implicit ssl
I suggest disabling the sslv3 protocol, leaving tls 1, tls 1.1, and tls 1.2 enabled.
You not going to get all 3 ports working for everyone as different providers have different configurations, but if you do as I suggested then at least one port will work for people. e.g. d2d4j confirmed he doesnt support 465, but 25 and 587 would likely work on his servers.
I think allowing the tls mode to be configurable is a good idea, no need to make it over complex, I would force 465 to be implicit ssl only. But for 25 and 587 allow choice of starttls or plain. All 3 ports disable sslv3.
-
OK, thanks, I'll see what I can do with that.
-
My implementation has a built-in timeout of several seconds, after which it returns a failure response. If the login details are wrong it returns a failure response, either after the timeout period or when the server responds, whichever is earlier.
User configurable parameter with the default being what you use now?
-
User configurable parameter with the default being what you use now?
I'm using the default, but I haven't found out how to change it.
-
I just had a resync but DSLstats 6.0.9 failed to send an email alert
15 Sep 2017 13:19:36 DSL connection dropped
15 Sep 2017 13:19:36 No DSL connection
15 Sep 2017 13:20:36 No DSL connection
15 Sep 2017 13:21:38 DSL connection restored. SNRMup = 15.1 dB, SNRMdown = 4.8 dB
15 Sep 2017 13:22:19 Failed to send email alert
15 Sep 2017 13:22:19 Alert: Modem/router re-sync
15 Sep 2017 14:27:40 Email alert sent: "Alert: Downstream SNRM fell to 4.80"
15 Sep 2017 14:27:40 Alert: Downstream SNRM fell to 4.80
DSLstats 6.0.6 also failed to send an alert email 15 Sep 2017 13:19:21 DSL connection dropped
15 Sep 2017 13:19:21 No DSL connection
15 Sep 2017 13:20:21 No DSL connection
15 Sep 2017 13:22:04 Failed to sent email alert - TSendMail: SMTP error: SMTP::MailData
???-Other undefined Status503 command out of sequence -- use RSET to resynchronise
15 Sep 2017 13:22:04 Alert: Upstream SNRM fell to 0.00
15 Sep 2017 13:22:04 DSL connection restored. SNRMup = 0.0 dB, SNRMdown = 4.8 dB
15 Sep 2017 13:22:44 Failed to sent email alert - TSendMail: SMTP error: SMTP::MailData
???-Other undefined Status503 command out of sequence -- use RSET to resynchronise
15 Sep 2017 13:22:44 Alert: Modem/router re-sync
15 Sep 2017 13:22:51 Email alert sent: "Alert: Downstream SNRM fell to 4.90"
15 Sep 2017 13:22:51 Alert: Downstream SNRM fell to 4.90
15 Sep 2017 13:27:24 Email alert sent: "Alert: Downstream SNRM fell to 4.80"
15 Sep 2017 13:27:24 Alert: Downstream SNRM fell to 4.80
-
I see, so the email alert function is working, but fails with a resync. I'll have a look at that.
-
Just noticed I didn't get a resync email this morning.
Looking further it appears DslStats didn't detect the resync at all. Version 6.0.9
Did it actually detect your resync Dray? It should have counted it on the stats tab, and showed the snrm up/down on the event log if configured to do so
Email alert for resyncs worked fine on 6.0.6 for me.
-
Yes, I posted the event log; "15 Sep 2017 13:21:38 DSL connection restored. SNRMup = 15.1 dB, SNRMdown = 4.8 dB"
-
I just noticed a new field in Configuration/Alerts called Email from.
This was blank so I just entered "joe@bloggs.com" which was then auto-populated in Login username which I blanked out.
I wonder if this was stopping the emails?
-
The "Email From" field is seen by the recipient of the email as the From address. The login username can be blank if that's what your email service accepts, otherwise it must be the SMTP username (which is commonly the same as the From address.
-
Hi
I hope you don't mind, but any email platform which accepts email without login credentials (username and password) is classed as an open relay/system, so credentials should be presented.
The from email address can however, depending upon the provider, be a different email address from the credentials used
Also, on our enterprise clusters, if there is no subject or return email address (i.e. Null), it is either immediately rejected or send straight to spam).
There is only 1 exception to the above, which could only be set by the system admin, and that is if it is whitelisted. So at which it is directly delivered
I stress though, not many people would have this option as to run your own enterprise mail server costs a lot of money, and takes your time up
@roseway sorry I do not use email alerts, but if I did, our connections are whitelisted.
It does sound as though dray had a blank sender though, but it's late and I could be wrong so I apologise in advance
Many thanks
John
-
Looking at Dray's extracts from the event log, it appears that DSLstats failed to send an email alert after a resync, but succeeded in sending one after the SNRM fell below the trigger threshold. That is why I concluded that the email alert function is working, but it fails when triggered by a resync. At present I'm unsure why this is, but I'll be looking at it today.
I'm still refining the rewritten email function. In particular, in v6.0.9 it didn't fully support StartTLS, and most users would find that only port 465 would work (or unencrypted port 25). I've corrected this now. In view of d2d4j's comments, I'll disallow blank subject lines.
-
Eric I just tested the current 6.0.9 implementation to send and email alert using port 465 and it worked fine for me.
I have just one suggestion to add to this, could you please allow the user to select which alerts to send an email for? I would only want re-sync alerts by email for example, the others would probably cause too much traffic on some occasions. Also if a re-sync is not detected on a boot up, say after a power failure, could we add a boot up alert email option? The re-sync and boot up, for me at least, are the two most significant alerts I would like to see as an email.
Stuart
-
Thanks Stuart. The problem with port 465, I've recently learned, is that it's unofficial and deprecated. It's still needed with some service providers, but port 587 is preferred for encrypted emails. So I've now modified the email function to allow either port (as well as port 25), with appropriate control of the encryption type.
Regarding your suggestion, I'm trying to reduce the number of options, but in this case it looks fairly straightforward so I'll see what I can do.
-
Just noticed I didn't get a resync email this morning.
Looking further it appears DslStats didn't detect the resync at all. Version 6.0.9
Resync detection in DSLstats has been a bit haphazard, and I've now stripped it out completely and rewritten it in a much simplified fashion, which should be a lot more reliable. It's now based on the uptime (AS) value - if AS is less that in the previous sample, then a resync is recorded.
-
Eric my email provider explicitly specifies port 465 so that needs to till be allowed. I will contact them to see why they still use it. Do you have a reference for where it is specified as deprecated ? I may need that to try to get them to allow 587 which does not work for me.
As to the other bits fine if you can and have time but its not a show stopper.
Stuart
-
Stuart, you will still be able to use port 465 if your provider specifies it.
For background information see (for example) http://blog.mailgun.com/25-465-587-what-port-should-i-use/
-
For information, BT still specify port 465
-
Stuart, you will still be able to use port 465 if your provider specifies it.
For background information see (for example) http://blog.mailgun.com/25-465-587-what-port-should-i-use/
Thanks Eric, I have spoken with my email hosting company and now have port 587 working OK. They say so many devices still only support port 465 which is why it is their default.
Stuart
-
465 is considered obsolete in the sysadmin community but yes of course various providers still use it as certain providers have to be dragged across the floor to update their systems, broadband providers are particularly guilty of this problem.
e.g. plusnet not so long ago still didnt even support encryption, I dont know if its still the case now, but this is what happens when someone provides email as a free addon rather than a core part of their services.
Many modern email clients have completely dropped support for 465 implicit ssl.
So my suggestion still stands, 25, 587 plain and starttls, and 465 implicit ssl for those (legacy) providers that have not updated their systems to modern standards.
I personally dont use plain ever on email, but I suggested to keep plain for 25 and 587 as there is probably providers out there that dont support starttls.
As far as I can see starttls is already functioning properly in 6.0.8 and 6.0.9 so I dont think there is anything for eric to fix there, it wont work if the provider doesnt support it.
-
So my suggestion still stands, 25, 587 plain and starttls, and 465 implicit ssl for those (legacy) providers that have not updated their systems to modern standards.
Agreed, and the next release will conform to this.
-
PN still doesn't, at least it didn't a month or so back. TBird always asks me to confirm I know the risks. OT, it's a pain also because TBird doesn't find settings automatically.
-
6.0.9 hasn't detected my last 2 resyncs.
1 was initiated by DLM a few days ago.
Resync 5 mins ago initiated by me rebooting the modem. DslStats remained running the whole time.
-
Also had a DLM resync, but DSLStats missed it.
-
As I've said a couple of times, the resync detection is a mess. I've rewritten that function in a much simpler and cleaner way, and I'll be uploading a new version incorporating this change (among others) shortly.