Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on May 27, 2018, 04:17:19 PM

Title: DSLstats pre-release version 6.5.8
Post by: roseway on May 27, 2018, 04:17:19 PM
The main feature of this version is that it adds the capability to upload snapshots to a remote (S)FTP server. As there are now three separate and independent FTP upload sections, I've rearranged the Configuration section to make "FTP uploads" a separate tab under Configuration, divided into "Data Store", "Webserver" and "Snapshots". All autosaved snapshots are uploaded, with the same timing as the snapshots.

Full list of changes since v6.5.6:
http://dslstats.me.uk/downloads.html

Title: Re: DSLstats pre-release version 6.5.8
Post by: marjohn56 on May 27, 2018, 05:03:24 PM
Nice one Eric - Online. Thanks for adding the email, much appreciated.
Title: Re: DSLstats pre-release version 6.5.8
Post by: tiffy on May 27, 2018, 05:54:19 PM
Many thanks Eric for the latest program revision.

v.6.5.8 now running on my RPi 3B+ and 2 remote RPi Zero W's without any recorded issues to date.
Not using any of the FTP uploading facilities so can't comment on these revisions.
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 27, 2018, 06:25:59 PM
Nice one Eric - Online. Thanks for adding the email, much appreciated.

I may not have made it clear that the email alert has to be set up in the Alerts configuration. Enable it as a trigger, and tick it in the "Send email alerts for..." section.
Title: Re: DSLstats pre-release version 6.5.8
Post by: broadstairs on May 28, 2018, 09:32:34 AM
Eric all up and running OK W7 version.

One question, with snapshots does the delete older files happen if files are being FTP'd or only locally?

Stuart

Also seems to have created an uploads folder where the snapshots are created locally (Not on FTP server).
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 28, 2018, 10:28:47 AM
Thanks Stuart.

The way in which snapshots are saved, and older ones deleted, hasn't changed at all. The uploaded snapshots are not deleted. The Uploads folder is just a temporary holding place for snapshots which will be uploaded. They are deleted afterwards.
Title: Re: DSLstats pre-release version 6.5.8
Post by: broadstairs on May 28, 2018, 10:30:18 AM
Thanks Eric, just thought I'd mention it.  ;)

Stuart
Title: Re: DSLstats pre-release version 6.5.8
Post by: g3uiss on May 28, 2018, 01:14:09 PM
Stuart

First this version working fine.

I see the option to specify webserver upload frequency has gone. How often are the web server files uploaded by FTP ? Also not clear to me what needs to be enabled, just to upload webserver files, do I need Datastore enabled as well ?

Thanks

Tony
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 28, 2018, 03:43:00 PM
I'm guessing that your question was intended for me?

All FTP uploads are now under a separate tab under Configuration, named "FTP Uploads", which has three sections for the Data Store, Webserver files, and snapshots. In the webserver files section you'll see all the configuration stuff, including the upload frequency. The three sections are independent of one another, and can upload to different servers if that's what you want.
Title: Re: DSLstats pre-release version 6.5.8
Post by: g3uiss on May 28, 2018, 04:38:54 PM
Yes - sorry it was meant for you, and my mistake, I didn't see the extra two tabs for uploads. Just saw the Datastore options. All is now 100% clear and again thanks for the sterling work.

Lots of very useful options  ;D

Tony
Title: Re: DSLstats pre-release version 6.5.8
Post by: marjohn56 on May 29, 2018, 10:15:51 AM
Hi Eric, there appears to be an issue with the snapshots. The snapshots folder locally appears fine, however the remote snapshots folder appears to only contain some of the files and the files it does contain, which are all *.png, are all the same file renamed - namely ES per hour. Here is what I see on the remote system.

(https://preview.ibb.co/k0D5TJ/Capture.png)

Title: Re: DSLstats pre-release version 6.5.8
Post by: broadstairs on May 29, 2018, 10:22:50 AM
Same here I discover, they are all copies of the 2 day ES png file, correct names wrong data. Local data good, FTP'd data bad.

Stuart

Also seems like the delete files older than is not being honoured as I have it set to 60 days but this morning it deleted files from May.
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 29, 2018, 10:38:18 AM
Having checked my own snapshot uploads I agree that they are all copies of the ES snapshots. I have no idea how this happened, but I'll start looking straight away.
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 29, 2018, 10:40:21 AM
@Stuart: I'll look into the "Delete older than..." function.
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 29, 2018, 11:01:11 AM
I've found the reason for the uploaded snapshots all being copies of the ES snapshots - it was a bit of careless copying and pasting of code. :-[

I'm fixing it now.
Title: Re: DSLstats pre-release version 6.5.8
Post by: marjohn56 on May 29, 2018, 11:02:24 AM
Yes, the programmers favourite error... do it a lot myself.  :'(
Title: Re: DSLstats pre-release version 6.5.8
Post by: broadstairs on May 29, 2018, 11:13:15 AM
Thanks Eric, regarding the delete older than I changed the value without restarting DSLStats if that makes a difference.

Stuart
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 30, 2018, 11:37:30 AM
Stuart: Do you recall exactly what the age of the deleted snapshots was? Did it correspond with the setting you had before you changed it?
Title: Re: DSLstats pre-release version 6.5.8
Post by: broadstairs on May 30, 2018, 12:09:28 PM
Yes I think so I'm pretty sure it was 10 days. It is still set to 60 days but today deleted the 19th May snapshots which would correspond to older than 10 days.

Stuart
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on May 30, 2018, 12:38:39 PM
I think it may be the case that the old setting is retained until you restart DSLstats. I'll test this on my own system, but it obviously shouldn't be necessary to restart.
Title: Re: DSLstats pre-release version 6.5.8
Post by: g3uiss on May 31, 2018, 06:28:36 PM
Eric

I can confirm FTPS is working fine on my setup. However, it only works on P21 not P990. I thinking its falling back to explicit mode. (FTP over SSL/TLS explicit encryption)

I'm not sure how significant this is really.

Thanks for adding the FTPS, I can do everything I want to. Unfortunately at the moment I haven't worked out how to enable the Server 2016 that my host uses to parse .PHP files  :o Then I can try your scripts

Thanks Tony
Title: Re: DSLstats pre-release version 6.5.8
Post by: roseway on June 01, 2018, 07:47:09 AM
Tony,

I've looked into this a bit more, and the curl --ssl option which I'm using at present does indeed specify explicit mode with fallback to port 21. There is an alternative which I could use: instead of --ssl I can specify the FTPS protocol in the URL by adding ftps:// instead of ftp:// in front of the domain name. This sets up implicit mode, but this mode does seem to be deprecated. So I suspect that the present situation is the preferred way to do it, but I would welcome your thoughts.

PS: One option which might be successful in your case would be to leave the port number blank. This causes curl to ask the server what port to use (passive mode). I have no idea if this will work, but it's easy to test.
Title: Re: DSLstats pre-release version 6.5.8
Post by: g3uiss on June 01, 2018, 10:08:43 AM
Tony,

I've looked into this a bit more, and the curl --ssl option which I'm using at present does indeed specify explicit mode with fallback to port 21. There is an alternative which I could use: instead of --ssl I can specify the FTPS protocol in the URL by adding ftps:// instead of ftp:// in front of the domain name. This sets up implicit mode, but this mode does seem to be deprecated. So I suspect that the present situation is the preferred way to do it, but I would welcome your thoughts.

PS: One option which might be successful in your case would be to leave the port number blank. This causes curl to ask the server what port to use (passive mode). I have no idea if this will work, but it's easy to test.

Eric. Thanks. Don’t spend too much thought time on it. As it works in explicit mode on port 21 it’s fine for my purpose. I can’t see any major security issue. I will try leaving the port space blank to see if it works, I suspect it will.

Thanks for taking the time to look at it. Your work is so great for the community.
Title: Re: DSLstats pre-release version 6.5.8
Post by: g3uiss on June 01, 2018, 06:20:30 PM
Eric. Thanks. Don’t spend too much thought time on it. As it works in explicit mode on port 21 it’s fine for my purpose. I can’t see any major security issue. I will try leaving the port space blank to see if it works, I suspect it will.

Thanks for taking the time to look at it. Your work is so great for the community.

Yes it did, but put 21 back in the port box. I’ve left it at 21 with FTPS selected.

Thanks

Tony