Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roseway on May 21, 2018, 10:53:51 AM
-
The main change in this version is that SFTP uploads are now working. An external program (curl) is used for this. Linux and Raspberry Pi users must install curl on their systems for this to work. This needs to be built with SSH support, and it's assumed that this will be the case with most modern Linux systems. An appropriate version of curl is included in the Windows version, and all the files in the archive should be copied to their destination.
Full list of changes since v6.5.2:
- SFTP upload now works properly on all supported platforms
- uploading is disabled after four upload/login failures in succession
- time/day stamps of data store files are now tied to the start of processing after collecting the data from the modem
- the time between successive webserver uploads can be adjusted between 1 and 60 minutes
http://dslstats.me.uk/downloads.html
-
Running here OK although only using plain FTP currently.
Stuart
-
@roseway:
Eric, many thanks for your latest program revision and hard work on the project, very much appreciated.
Ref. installing "curl" on RPi's:
Currently running a RPi 3B+ with Raspbian Stretch on my home system, tried to install curl by the simple method:
sudo apt-get update
sudo apt-get install php5-curl (errors produced, would not install)
Googled RPi support forums and this was suggested:
sudo apt-get update
sudo apt-get install libcurl4-openssl-dev (this installed without errors)
Tried the same "curl" install process on one of my remote RPi Zero W's still running Raspbian Jessie:
Both "curl" install options worked without any errors, installed the later "libcurl4" last so presume it will be active.
Unfortunately, I still have the same issue with WS Upload to my LAN FTP server on my Win 8.1 desktop PC, all permutations of FTP, Upload URL & Upload Dir still produce a test failure although the previously reported program "lock up" when using PC IP address has now been cured.
Otherwise the program revision is up and running on all my RPi's with no noted anomalies to date.
-
Nice one Eric.. excellent work, thank you very much.
-
@tiffy: To install curl, this is what you should do:
sudo apt-get update
sudo apt-get install curl
This installs curl and libcurl3. You don't need the other packages you mentioned.
-
Eric, I appear to have an issue with data upload, The stats.log is being uploaded into the data folder, i.e. "server/DSLstats_data" not "server/DSLstats_data/2018-05-21", are you aware of this?
Also I am seeing fec.html and crc.html in the DSLstats_data folder too...
-
@tiffy: To install curl, this is what you should do:
sudo apt-get update
sudo apt-get install curl
This installs curl and libcurl3.
OK Eric, even more simple and obvious !
Was not sure on the format so Googled and got the options quoted.
Just making the point that Raspbian Stretch and Jessie appeared to react differently to the first option.
Will try the obvious option you suggested on my second remote RPi Zero W which is not updated yet.
-
Eric, I appear to have an issue with data upload, The stats.log is being uploaded into the data folder, i.e. "server/DSLstats_data" not "server/DSLstats_data/2018-05-21", are you aware of this?
Also I am seeing fec.html and crc.html in the DSLstats_data folder too...
On the second point, those two files are uploaded when you press the Test button. I meant to change their names to test1 and test2 to make their function clear, but didn't get around to it.
On the first point, I didn't specifically test the data store uploads this time because I didn't think the behaviour had changed in this version, but it looks as though I was wrong. I'll look into this and report back.
-
Yes the FTP of stats.log goes to the datastore directory not the daily directory as it used to, however the local file is correctly in the datastore directory in a daily directory.
Stuart
-
On the second point, those two files are uploaded when you press the Test button. I meant to change their names to test1 and test2 to make their function clear, but didn't get around to it.
You're just trying to confuse me.. :)
On the first point, I didn't specifically test the data store uploads this time because I didn't think the behaviour had changed in this version, but it looks as though I was wrong. I'll look into this and report back.
Cool. I think Ned has PM'd you with more info so I know your on the case.
KR
-
Special thanks from me to Eric on the continued work he is putting into dslstats.
The SFTP support significantly lowers resource usage on both client side and server side for uploads vs standard FTP as well as speeding it up, so this will help greatly when we get a new centralised service up and running.
-
Thanks Chris.
-
Concerning the Data Store upload issue, I've located the cause and fixed it for the next release.
-
Stuart
I found but after one upload using SFTP , DSLstats hung, needing a forced restart. Then I found all the SFTP settings were blank, I've tried twice with the same results
However I’ve just realised my hosting needs FTPS I assume that’s not included. I can connect and upload using P21 and the middle option, but as above crashes after one upload.
Tony
-
This one crashes out with the error "" is an invalid integer, looks like that's been there since build 6.5.2
I believe you narrowed that down to pbparams and the figures I see on 6.5 using a G.fast connection are here
bcm_xdslctl info --pbParams
bcm_xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 0
Last initialization procedure status: 0
Max: Upstream rate = 34767 Kbps, Downstream rate = 198057 Kbps
Bearer: 0, Upstream rate = 34651 Kbps, Downstream rate = 198057 Kbps
Discovery Phase (Initial) Band Plan
US: (368,1930)
DS: (368,1930)
Medley Phase (Final) Band Plan
US: (368,1339)
DS: (368,1211)
VDSL Port Details Upstream Downstream
Attainable Net Data Rate: 34767 kbps 198057 kbps
Actual Aggregate Tx Power: 2.8 dBm 0.0 dBm
====================================================================================
VDSL Band Status U0 U1 U2 U3 U4 D1 D2 D3
Line Attenuation(dB): N/A 0.0 N/A N/A N/A 37.7 N/A N/A
Signal Attenuation(dB): N/A 0.0 N/A N/A N/A 37.7 N/A N/A
SNR Margin(dB): N/A 0.0 N/A N/A N/A 0.0 N/A N/A
TX Power(dBm): N/A 0.0 N/A N/A N/A 0.0 N/A N/A
kl0(dBx100): N/A N/A N/A N/A N/A N/A N/A N/A
/ #
New territory!
-
As you say, this is new territory. G.Fast isn't VDSL2, so I guess that my formal answer at present has to be that DSLstats doesn't support G.Fast. And to be honest I don't know if I've got the energy/capability/time to add that support. I will see if I can locate the source of that error and trap it, but I'm afraid it will take some time.
-
Stuart
I found but after one upload using SFTP , DSLstats hung, needing a forced restart. Then I found all the SFTP settings were blank, I've tried twice with the same results
However I’ve just realised my hosting needs FTPS I assume that’s not included. I can connect and upload using P21 and the middle option, but as above crashes after one upload.
Tony
It was my understanding that FTPS is deprecated these days. I can add SSL support if it's really required, but I would appreciate advice on its security status.
-
Hi roseway
Kudos to you for sftp/curl which works lovely on our platforms
I thought only implicit ftps was deprecated and explicit still ongoing
I suppose in reality, there will always be a mixture but using explicit ftps, could allow for insecure ftp to be used if secure handshake fail, whereas implicit ftps would drop connection
We do not use ftps, only sftp as sftp is to us, a better system
Many thanks
John
-
Thanks for that, John.
-
It was my understanding that FTPS is deprecated these days. I can add SSL support if it's really required, but I would appreciate advice on its security status.
Well I use 1&1 on a hosted MS Server 2016, and the only permitted access is FTPS on Port 990. I'm not sure on the security, but its quite a major provider and this is a "Pro" package. They changed from FTP around 1 year ago
Tony
-
Hi g3uiss
That’s a window server so explicit ftps would be used, so it has nothing to do with size of provider
This is most likely why explicit ftps is still in use, and as you stated, fallback to 21 works
Many thanks
John
-
Thanks for the extra information. I'll add SSL support in the next release.
-
Hi g3uiss
That’s a window server so explicit ftps would be used, so it has nothing to do with size of provider
This is most likely why explicit ftps is still in use, and as you stated, fallback to 21 works
Many thanks
John
John. Thanks. I was meaning to say exactly that and suggested that I would expect that provider to have the required security. I believe they moved from Windows2012 which still supported FTP or rather my Server 2012 does 😄
Thanks to Eric in advance for adding the support.
Tony
-
Minor issue raised its head yesterday, something prevented the sftp sessions from connecting. After four attempts dslstats gives up.
Would it be possible to send a warning email when this happens?
@roseway - I realise the list of requests and additions is never ending and this is not a major issue.
-
@marjohn56: I'll see what I can do. The logical thing would be to add it to the alerts system.