Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: [1] 2

Author Topic: A script to display saved DSLStats graphs  (Read 2713 times)

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
A script to display saved DSLStats graphs
« on: March 31, 2018, 12:28:05 PM »

This is very much a work in progress at present. I run a full blown Apache webserver locally and have started on a php script to display the archived graphs from DSLStats. Now this will not work using the DSLStats built in webserver but it should (I hope) work using a server installed using xampp on Windows as this provides both Apache and PHP. Attached is a screen shot showing the initial test. It allows selecting the date directory and then the graph type then will display all of the graphs for that day in descending date/time order. I hope to have a version cleaned up in a few days which I can make available to anyone who wishes to use it.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #1 on: March 31, 2018, 05:42:02 PM »

I have now tested this version on my Windows xampp installation and it works, obviously it works on my Linux server as well. Currently the script will only display graphs saved in daily folders in name order by DSLStats and this directory tree must be accessible to xampp. The easiest way to do this is to get DSLStats to use a folder in the xampp/htdocs directory to save the snapshots in. So far I've not tested this accessing the snapshots on anything other than a disk on Windows (or Linux). Later I'll see if I can add using network attached disks but that will have to wait for now.

If anyone would like to try this either using xampp or their own webserver with PHP support or a linux box then please either reply here or PM me and I will let you have the code.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31845
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: A script to display saved DSLStats graphs
« Reply #2 on: March 31, 2018, 08:45:11 PM »

Well done you :) 

I've also been playing and managed to get something working which is uploading files to a remote webserver, with the idea that it would give me more power with PHP to do something for history. 
It's still a bit messy right now, but the idea is that it keeps storage small for those who may want to run it from cheap hosting.
 
atm it's here whilst I'm testing, but eventually I'll be moving it to another home that doesn't belong to  kitz.co.uk
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Westie

  • Kitizen
  • ****
  • Posts: 1506
Re: A script to display saved DSLStats graphs
« Reply #3 on: March 31, 2018, 10:39:19 PM »

ATM I am monitoring with RPi and saving daily snapshots in categories on my local NAS, on the assumption that it would then be easier to scan through the history of any given parameter.

Sadly I have no knowledge of PHP or running a webserver, but what you are both doing sounds as though it has great potential for uneducated people like me, when you have it working.

I shall watch this thread with interest! :fingers:
Logged

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #4 on: April 01, 2018, 10:09:05 AM »

I now have managed a method of retrieving the required data from an FTP server running on the network and displaying the graphs. The only thing is that so far the user/password combination is in clear in the url used which is not very secure so it would be fine on a local lan but not sure I'd want to use it in the wide world internet. OK I guess if your server does support anonymous but mine on my NAS does not. I'm guessing that most of the use of this would be on a private lan to view data so not too much of a security issue.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31845
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: A script to display saved DSLStats graphs
« Reply #5 on: April 01, 2018, 09:35:33 PM »

I've managed to get something working.     It still needs a bit of additional tidying up and it's designed for working on a remote webserver.   I'm conscious that if anyone is buying hosting, then storage will be more problematic than from a local drive, so atm it's only displaying history for SNRM and Err/Secs which TBH are about the only ones I do look at (and that's only because of the weird spiking I get with Errors), but I could easily add others.

Live graphs appear to be working fine.  I'm automatically uploading to the server and overwriting the current file so it's hardly using any storage.  :)

I think I may have to rethink my dslstats graphing habits though when it comes to the display settings.
I currently have my display set to 8hrs for easy viewing,  but I have DSLstats snap shotting every 2 hrs.   I learnt on several occasions long ago that an unexpected shut down could cause the loss of up to 8hrs data. :/ 
As such that means that I do have quite a lot of graphs per day (12) for each statistic which could soon add up. Unless I've missed some setting in DSLstats I can't think of any other way other than 24 hr graphing.  May I ask how often you get DSLstats to capture graphs as its a bit hard to see from bitswap.

 
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #6 on: April 01, 2018, 09:44:02 PM »

I now have a version which can be configured to use either a directory on the web server or an FTP server to obtain and display the saved DSLStats graphs. It is still a bit of a work in progress as it needs to have so error checking added but it does work providing you get the user configuration variables correct  ;) Obviously DSLStats does not FTP these graphs to a server but if you manually archive these files say daily then the disk space required locally is much less as there is no need to keep more that a couple or so days of graphs on say an RPi. I have a scheduled job running every 24 hours to place all my graphs on my  2TB NAS FTP server. In both modes it works fine on my W7 mini system using xampp as well as my full blown Linux Apache web server.

At present it only handles the png graphs but could be expanded to display the text files as well if needed.

A future plan is to have another script to try to handle the Stats file being FTP'd by DSLStats and draw some graphs however using server side code like PHP I suspect this may well be beyond  the processing power for RPi machines or smaller Windows machines like my Brix but it might be interesting to try. Of course javascript may be a better way to go to draw graphs as the processing is done on the browser but currently that would be a big learning curve for me however if we have a javascript guru here it might make an interesting project  ;)

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #7 on: April 01, 2018, 09:52:04 PM »

Kitz I display 2 hours on screen and currently snapshot every 2 hours giving me 12 files for each graph save and save most of them. I offload once per 24 hours to my 2TB NAS which is where I access the older graphs by FTP. I only keep 5 days of graphs on my local disk but as it is a 500Gb HDD I'm not short on space! I would not consider using my hosting service for this for a number of reasons, not least that I use it for my website and only have 5Gb space and that costs about 40/year.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5577
Re: A script to display saved DSLStats graphs
« Reply #8 on: April 02, 2018, 06:53:58 AM »

I am interested, thanks.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #9 on: April 02, 2018, 08:34:00 AM »

I am interested, thanks.

You have a PM

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #10 on: April 02, 2018, 08:37:39 AM »

With reference to the javascript I have found a downloadable (free) javascript which does graphing so all I need is to wrap PHP round the stuff to get the data and build the page for graphing the stats and this offloads the actual create/display of the graph to the browser which is good for low power processors. However it will take some time to get something working.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #11 on: April 02, 2018, 05:40:06 PM »

I've made some improvements in my script and now have a demo version on my website. To try it out go to https://www.stella-maris.org.uk/DSLgraphs.php. This is the web server version not the one which uses an FTP server for access to the files. Note that my traffic files actually do contain the word 'blank'  ;)

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

tiffy

  • Reg Member
  • ***
  • Posts: 483
Re: A script to display saved DSLStats graphs
« Reply #12 on: April 02, 2018, 09:47:21 PM »

@broadstairs:

Have been following your thread with interest, thought I would install XAMPP suite on my desktop PC running Win 8.1 64b. just to have a little play around but was somewhat put off with the pre-install warnings ref. not installing to Program Files (x86) and/or deactivating UAC.

I note you mention running XAMPP on a Win 7 environment so perhaps you could advise:
Have you found it necessary to deactivate UAC ?
Is it deemed safe to do so ?
Is it necessary to avoid installing XAMPP to Program Files (x86) ?

Have currently aborted the install, aways use "Revo Uninstaller" for all new apps. so easy to back out. 
Logged
Plusnet 40/10 - DSLStats/RPi - ZyXEL VMG1312-B10A - Huawei Cab - MDWS (sadly gone 12/03/18)

g3uiss

  • Reg Member
  • ***
  • Posts: 303
  • You never too old to learn
    • Midas Solutions
Re: A script to display saved DSLStats graphs
« Reply #13 on: April 02, 2018, 10:11:23 PM »

I've made some improvements in my script and now have a demo version on my website. To try it out go to https://www.stella-maris.org.uk/DSLgraphs.php. This is the web server version not the one which uses an FTP server for access to the files. Note that my traffic files actually do contain the word 'blank'  ;)

Stuart

Very impressive work. Thanks Tony
Logged
Zen FTTC and Plusnet ADSL HG612 modems with Vigor 2925

broadstairs

  • Kitizen
  • ****
  • Posts: 3160
Re: A script to display saved DSLStats graphs
« Reply #14 on: April 02, 2018, 10:38:17 PM »

Tiffy I always install programs like this on the D drive never on the C drive and to be honest UAC has never been a problem and never had to turn it off. Windows gets funny about some stuff. I have never used anything higher than W7 (and never will  ;) ) so not sure about W8 or W10 but my guess is it will work OK as long as you dont install it on C and UAC should be fine left on.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220
Pages: [1] 2
 

anything