Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 [2] 3

Author Topic: Another custom web interface for DSLStats data (by Broadstairs aka Stuart)  (Read 3007 times)

d2d4j

  • Reg Member
  • ***
  • Posts: 709

Hi Broadstairs

quote - If only hosting companies had certificates which represented the sites correctly none of this wold be needed

Please see this link why the keys have to be accepted/rejected

https://en.wikipedia.org/wiki/Trust_on_first_use

Many thanks

John
Logged

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5434

There is an option in ssh_config which allows this but so far I've not found anything in PHP which allows me to bypass the certificate verification. You are also supposed to be able to have a config file in the user space in Linux which should override defaults but no way have I been able to get this working. Also a shell script can be run to issue an ssh command but this is problematic as not all hosting package allow shell scripts.

I am starting to feel that providing SFTP in my scripts is more trouble than it is worth. For folks who have their DSLStats data on a hosted site you do not need SFTP in the scripts if the script is run from the same host. Again if the data is all on a local network then again SFTP is not really needed, nor is it if the data is on a local disk accessible by a local server.

If anyone can come up with something in PHP which will overcome my issue with certificates then I will have another go at this but right now I cannot continue to develop SFTP support.

Stuart

yeah I am not sure of the purpose of your ftp/sftp stuff, so I wouldnt stress over it too much.

It seems to make sense to have the files hosted on the same machine the script is run on.  I dont use the function, the files are uploaded to my server prior to the script been used.

This is what I have in roundcube to make it ignore trust issues on imap/smtp connections, since there is no trusted localhost cert.

$config['smtp_conn_options'] = array(
    'ssl'         => array(
    'verify_peer'  => false,
    'verify_peer_name' => false,
    'verify_depth' => 3,
    ),
);

These flags set in php itself, that syntax is just how its applied in the roundcube config.

So you basically need to disable peer verification.

Another option is just to add your own CA as a trusted provider in your local SSL cert repo.
« Last Edit: May 13, 2018, 11:24:57 AM by Chrysalis »
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5434

Hi Broadstairs

quote - If only hosting companies had certificates which represented the sites correctly none of this wold be needed



It isnt common because its an administrative nightmare, having a trusted cert for every single endpoint into the server including localhost.  It also would be expensive prior to lets encrypt.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

While I agree about the administrative problems the issue with most certs from hosting companies is that you get a warning that it does not represent the site you are accessing which worries a lot of people. I do understand that certs should be verified on first use. In part that is an issue for me in that I am still having problems with PHP even though using Filezilla I have accepted the certs and they are in my known hosts file!

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5434

Filezilla is different software to PHP.

I believe you need to disable peer verification then you good, or add the cert as trusted in the local repo (not the same as been in known hosts).

https://secure.php.net/manual/en/context.ssl.php
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

I have decided now to stop all work on providing SFTP capability in my scripts. For two reasons, first and foremost it is unnecessary if using a hosting site for the data because the scripts both run in web server mode perfectly well on the same hosting site, and secondly I have other things I want to do rather than spend time on this. I will leave the vanilla FTP capability because it is useful for those who host their data on a private lan and run their own server on the lan as well, as I have already mentioned I run using the FTP mode to access my historical data on my NAS which supports FTP but does not have a web server on it.

I will enhance the historical script to add more graphs from the stats.log file if anyone wants to request them. Eric does provide a file on the DSLStats download site which explains what is available  in the file so you can work out what is possible.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5434

Yep, ideally we could get graphs for everything MDWS had.  So I request that functionality if possible.  IF you want specific requests I will check Eric's list later.  For sure a visual graph for QLN and HLOG would be nice as the historical file is just a text file with numbers.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

Well in my local test script I've been able to graph one of the hlog files now I need to enhance it to extract the data from the zip file and allow the user to select the relevant zip file. How often does Hlog and QLN change as there are 22 zip files on my system for both Hlog and QLN. In fact there are 22 zip files for each category.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5434

I think they just change on a new sync event.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

I think they just change on a new sync event.

That's what I thought, the issue is which one do I display or do I display graphs for all available files which is likely to be quite a workload with 22 zips to extract and 22 graphs to display?

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

vic0239

  • Reg Member
  • ***
  • Posts: 329

Many thanks for the script Stuart.  Working perfectly on my two Pi systems.

https://pi4.snaddy.me.uk/dslstats/Stats.php
https://pi3.snaddy.me.uk/dslstats/Stats.php
Logged

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

@vic0239 glad to see you have it working.

As for the work in progress I now have it graphing (in test) the data from the zip files for Bits, Hlog, QLN and SNR. However this raises issues, there are a number of these zip files stored for each day my system has 23 numbered 00 to 22 for each category so I'm not sure people will need all graphed at once but I'm guessing people will want to select which one. The other issue is Eric's graphs are colour coded for u/s d/s and other, right now I dont know if I can do this with the javascript program and second if I can how it might be done or what boundaries Eric uses and are they standard?

So a number of questions there which I'd appreciate some opinions on.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 39334
  • Penguins CAN fly
    • DSLstats

Stuart, the US and DS bands are derived from the pbParams data. In the UK I understand that there are just two band plans in place, one for Huawei DSLAMs and one for ECI, so you could look in the Stats summary for the DSLAM chipset (BDCM = Huawei DSLAM or IFTN = ECI DSLAM). Each will have its own known band plan.
Logged
  Eric

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

I know have my test code being able to graph the Bits, Hlog, QLN and SNR from the zip files OK, and allowing the user to select which hourly file they want to view from all the zips that day. I have not yet investigated change the colouring per band plans but the actual graphs look OK when compared to the DSLStats ones apart from colours. I think sorting the colours will take a bit of effort as well as determining the correct plan details fo Huawei and ECI.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220

broadstairs

  • Kitizen
  • ****
  • Posts: 3100

I have put the latest versions of my script up so people can see what they now do. There has been some re-work and the radio buttons are now gone in favour of dropdown menus and now both script show the latest code and graphs. I now have to organise the  Bits, QLN SNR and Hlog zip graphs to do the colour coding according to band plans but this might take a while because the javscript code does not support what I want so the re-work is more than I intended but less that using a different javascript program.

The latest scripts can be seen at https://www.stella-maris.org.uk/DSLgraphs.php.

Stuart
Logged
ISP:TalkTalk Connection:FTTC Cab:ECI Router:Netgear D6220
Pages: 1 [2] 3