Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on March 05, 2018, 07:01:07 PM

Title: DSLstats after the demise of MDWS
Post by: roseway on March 05, 2018, 07:01:07 PM
I've been giving some thought to what I might do to partly fill the gap. Please be aware that personal circumstances mean that I won't be willing to take on any extensive new development, but I can do some small things.

One very simple option is the proposal by ktz392837 to save frequent zipped copies of the raw telnet data. This would at least preserve the history for as long as the user wants.

Probably a more interesting option would be to retain most of the MDWS-related code but copy the data to a local folder instead of uploading it to MDWS.. The data is in a zipped CSV format, so it can be displayed in a spreadsheet for viewing or further processing. I could also upload this data to a publicly accessible site if someone is prepared to set this up and administer it. This would of course represent a substantial commitment by someone.

These are just some initial thoughts. What do you all think?

Title: Re: DSLstats after the demise of MDWS
Post by: burakkucat on March 05, 2018, 07:11:24 PM
Probably a more interesting option would be to retain most of the MDWS-related code but copy the data to a local folder instead of uploading it to MDWS. The data is in a zipped CSV format, so it can be displayed in a spreadsheet for viewing or further processing.

The first part of your second suggestion (above) would be of interest to me, please.
Title: Re: DSLstats after the demise of MDWS
Post by: broadstairs on March 05, 2018, 07:37:22 PM
Eric I think your suggestion to archive a zip file on the local machine would be a good one as in the past it is the old data which has been of use to me when arguing with my ISP.

Stuart
Title: Re: DSLstats after the demise of MDWS
Post by: Deathstar on March 05, 2018, 07:43:54 PM
Another option for me is to extend the "delete snapshots after X days" to indefinite.
That way allow the user to store as much historical data as they please.
I've not looked into the MWDS folder as of yet due to my own time constraints, but I assume it's an a format that is excellable...
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 05, 2018, 07:46:33 PM
Roseway I am happy for anything that keeps an archive of data hopefully by the minute.

I just suggested the saving the raw telnet data as a guess of something that might be possible with your limited time.  If there is something easier please go with that just anything to keep a historic minute by minute log of line stats just in case there is a problem so it can be used for diagnostics with our isps would be brilliant.

The saving of MDWS data to a local folder sounds like it would be ideal (would need to include data only gathered at sync or per hour not just the per minute stuff) and as it would probably be more usefully formatted than raw telnet data this sounds like the better option.
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 05, 2018, 08:13:46 PM
Quote
I could also upload this data to a publicly accessible site if someone is prepared to set this up and administer it. This would of course represent a substantial commitment by someone.

I don't want to cause you additional work, but a thought on that.   
Whoever takes it over is going to be responsible for a substantial amount of bandwidth for everyone stats and take care of GDPR.

Is there anyone prepared to make some standardised GUI's for the graphs  (I realise DSLstats has some basic ones) based on the data that is normally uploaded to MDWS

THEN making each person responsible for their own data upload to either a site of their choice or on the local machine (in a similar way to what you can with DSLstats atm.)

I haven't really played with the web-server already in DSLstats, all I have done is a while ago I got it mapping SNRM on the local machine.    I perhaps need to revisit the inbuilt web server again, but a couple of things that I seem to recall:-

- The pages such as snrm.html, crc.html etc could perhaps do within being within their own subfolder rather than at the root.
- Option to designate a specific url for the graphing rather than IP    eg kitz.co.uk/linestats/snrm.html   (I couldn't seem to find a way to change this).


Perhaps I'm speaking out of turn here and I'm not sure how viable the next part is, but John from d2d4j offers cheap hosting to kitz forum members. 
Is there a package that he could put together for displaying DSLstat graphs.    Obviously it must be sufficient to cover his own running costs, but by making each member responsible for their own hosting then that completely does away with the GDPR requirements.   Apols to John if this isnt viable or not possible.

Title: Re: DSLstats after the demise of MDWS
Post by: V_R on March 05, 2018, 08:21:47 PM
Is there any way one could run the site locally and have DSLStats 'upload' to the local host, thus having the same access to the graphing and stats etc, juts obviously it would only be your own stats?

That too me would be the simplest option, if its doable that is. At least that way those who what to continue graphing can?

I would take over the site in a heartbeat if I had the time and knowledge to maintain it.
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 05, 2018, 09:51:48 PM
The first part of your second suggestion (above) would be of interest to me, please.
And to me too, please.
Title: Re: DSLstats after the demise of MDWS
Post by: NewtronStar on March 05, 2018, 10:25:58 PM
You would think the world has ended because MDWS is going offline, You can get all the info that you need on your circuit from DSLstats when using your PC/RPI 24/7 to gather stats and graphs have set it up see 14 days worth of stats it shows all the normal graphs you really need.
Title: Re: DSLstats after the demise of MDWS
Post by: jelv on March 05, 2018, 11:00:21 PM
Would it be possible to be able to specify a directory path for the web server files without running the web server?

I have a NAS box running a web server and can map a directory to a drive  (or use a \\servername\directory path) which would allow the DSLstats web pages to be viewed. I'm sure many other people already have web servers which could be used if DSLstats could write to the correct place.
Title: Re: DSLstats after the demise of MDWS
Post by: jaydub on March 06, 2018, 12:09:26 AM
My issue with DSLStats is that I am running it on an iMac using Nallar's Docker version.

When I look at where the snapshots are held it gives the location as /root/.dslstats/, which presumably is where the files will be stored under Linux.

However this doesn't relate to an identified location on teh Mac.  Is it possible to access these files?
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 06, 2018, 12:21:39 AM
On a RPi (Raspbian Linux), there is a button next to the 'Snapshot directory' box, so you can change the location to somewhere else if you wish.

The initial period in the name of the directory .dslstats means it is a hidden directory. Are you able to view hidden directories?
Title: Re: DSLstats after the demise of MDWS
Post by: jaydub on March 06, 2018, 12:33:00 AM
On a RPi (Raspbian Linux), there is a button next to the 'Snapshot directory', so you can change the location to somewhere else if you wish.

The initial period in the name of the directory .dslstats means it is a hidden directory. Are you able to view hidden directories?

I think the problem is that the directory locations on offer are the Linux choices and IO can't point it to something meaningful from a Mac perspective.

I have also just tried to set up the web server configuration and can't get any joy in terms of being able to access the stats.  I am running a Netgear Orbi so have the PC on the Orbi LAN as 10.0.0.2 and have separate network configuration set up with the iMac as 192.168.1.100 for picking up the sats from the modem.  Trying to access either 192.168.1.100:55555/snrm.html or 10.0.0.2:55555/snrm.html just gives me a site not reached message in Chrome.

Am I missing something or is this just an issue with running DSLStats under Docker?
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 06, 2018, 01:18:20 AM
I know nothing about Docker, and little about the Mac, so I am unable to help further. Sorry.  :no:

Perhaps someone more knowledgeable will be along soon.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 06, 2018, 07:30:55 AM
@jaydub: I'm afraid I'm in the same position as Westie, so this one is going to have to be shelved for the present, while I work through the more widely needed options.

@jelv: Yes, I can add an option to specify the location of the webserver files. I don't know if the directory selector dialog will be able to link directly to the NAS box, so you may have to map it to a local directory as you mentioned.

To everyone else: I'm going to have to do this in stages, and I haven't digested everything yet, but I'm fairly sure that my first job will be to save the MDWS zips locally.
Title: Re: DSLstats after the demise of MDWS
Post by: d2d4j on March 06, 2018, 07:57:49 AM
Hi

I’m sorry for lateness, just getting over a chest infection sorry

@kitz, thank you for thinking about me and yes, would certainly look at your suggestion for those who would want to use it

Many thanks

John
Title: Re: DSLstats after the demise of MDWS
Post by: KingJ on March 06, 2018, 09:49:12 AM
My issue with DSLStats is that I am running it on an iMac using Nallar's Docker version.

When I look at where the snapshots are held it gives the location as /root/.dslstats/, which presumably is where the files will be stored under Linux.

However this doesn't relate to an identified location on teh Mac.  Is it possible to access these files?

When you first ran the container, you should have specified a -v flag. In Docker, this mounts an external directory path from your host inside the container. The setup instructions specify that you should mount an external folder on to the /config directory internally - so anything inside the container that's written to /config will be persisted and accessible on the mapped folder on your external host (in this case, your iMac).

So, to persist and view the snapshots you've got two options;

Title: Re: DSLstats after the demise of MDWS
Post by: andyfitter on March 06, 2018, 11:06:19 AM
I think the problem is that the directory locations on offer are the Linux choices and IO can't point it to something meaningful from a Mac perspective.

I have also just tried to set up the web server configuration and can't get any joy in terms of being able to access the stats.  I am running a Netgear Orbi so have the PC on the Orbi LAN as 10.0.0.2 and have separate network configuration set up with the iMac as 192.168.1.100 for picking up the sats from the modem.  Trying to access either 192.168.1.100:55555/snrm.html or 10.0.0.2:55555/snrm.html just gives me a site not reached message in Chrome.

Am I missing something or is this just an issue with running DSLStats under Docker?

Aren’t there problems with the web server under Docker, mentioned on the github page? I believe it runs its own web server on port 8080
Title: Re: DSLstats after the demise of MDWS
Post by: les-70 on March 06, 2018, 11:39:29 AM
   Might it be possible to extend the time per page on DSLstats graphs to more than 24 hours?   Up to 7 days or even a month would give more MDWS like visibility of things if you don't look at things every day.  If a week is possible then an option to save snapshots every 7 days would be nice.  If a month is possible I think the monthly option is already there.   This would also make looking a things back in time via the saved snapshots a much easier task.

   I have no idea how easy or hard that might be.    I doubt that it would matter if for graphs longer than 24 hours days the sample times needed to have a bigger minimum value. 
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 06, 2018, 11:56:48 AM

Is there anyone prepared to make some standardised GUI's for the graphs  (I realise DSLstats has some basic ones) based on the data that is normally uploaded to MDWS
Are the current MDWS graphs created by Google Charts? https://developers.google.com/chart/
Title: Re: DSLstats after the demise of MDWS
Post by: d2d4j on March 06, 2018, 11:58:13 AM
Hi dray

That was my understanding re google charts

Many thanks

John
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 06, 2018, 12:48:02 PM
Are the current MDWS graphs created by Google Charts? https://developers.google.com/chart/

Yes I was looking at google charts last night. In fact I was actually looking at their Line charts here (https://google-developers.appspot.com/chart/interactive/docs/gallery/linechart).

But its not quite so simple as having the charts, you'd also need something which would accept the data from DSLstats, update the data then send it to be able to graph it to the chart.
If you were running it on your own website, you'd need to update and retain the data.    I'm not sure how its currently done, but last night I was looking at the info sent from kitz_live_modem_stats.log.

It's currently sent as space delimited file, so first step would be to break down into an array for each item, something like the function explode should do that.
Code: [Select]
$file = "kitz_live_modem_stats.log";
$stats = file_get_contents($file);
$arr_stats = explode(" ", $stats);
output would be something like

Code: [Select]
print_r($arr_stats);

Array
(
    [0] => 06/03/2018
    [1] => 12:20
    [2] => 72629
    [3] => 20000
    [4] => 6.2
    [5] => 6.8
    [6] => 6.8
    [7] => 72641
    [8] => 28686
    [9] => PTM
    [10] => 0.00
    [11] => 0.00
    [12] => 0
    etc
)

You'd then need to identify what each stat in the array is and name it, so you can identify which piece of data you want for the particular graph you want to display....  and that's just the easy bit.

There's lots of other areas you need to think about.
 - How is the data automatically going to be uploaded to your webserver
 - How are you going to keep a permanent record of all the stats so that each new incoming data is added to the record
 - Code for sending an array to the graph...  there's some code here which may help (http://bl.ocks.org/espinielli/8d106a5cf9aa99734d43)
 - You'd need .html templates to output the graph to.

Unfortunately this isn't going to be a 2 minute job just because there is such a thing as google charts  :(
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 06, 2018, 01:00:30 PM
Although GDPR is one of the reasons Tony cant do this any more, its not the only reason.

Quote
The server is also showing signs of overload recently with a rapidly increasing user base and would likely need a rewrite to get over this.

Think about it, his database is currently holding data for many hundreds of users.
For each and everyone of those users there is rows upon rows of data being updated every minute for numerous types of graphs.

He had a pretty decent spec server running in his own home, so if there were any probs he could quickly fix his code and have control over his own db.

Whilst a single user graphs would probably cope perfectly well on a shared server and fairly basic type hosting, when you start having to deal with data from hundreds of people then you need something a lot more powerful running on its own system.    That doesn't come cheap and is going to require a huge commitment from anyone wanting to take it over :(

 
Title: Re: DSLstats after the demise of MDWS
Post by: jelv on March 06, 2018, 01:13:48 PM
@jelv: Yes, I can add an option to specify the location of the webserver files. I don't know if the directory selector dialog will be able to link directly to the NAS box, so you may have to map it to a local directory as you mentioned.

Does that include changing the current check box "Enable webserver" to have three options (Off, Create pages only, Create pages and enable webserver)? The port would only be needed for the last option.

I've tried changing the snapshot directory to \\nasbox\sharename\DSLStats and it worked - and I can view the contents of the webserver directory remotely via my existing webserver port forwarding (the problem with using this method this is that it also puts the snapshots directories on the webserver).

One thing I didn't point out in my previous post is point out the advantage that being able to write the webserver files to my existing webserver means I don't have to set up an additional port forward for 55555.
Title: Re: DSLstats after the demise of MDWS
Post by: broadstairs on March 06, 2018, 01:15:18 PM
To be honest given Eric's situation with his time available I think the best answer is for some form of file saved as perhaps a zip file containing the data currently sent to MDWS. As long as the format is known then it would be easy for individuals to decide whether or not it is worth it for them to create some kind of database to contain it or just to archive it in case it is needed. As has already been noted the saved graphs etc can contain pretty much all the data needed. I have started archiving this off to my NAS pending something more compact being available.

Stuart
Title: Re: DSLstats after the demise of MDWS
Post by: Ronski on March 06, 2018, 01:29:23 PM
With hg612 stats it saves all the stats to one log file, although I do split them off every year, so the stats data is always there. Theres also the graphs but unless manually run are only for  a 1 day period.

My brother might be tempted to write something for better on the fly graphing, but given his total lack of time I'm not holding out much hope.
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 06, 2018, 02:11:31 PM


   Might it be possible to extend the time per page on DSLstats graphs to more than 24 hours?   Up to 7 days or even a month would give more MDWS like visibility of things if you don't look at things every day.  If a week is possible then an option to save snapshots every 7 days would be nice.  If a month is possible I think the monthly option is already there.   This would also make looking a things back in time via the saved snapshots a much easier task.
I too would really love it if the graphs could cover more than 24h (7d would probably fit on the existing size of graph not sure whether 30d would be a bit of a squeeze) but roseway has already stated his time is limited so I didn't want to push my luck above what I already suggested ;)

Think most graphs are a rolling 24h (ES isnt though for me).  Having the ability to increase the timeframe covered for all charts would reduce the need for as many snapshots.

Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 06, 2018, 02:46:44 PM
@jelv: I've put it on the todo list, but the list is getting longer...
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 06, 2018, 02:48:56 PM
@les-70: also on the list.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 06, 2018, 04:34:58 PM
It's currently sent as space delimited file, so first step would be to break down into an array for each item, ...

I could make it a CSV file, but DSLstats has some users in countries where they use a comma as a decimal point. The other zipped files for HLog, QLN, etc are in CSV format because all the values are whole numbers.
Title: Re: DSLstats after the demise of MDWS
Post by: g3uiss on March 06, 2018, 06:10:45 PM
Hi All

Just in case I’m always willing to help with the GDPR.

I always used HG612 stats, but will now change to DSLstats.

From what I read going forward  the two differences really are only the loss of the ability to see a large range of users stats, and it’s availability from any device anywhere.

The first issue isn’t easily overcome but I’m sure KITZ could create an area here for members to post any graphs they require input or advice on from DSL stats, and the second exists to view data locally on a network via a browser already.

All the work is much appreciated I can help if necessary as well with server space and SMTP servers for non ISP specific email, as I did for some time for MDWS.

Tony
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 06, 2018, 07:09:34 PM
I could make it a CSV file, but DSLstats has some users in countries where they use a comma as a decimal point.

I don't think it makes much difference Eric, you'd still need a function to read the file and break it down into an array.
Thus explode for space delimited files or something like fgetcsv for csv files.   The end result is the same. 
Maybe its just me but I find explode easier for files which dont contain text strings that contain spaces.
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 06, 2018, 07:22:12 PM
The first issue isn’t easily overcome but I’m sure KITZ could create an area here for members to post any graphs they require input or advice on from DSL stats, and the second exists to view data locally on a network via a browser already.

I already allow attachments and with all due respect I think a lot of people fail to appreciate that I have my own running costs.   :(

Not wanting to moan but one of the largest parts of my storage is already graphs from stats as it is and as I mentioned on the main site earlier this year I am going to have to buy additional storage myself this year.   I'm one of the few places that allows them.  Whilst it may only be £10+VAT more per month, my hosting costs have already increased this year due to inflation.

My income from advertising has been a zilch for the past 3 months due to ad-blockers.  Not a single penny this year!   ISP clicks have totally dried up because of cash back sites.    I get very few donations and if it wasn't for a couple of people who do support then to be quite frank I'd be up the creek without a paddle long before now.       
Title: Re: DSLstats after the demise of MDWS
Post by: g3uiss on March 06, 2018, 07:40:19 PM
I already allow attachments and with all due respect I think a lot of people fail to appreciate that I have my own running costs.   :(
 

Sorry didn’t want to suggest anything that would increase your costs or admin. It was just a thought to address the concerns users seem to have. For me I’m just interested in seeing my stats periodically.

I know you may have concerns about GDPR (which if you need I’m happy to help) but if users want a facility perhaps some donation for use of certain areas appropriate. You do great work maintaining this site. Great stuff can’t always be free.

Again sorry for any concern my post created

Tony
Title: Re: DSLstats after the demise of MDWS
Post by: d2d4j on March 06, 2018, 08:27:27 PM
Hi

I was thinking about this, and from what I can understand, part of the issue is been able to view graphs publicly, including other users graphs

I know it’s not the same, but I was thinking perhaps about a gallery to allow registered users to upload their graphs.

Please would anyone be interested in this and if so, what would you be prepared to pay per year

This would give you full gallery access from anywhere with broadband on any device (including mobiles iPads etc through an app or browser)

Many thanks

John
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 06, 2018, 08:48:23 PM
Quote
I was thinking about this, and from what I can understand, part of the issue is been able to view graphs publicly, including other users graphs

I believe so.   Hence my suggestion for making each user responsible for their own graphs and no problems if they want to publicly share with others.

Quote
I was thinking perhaps about a gallery to allow registered users to upload their graphs.
If the graphs are images, then file sizes and storage could soon mount up.
Using raw data from DSLstats and then something like google charts to draw the graphs would keep storage to a minimum.

Alternatively there's also this (http://www.jamieidavies.co.uk/dslstats/webserver/allstats.html) which JID did a couple of years ago.    In its current format its only doing live graphing though and I'm not sure about historic info. 
Title: Re: DSLstats after the demise of MDWS
Post by: d2d4j on March 06, 2018, 09:40:17 PM
Hi

@kitz thank you - yes your totally correct sorry

In that case, for a simple quick solution, I could create a single domain with sub domains, for users to ftp the webserver file into their folder for subdomain and then visible from anywhere

Their time frequency would be up to themselves to decide (minute, 5 minutes hourly etc) although it would not keep historic data so guess it might be simplistic not to do it as users would use dslstats webserver

The current users of our hosting could just do this anyway, as they have full hosting capabilities as well as email

It’s a hard one and just shows the excellent work tony did

Many thanks

John
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 06, 2018, 10:51:15 PM
My plan is that the data which DSLstats saves (and perhaps uploads somewhere) will have a filename which includes the date and time, so history will be preserved for as long as the user wants and the store has capacity. If all the available data is saved every minute, this will amount to about 4 GB a year.
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 07, 2018, 12:05:06 AM
Quote
about 4 GB a year.

That's a bit larger than I anticipated.   :(
I have a HG612 modem stats log from 25/07/2016 which is 660MB

But when I look at the data contained, it appears to be only basic line stats.   I guess its bits/SNR per tone QLN/hlog etc type data which will increase file size.

Code: [Select]
25/07/2016 20:13 79987 20000 7.0 12.5 5.9 5.9 83805 30404 PTM 850 18 0 390 731 0 218562 0 0 0 0 60.712 0 390 0 132124761 76362 0 850 18 0 0 3400835 0 0 0 0 1 1 122 62 12 0 0 0 0 0 0 0 0.7 18.5 28.1 0.0 11.0 24.5 38.5 0.6 18.3 28.1 0.0 10.7 24.4 38.5 14.2 13.1 12.3 0.0 7.0 7.0 7.0 -6.0 -25.6 5.6 0.0 12.0 7.4 7.4 Showtime 0 0 L0 17a ON ON No Showtime 0.0 0.0 239 236 1 1 23 5 0 16 0.0955 0.3771 20104 5410 30 0 30 0 3547761395 0 29704784 0 0 0 0 59 59 0.00 0.00 1.65 6.15 0.00 0.00 116.09 202.87 588 12 VDSL2 0 0 0 0 0.00 0.00 0.00 0.00 0 0 1 0 0 0 7 0 0 0 0 0 0 0 NO_G.INP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00 0.00 0.00 0.00 0.00 0.00
Title: Re: DSLstats after the demise of MDWS
Post by: burakkucat on March 07, 2018, 01:05:42 AM
. . . when I look at the data contained, it appears to be only basic line stats.

Yes, I can confirm that the data is that harvested from "xdslcmd info --stats" and "xdslcmd info --pbParams" commands, along with selected deltas.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 07, 2018, 07:22:50 AM
As suggested above, the considerable majority of that 4 GB is in the per-tone data, not the main stats. What Tony did to cope with this is to have the per-tone data uploaded only once per hour. If I stick to this arrangement, the annual total will be a lot less.
Title: Re: DSLstats after the demise of MDWS
Post by: skyeci on March 08, 2018, 10:42:05 AM
I know its slightly off topic but with MDWS closing will there be any way we can somewhere find other TBB charts. Its useful to see other connections latency etc. I know not everyone possibly uses this feature but be good if we can somehow see other charts that were linked from mdws...?

Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 08, 2018, 11:01:18 AM
Sorry, but I'm not going to be able to fit that in.
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 08, 2018, 12:11:44 PM
Just in case I’m always willing to help with the GDPR.

Thank you.    I'm currently trying to wade though the minefield myself to aid better understanding, but there is one area I'm still hazy about.   There's a couple of things I do need to check first, but if you don't mind I may at some time pick your brains if I get stuck.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 08, 2018, 03:12:26 PM
Another option for me is to extend the "delete snapshots after X days" to indefinite.
That way allow the user to store as much historical data as they please.
I've not looked into the MWDS folder as of yet due to my own time constraints, but I assume it's an a format that is excellable...

It's already possible to configure snapshots to never be deleted. Simply untick the option "Delete snapshots older than".

The format for the MDWS uploads is CSV for the four tone related sets (SNR per tone, QLN, HLog and Bitloading) and space separated for the regular stats.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 08, 2018, 03:15:27 PM
   Might it be possible to extend the time per page on DSLstats graphs to more than 24 hours?   Up to 7 days or even a month would give more MDWS like visibility of things if you don't look at things every day.  If a week is possible then an option to save snapshots every 7 days would be nice.  If a month is possible I think the monthly option is already there.   This would also make looking a things back in time via the saved snapshots a much easier task.

   I have no idea how easy or hard that might be.    I doubt that it would matter if for graphs longer than 24 hours days the sample times needed to have a bigger minimum value. 

I can certainly do this - the current limits are fairly arbitrary anyway. But I'll need to make sure that increased values don't have any unexpected side effects, so this will have to join the todo list.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 08, 2018, 03:51:27 PM
The current state of the development

I've stripped out the MDWS upload capability and changed the handling of the MDWS data so it's saved locally. At present the main data is saved after every sample and the zipped per-tone data is saved once an hour. The files have all got a time stamp in their filenames and they're saved in daily folders with names like 2018-03-08 (yyyy-mm-dd).

With the sample time set at one minute, the amount of data stored per day is about 900 kB for a VDSL2 profile 17a connection. In a full day there will be 1440 files of main stats and 96 files of per-tone data in the daily folder. I'm assuming that the number of files in each folder isn't a problem on any of the supported operating systems, but could someone please let me know if I'm wrong?

The next stage is to add some user control of the frequency with which the data is saved, and I would be interested in views on that.
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 08, 2018, 04:29:25 PM
At the moment (v6.1.3) there is the option to save snapshots in daily folders or category folders.

I find the latter more useful, as it is then easier to compare how a given parameter has changed over time, because all the time-stamped snapshots for a particular category are collated together. Since I am only saving 24-hour snapshots once per day I find it a great way to get an overview.

I can understand that data saved after each sample might be best stored in daily folders, but if the ability to save daily snapshots by category is retained, I think it's easier to identify which day & category should be examined at raw data level.

Just my personal opinion...
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 08, 2018, 04:53:36 PM
Roseway I would try to keep things simple at first.

Keep the new raw data snapshots separate from the current snapshots (but use /raw/yyyymmdd and /graph/yyyymmdd folders) so retention days can be configured separately.  E.g
 I may have 28d for the current graph like snapshots but I may want 365d of the new minute by minute raw data.

Personally the raw data is such a small amount of disk space per day I would just always do it per minute and the size can be set by the number of days kept.

Anyway I am happy just to have something after mdws so these are just my personal preferences I am sure I will be happy with whatever is decided.

Cheers and thanks again for spending time enhancing your code to partially fill the gap.
Title: Re: DSLstats after the demise of MDWS
Post by: Ronski on March 08, 2018, 08:16:37 PM
I'm assuming that the number of files in each folder isn't a problem on any of the supported operating systems, but could someone please let me know if I'm wrong?

Not a problem, but on some systems 900kB of data in 1440 files may well take up far more space than 900kB, I believe it depends on the sector size of the disc.
Title: Re: DSLstats after the demise of MDWS
Post by: jelv on March 08, 2018, 09:47:10 PM
The Windows default cluster size on NTFS is 4K so 1440 files will take a minimum of 5760K. That works out to over 2GB for a year's worth of files!
Title: Re: DSLstats after the demise of MDWS
Post by: g3uiss on March 08, 2018, 10:40:31 PM
Thank you.    I'm currently trying to wade though the minefield myself to aid better understanding, but there is one area I'm still hazy about.   There's a couple of things I do need to check first, but if you don't mind I may at some time pick your brains if I get stuck.

Fine. Just let me know what clarifications you need.
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 10:50:59 AM
Yes I was looking at google charts last night. In fact I was actually looking at their Line charts here (https://google-developers.appspot.com/chart/interactive/docs/gallery/linechart).

But its not quite so simple as having the charts, you'd also need something which would accept the data from DSLstats, update the data then send it to be able to graph it to the chart.
Unfortunately this isn't going to be a 2 minute job just because there is such a thing as google charts  :(
Yes but as what we want it a way of generating charts and Google Charts is already being used to do this, then Google charts seems the most appropriate way of producing charts for this replacement project.

As I understand it, DSLstats extracts the data on a regular interval and stores it on my PC. It then uploads the data to the MDWS website.

MDWS now accumulates the data and stores it in a way it can be easily charted. This involves asking Google Charts to chart this specified data and output a Chart which is then displayed to the user.

Within this process the data items have already been named by DSLstats.
Title: Re: DSLstats after the demise of MDWS
Post by: d2d4j on March 09, 2018, 11:08:27 AM
Hi

@dray - sorry I thought the naming was completed by tony mdws

I could be wrong sorry but it’s easy to check by looking at the files that are uploaded

Many thanks

John
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 11:15:02 AM
Yes it would be nice to see a list
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 11:15:45 AM
Youre correct John, its just raw data.
There's  a sample of raw data in my post here (http://forum.kitz.co.uk/index.php/topic,21159.msg367060.html#msg367060).

There's a lot of code needed to get from that to sending the info to google charts.  As I said in my earlier post there's a lot of other areas that need to be thought about ie


It's currently sent as space delimited file, so first step would be to break down into an array for each item,
You'd then need to identify what each stat in the array is and name it, so you can identify which piece of data you want for the particular graph you want to display....  and that's just the easy bit.

There's lots of other areas you need to think about.
 - How is the data automatically going to be uploaded to your webserver
 - How are you going to keep a permanent record of all the stats so that each new incoming data is added to the record
 - Code for sending an array to the graph...  there's some code here which may help
 - You'd need .html templates to output the graph to.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 09, 2018, 11:17:32 AM
Quote
Within this process the data items have already been named by DSLstats.

The main stats as uploaded to MDWS are simply a space delimited list of values. They're not in any logical order for historical reasons, so I'll be providing a conversion chart to relate element numbers to what they represent.
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 11:21:15 AM
Sounds good :)
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 11:44:08 AM
@dray..  but that will only be a reference for who ever does write the code.    The data is always going to have to be sent and stored as either space delimited or csv files otherwise they become very large.
You still then have to approach all the other steps I outlined.  :(
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 01:42:16 PM
Hi Kitz,

Just to echo what you asked earlier - can anyone provide a list of steps that can be used by MDWS to create a chart, to keep it simple lets say the SRNM dB chart?
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 02:09:04 PM
I know the steps although it may take just a while to type them all out if you mean the actual processes and functions required. (and no I dont mean the code)
I have covered the basic steps already.

The hard part and time consuming part is the bit in the middle between having the raw data and then being able to use that data to output to the graph.  As I said earlier outputting the data to the graph is then easy.

 
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 02:26:21 PM
Definitely not the code. And I'm not trying to find work for you to do neither.

Currently we now have the data stored like this: (although I haven't seen it)

I've stripped out the MDWS upload capability and changed the handling of the MDWS data so it's saved locally. At present the main data is saved after every sample and the zipped per-tone data is saved once an hour. The files have all got a time stamp in their filenames and they're saved in daily folders with names like 2018-03-08 (yyyy-mm-dd).

Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 02:34:13 PM
Currently we now have the data stored like this: (although I haven't seen it)

I gave a sample of what one row of data contains a few days ago..  and linked to it again this morning :(

Admittedly I got my sample from HG612-stats because that is what I was using to upload to MDWS , but the data will be practically the same whether its coming from DSLstats or HG612stats.

This is just basic line stats.   There's a lot more to cover things like hlog etc.


Code: [Select]
25/07/2016 20:13 79987 20000 7.0 12.5 5.9 5.9 83805 30404 PTM 850 18 0 390 731 0 218562 0 0 0 0 60.712 0 390 0 132124761 76362 0 850 18 0 0 3400835 0 0 0 0 1 1 122 62 12 0 0 0 0 0 0 0 0.7 18.5 28.1 0.0 11.0 24.5 38.5 0.6 18.3 28.1 0.0 10.7 24.4 38.5 14.2 13.1 12.3 0.0 7.0 7.0 7.0 -6.0 -25.6 5.6 0.0 12.0 7.4 7.4 Showtime 0 0 L0 17a ON ON No Showtime 0.0 0.0 239 236 1 1 23 5 0 16 0.0955 0.3771 20104 5410 30 0 30 0 3547761395 0 29704784 0 0 0 0 59 59 0.00 0.00 1.65 6.15 0.00 0.00 116.09 202.87 588 12 VDSL2 0 0 0 0 0.00 0.00 0.00 0.00 0 0 1 0 0 0 7 0 0 0 0 0 0 0 NO_G.INP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00 0.00 0.00 0.00 0.00 0.00
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 02:39:19 PM
Dunno how I missed that. Sorry  :-[
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 02:50:42 PM
can anyone provide a list of steps that can be used by MDWS to create a chart, to keep it simple lets say the SRNM dB chart?




There's pro's and cons' for both methods. Setting up the (SQL) database in itself will take some time to create all the tables you need.
What a approach you take could also depend on whether you are using a webserver running on your local PC OR if you are going to be putting the scripts on your website. 
If you have a web-server running on your local PC it gives you a lot more flexibility, but google graphs will I believe only run live because they are parsed by your browser.   
I can understand why Tony needed to run this locally using his own equipment because in some respects it makes some things a bit easier to control. 


-----------------------   

Then move on to the next steps, these are just off the top of my head and there will probably be a lot more functions to code, but I think I've covered the main ones:-




Now create numerous other scripts which will do the same for atten, sync speed etc.   
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 03:01:54 PM
Thanks for that. I can't see any advantage in using SQL so text files will be used. I'll currently ignore that last point to remove the pressure point ;)
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 09, 2018, 03:57:15 PM
I can.   If they are stored in an SQL database then it will be much faster and easier to when it comes to producing the graphs.

Over time the text files will become large and have a lot of data to sort through to just find the SNRM values.  Look at that one row of data I posted and you will see how many values are in the text file...  but you are only searching for one figure out of that whole row.    If stored as text then your program will have to loop through 100's of 1000's of data to find the one figure in each row that it is looking for.

If you create a database that stores all the SNR values in one table, then you can do a direct query on that table and ask it to pull all the data between 'x & y' date and then send those figures to the google chart code.   More code at the beginning to set up all the tables and to create a script which will add the incoming data from DSLstats to your database.   But the graph execution and webpage load will be a heck of a lot quicker.

I don't know if you've ever used HG612stats GUI to produce graphs, but it can take a few mins to generate the graphs, that's because all the data is stored as text files.   Even on a fast PC it can still take a while..  it wouldn't be the best approach if you're running it from a remote web-site.

You need to sit down and do some serious thinking how you are going to be outputting the data... and which method is going to be best depending on how you are going to output that data.  What may work best on a machine running a local web-server, or something to just generate static graphs is going to have different requirements to running it on a remote website.  It also depends on how often you think you are going to view them. 

---

There is really an awful lot of thought that has gone behind what Tony did, its not something that will have taken him a couple of days.   Unless you have to do this sort of thing, then people don't realise how much work goes into being able to present the data.     

To give an example...  my broadband checker gets all the data from the BTw database I do a simple query and it returns all the raw data in xml format.    I immediately have the info that is needed.   
The bit that took me weeks/months to code is being able to break down the raw code and then output the data so that its readable to humans and looks pretty.   With MDWS, google graphs is doing the pretty bit, but there is still an awful lot of code in the middle that will be needed to make sure that the graphs get the right info they need. :(
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 09, 2018, 05:14:34 PM
I guess everyone is paying particular attention of what bits they use the most in MDWS.

I know roseway has a list over 10 miles long now but from things that haven't been mentioned yet, the traffic light overview is something I look at frequently as it is a quick way of checking status of the line.  I also hope the ES graph gets a rolling 24h like most of the other grqphs.

I am trying to remember to use dslstats web server instead of mdws ready for Monday :(
Title: Re: DSLstats after the demise of MDWS
Post by: Dray on March 09, 2018, 05:43:41 PM
I should just point out that I am not going to be developing a replacement for MDWS.

My ability ran out while getting the  DSLstats webserver to produce a useful page of graphs
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 10, 2018, 12:28:29 AM
I also hope the ES graph gets a rolling 24h like most of the other grqphs.

I think that is one of the things that a lot of us are going to miss the most because of the way DLM works. 
Err/Secs and to a lesser extent SNRM are about the only thing I really look at on my line.   

Eric has got a lot to do as it is and I'm not sure how hard it would be to add this.   I am a bit concerned that the requests are piling up for him.  :-\
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 10, 2018, 12:37:39 AM
I should just point out that I am not going to be developing a replacement for MDWS.

My ability ran out while getting the  DSLstats webserver to produce a useful page of graphs

I'm bowing out of this convo now as I feel its just distracting this thread from what Eric can do to do with the DSLstats data output. 

Anyone who has the ability will likely know what is going to be involved and I've already put far more time into explaining things than was my original intention as I covered the basic steps a few days ago.  Just to re-iterate, its your browser that renders the google graphs and its easy to put their code into a html web page.  The hard part is getting the data, storing the data, then being able to call it when you want to be able to pass it to the graphs.   
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 10, 2018, 07:23:47 AM
Quote
I also hope the ES graph gets a rolling 24h like most of the other grqphs.

What MDWS does is put in rolling graphical form the data which DSLstats presents as a daily bar chart. It's the same data, and DSLstats retains 100 days worth. All of this data can be viewed by clicking on the "... days ago" control.

If you really want a per-minute display of ES I could present it in the same form as CRC, FEC, etc. But with the jobs piling up, it's not going to be quick.
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 10, 2018, 09:14:11 AM
If you really want a per-minute display of ES I could present it in the same form as CRC, FEC, etc. But with the jobs piling up, it's not going to be quick.

Thanks for replying roseway.

My reasoning was when you are looking at the stats via the web browser if you are looking at the start of the day there is not much information as it resets at midnight so if you look at 0600hr say before you go to work there is only hours 00, 01, 02, 03, 04, 05 visible.  If you had an ES spike at 23 or 18 you cant see it at least via the browser.

I just thought it would be useful if people posted how they used MDWS and then you can decide what is worth your very limited time. 

Thanks for dslstats I get you can't do everything I appreciate you are trying to enhance to fill the gap as much as feasible possible with the time you have available.
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 10, 2018, 12:27:56 PM
In DSLstats, if you look at the ES/hour for the previous day, you will see any spikes that occurred before midnight. As I understand it the DLM resets at the end of the day anyway, so a spike the previous evening will either already have had an effect, or won't affect the current day anyway.

With the demise of MDWS, I think it's important for each of us to look for alternative ways to get the information we would like from existing resources as far as possible, rather than just adding to the list of jobs Roseway already has.
Title: Re: DSLstats after the demise of MDWS
Post by: j0hn on March 10, 2018, 04:18:23 PM
Quote
As I understand it the DLM resets at the end of the day anyway, so a spike the previous evening will either already have had an effect, or won't affect the current day anyway.

It works on a rolling 24 hour, not midnight to midnight.
Title: Re: DSLstats after the demise of MDWS
Post by: Westie on March 10, 2018, 07:14:36 PM
Thanks for the correction. :)
Title: Re: DSLstats after the demise of MDWS
Post by: Chrysalis on March 11, 2018, 10:58:15 AM
eric i will be sending you a PM this week about a MDWS replacement as you dev dslstats i obviously would need you onboard

probably to get a sample of a data dump so a compatible system can be developed, this is to minimise work your side.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 11, 2018, 11:32:32 AM
Chrys: I'll soon be uploading an experimental version of DSLstats which saves all the data in the same format as the MDWS uploads, but retains them in a local store. It should be fairly straightforward to upload this data somewhere, although I would very much prefer to use FTP for the upload rather than the complicated curl command required by MDWS.
Title: Re: DSLstats after the demise of MDWS
Post by: Chrysalis on March 11, 2018, 01:01:46 PM
FTP wont be a problem, I will wait for the new build so I can check the format, thanks.
Title: Re: DSLstats after the demise of MDWS
Post by: j0hn on March 11, 2018, 03:04:37 PM
FTP wont be a problem, I will wait for the new build so I can check the format, thanks.

It was posted above.

Code: [Select]
25/07/2016 20:13 79987 20000 7.0 12.5 5.9 5.9 83805 30404 PTM 850 18 0 390 731 0 218562 0 0 0 0 60.712 0 390 0 132124761 76362 0 850 18 0 0 3400835 0 0 0 0 1 1 122 62 12 0 0 0 0 0 0 0 0.7 18.5 28.1 0.0 11.0 24.5 38.5 0.6 18.3 28.1 0.0 10.7 24.4 38.5 14.2 13.1 12.3 0.0 7.0 7.0 7.0 -6.0 -25.6 5.6 0.0 12.0 7.4 7.4 Showtime 0 0 L0 17a ON ON No Showtime 0.0 0.0 239 236 1 1 23 5 0 16 0.0955 0.3771 20104 5410 30 0 30 0 3547761395 0 29704784 0 0 0 0 59 59 0.00 0.00 1.65 6.15 0.00 0.00 116.09 202.87 588 12 VDSL2 0 0 0 0 0.00 0.00 0.00 0.00 0 0 1 0 0 0 7 0 0 0 0 0 0 0 NO_G.INP 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.00 0.00 0.00 0.00 0.00 0.00 0 0 0.00 0.00 0.00 0.00 0.00 0.00
Title: Re: DSLstats after the demise of MDWS
Post by: burakkucat on March 11, 2018, 05:06:41 PM
Regarding the UploadStats.ods file contents --

When time permits would it be possible to annotate each field with its type and size, please?

For example: 16-char string, 8-bit unsigned integer, 64-bit signed integer, float, etc.

[Edited to fix a typo.]
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 11, 2018, 06:31:07 PM
iirc Eric said earlier that he would provide a reference for the various fields.

Unless I've misunderstood what you mean,  I'm not sure if it would be good idea appending it to the actual file, as that would substantially increase the file size.
Title: Re: DSLstats after the demise of MDWS
Post by: burakkucat on March 11, 2018, 06:41:34 PM
iirc Eric said earlier that he would provide a reference for the various fields.

That is correct. The UploadStats.ods file.

Quote
Unless I've misunderstood what you mean,  . . .

I believe that you have misunderstood.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 11, 2018, 07:21:30 PM
Quote
When time permits would it be possible to annotate each field with its type and size, please?

All of the fields are strings with varying length. The whole stats file is one long string with spaces for delimiters. If you opt for a single stats file each day, the separate sets of stats are delimited by double spaces. The per-tone zipped files are also strings, with comma delimiters in this case (because all the values are whole numbers in string form).
Title: Re: DSLstats after the demise of MDWS
Post by: marjohn56 on March 11, 2018, 08:17:26 PM
Chrys: I'll soon be uploading an experimental version of DSLstats which saves all the data in the same format as the MDWS uploads, but retains them in a local store. It should be fairly straightforward to upload this data somewhere, although I would very much prefer to use FTP for the upload rather than the complicated curl command required by MDWS.

Thinking aloud....

How about XML and HTTP/S. HTTP/S will give the secure data connection needed and there are some great XML libraries about that are really quick.
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 11, 2018, 10:38:09 PM
Thanks for the suggestion, but there are limits to what I can do in the time I have available.
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 23, 2018, 04:39:02 PM
I realise time is limited but an idea for the future could you consider when generating snapshot also generating a JavaScript file containing a list for all the existing  snrm.PNG files in the snapshot directory.  Then in the webserver page you could have previous and next buttons that would use JavaScript to replace the "static" images with pngs stated in this javascript file.   This would allow the webserver users to benefit from the snapshots.  Obviously wouldn't just be limited to snrm - es, ses, CRC, speed would work the same way etc
Title: Re: DSLstats after the demise of MDWS
Post by: roseway on March 23, 2018, 04:44:18 PM
It's a nice idea, and I can put it on the list, but I know nothing about JavaScript unfortunately.
Title: Re: DSLstats after the demise of MDWS
Post by: kitz on March 23, 2018, 07:11:18 PM
I realise time is limited but an idea for the future could you consider when generating snapshot also generating a JavaScript file containing a list for all the existing  snrm.PNG files in the snapshot directory.  Then in the webserver page you could have previous and next buttons that would use JavaScript to replace the "static" images with pngs stated in this javascript file.   This would allow the webserver users to benefit from the snapshots.  Obviously wouldn't just be limited to snrm - es, ses, CRC, speed would work the same way etc

Would it work in javascript which is client side rather than a server side language?   
I was trying to think of an easy solution for listing all the files in the folder when I was playing with the history page.   You would need some way of generating a list of snrm.png files in the folder that would be up to date depending upon which files where there.     I was trying to avoid using a server-side language (such as php) to generate the list because not everyone may want to install php or WAMP/LAMP.   You can perhaps see with the history page on the advanced web interface, that I started dabbling with AJAX but I can't see how that would work if viewing via the Internet either   

Unless I'm missing something I don't see how getting DSLstats to produce a javascript file would work. :hmm:

I think it would need a small program running on the PC to generate a list of all current 'x'.png files in the folder. 
You could then perhaps then use something like JQuery to output the contents of list to a html page.
Ideally this program would output the list in something like JSON or XML format.    If you had a JSON file, then you'd be able to use javascript to parse the data to html.   
You'd still need to write the javacript/Jquery functions to be able to output the data though and these would need to be included in the source code of the html page. 

If there was something that listed the relevant .png files to a text or JSON file, then I'd be able to use JQuery to write the functions needed to output to a html page, but even so that would require dev time.
 
Without using a server side language (or a separate program to generate the list) I couldn't see a way this would work in the current format on the existing inbuilt webserver just using something which ran on the client side.
Title: Re: DSLstats after the demise of MDWS
Post by: ktz392837 on March 23, 2018, 09:06:38 PM
Kitz yes I was thinking it would rely on dslstats generating / distributing the JavaScript files with the relevant snapshot information when a snapshot is generated.  Then just tweaks to existing HTML to add in the relevant JavaScript. It would require dev time but it could be quite a bit of functionality for a minimal amount of dev time.

The idea has peaked my interest so I may see if I can cobble something together  but I suspect someone who knows JavaScript/HTML can get a template  running in a few hours I will still be working out how to specify a variable but I like a challenge.

Whatever the result it will either need dslstats changes or something else generating the files.  My hope would be if the JavaScript and HTML could be written that would help roseway significantly.

Anyway I don't even know if you can use JavaScript to change the location of an image so it may all be a pipe dream anyway!