Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on March 18, 2018, 01:42:41 PM

Title: DSLstats pre-release version 6.3.1
Post by: roseway on March 18, 2018, 01:42:41 PM
This version is derived from the experimental version 6.3 which I recently provided. It adds the following features:
http://dslstats.me.uk/downloads.html (http://dslstats.me.uk/downloads.html)

The data store option added in v6.3 is going to remain in that format unless there are strong reasons to change. The present format is compatible with the logs produced by HG612-Stats. For reference, the format of the main stats file is detailed in this spreadsheet:

http://dslstats.me.uk/files/UploadStats.ods (http://dslstats.me.uk/files/UploadStats.ods)

Files are now available (as from about 14:40). I forgot to upload them.  :-[
Title: Re: DSLstats pre-release version 6.3.1
Post by: tiffy on March 18, 2018, 03:56:13 PM
V.6.3.1 running on my RPi 3B, all new listed features visable and operational, no unwanted by-products noted to date.

I was struggling today with an issue on V.6.3 where the kitz webserver interface would not display Stats or Telnet data even though these were selected on the webserver setup page, all other displays data was visable, the event log displayed "not all webserver data available" (or text to that effect) at every sample attempt.
This issue has been cured by installing the new program revision whether coincidentally or otherwise !

I also had a previous issue with the kitz webserver interface not displaying any data, found this was due to having previously added a "snapshots" folder to /home/pi/.dslstats, configured with the "snapshots directory" option, your inclusion of the option to save "webserver files" to an area of choice should eliminate this possibility in future.

Many thanks for your continuing development work with DSLStats, very much appreciated.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 18, 2018, 04:26:35 PM
Thank you for that helpful report. I think that the changes made to choose a location for the webserver files very probably did have the unexpected side effect of fixing your issue. :)
Title: Re: DSLstats pre-release version 6.3.1
Post by: skyeci on March 18, 2018, 04:35:55 PM
Thanks Eric

All good on my win 10 pc

cheers
Title: Re: DSLstats pre-release version 6.3.1
Post by: burakkucat on March 18, 2018, 04:55:15 PM
Downloaded to, and executing at, The Cattery.

Nothing abnormal has been observed, so far.  :)
Title: Re: DSLstats pre-release version 6.3.1
Post by: banger on March 18, 2018, 08:07:37 PM
Looking good on my Win 10 PC.
Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 18, 2018, 09:50:33 PM
Thanks Eric,

Installed & running well on RPi Zero W.  :)
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 18, 2018, 10:56:38 PM
Thank you all.
Title: Re: DSLstats pre-release version 6.3.1
Post by: burakkucat on March 18, 2018, 11:20:52 PM
I would like to flag a very minor cosmetic inconsistency.

In the configuration, all errors are set to be monitored. Under the top level "Errors" tab, there are secondary tabs for "CRC Errors", "FEC Errors", "SES", "ES per hour" and "Averages". The tick-box for "Logarithmic Y axis" is selected on the first three secondary tab screens.

For the "CRC Errors" and "SES" screens the Y axis starts at 0.1 but for the "FEC Errors" screen the Y axis starts at 1.

Would it be possible for the Y axis of all three screens to begin at 0.1 and, thus, be consistent, please?
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 07:29:18 AM
I'll have to remind myself of how this works internally. I don't remember now how the logarithmic graphs are dimensioned.
Title: Re: DSLstats pre-release version 6.3.1
Post by: V_R on March 19, 2018, 07:59:40 AM
For some strange reason I get a 'Choose Application' dialog appear when I try and run the dslstats executable with this 6.3.1 RPi release. Yet no such issue with the older 6.1.3 version.

To add, I have tried a fresh copy but get the same error. (I download on my PC and transfer via VNC due to Chromium being painfully slow to use on the RPiZW)

Any idea's why?
Cheers.
Title: Re: DSLstats pre-release version 6.3.1
Post by: jelv on March 19, 2018, 08:28:15 AM
Error seconds y axis. Shouldn't the Y axis on all the graphs start at 0 irrespective of whether it is logarithmic or not? If you have per sample selected it appears the lowest you can get is 0.1 on logarithmic - how can you have 0.1 of an error?
Title: Re: DSLstats pre-release version 6.3.1
Post by: Deathstar on March 19, 2018, 09:58:45 AM
All working on my Pi3 Inc autostart.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 10:25:12 AM
For some strange reason I get a 'Choose Application' dialog appear when I try and run the dslstats executable with this 6.3.1 RPi release. Yet no such issue with the older 6.1.3 version.

I would guess that the executable marker on the DSLstats executable has been lost in unzipping and transferring the files. In the RPi, open the file manager and point it to the location of the DSLstats files. Right-click on the dslstats executable and select Properties from the popup menu. In the Permissions section, set the Execute option to Anyone.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 10:41:27 AM
Error seconds y axis. Shouldn't the Y axis on all the graphs start at 0 irrespective of whether it is logarithmic or not? If you have per sample selected it appears the lowest you can get is 0.1 on logarithmic - how can you have 0.1 of an error?

On a logarithmic scale a value of zero doesn't exist. The log of zero is minus infinity.

A value of 0.1 can exist in real life, because these values shown in the error graphs are per-minute values. For example, if you have one CRC in 10 minutes, that's 0.1 CRC/minute.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 10:56:08 AM
I would like to flag a very minor cosmetic inconsistency.

In the configuration, all errors are set to be monitored. Under the top level "Errors" tab, there are secondary tabs for "CRC Errors", "FEC Errors", "SES", "ES per hour" and "Averages". The tick-box for "Logarithmic Y axis" is selected on the first three secondary tab screens.

For the "CRC Errors" and "SES" screens the Y axis starts at 0.1 but for the "FEC Errors" screen the Y axis starts at 1.

Would it be possible for the Y axis of all three screens to begin at 0.1 and, thus, be consistent, please?

Having reminded myself of how these graphs are dimensioned, I see that the minimum scale values are hard coded, not autoscaled. The reason for the difference of course is that FECs generally come in much larger numbers than the other types of error, and I wanted to reduce the number of gradations on the Y axis for better readability. (The connection speed graph can also be made logarithmic, and in this case the minimum value is 10).

As the maximum values on the various error graphs are autoscaled, and can vary widely, I'm not sure that making the minimum values all the same will help a lot. What do you think?
Title: Re: DSLstats pre-release version 6.3.1
Post by: jelv on March 19, 2018, 11:44:44 AM
On a logarithmic scale a value of zero doesn't exist. The log of zero is minus infinity.

A value of 0.1 can exist in real life, because these values shown in the error graphs are per-minute values. For example, if you have one CRC in 10 minutes, that's 0.1 CRC/minute.

The problem is that it is not correct when in a sampling period there were no errors.

I don't know if you can do this, but could you set the minimum to 0.01 and then set the display to have only one decimal place which would round to 0.0?
Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 19, 2018, 11:54:13 AM
On my line both CRC Errors and FEC Errors are low enough that I find a linear Y axis more readable, so I was unaware of the inconsistency. Having said that, my upstream FEC Errors disappear altogether if I choose a logarithmic Y axis for that graph.

I think G.INP is applied to the downstream more readily than the upstream, so maybe upstream FEC errors will always be an order of magnitude lower than downstream. (Is this correct?)

On balance then, I tend to agree with b*cat, although more from an accuracy of reporting viewpoint than a cosmetic one. However, my logic may be completely wrong - and I probably wouldn't notice if it was changed anyway!
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 12:48:35 PM
Quote
I don't know if you can do this, but could you set the minimum to 0.01 and then set the display to have only one decimal place which would round to 0.0?

I could do that easily enough, although it would add an extra gradation on the Y axis (two gradations for the FEC graph), making the data a bit less readable. I really don't know what to do here.
Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 19, 2018, 04:13:19 PM
In my opinion, the ideal logarithmic scale minimum is less than 1 (to avoid missing single errors) but equal to or greater than 0.1. Anything smaller than this is relatively insignificant, and would be better analysed by reducing the sample period (which is set by the user).

Explanation
A minimum graph scale of 1 does not identify single errors.

In the "Errors" graphs, an error value between 0 and 1 can only occur if the graphing option is set to "Per minute" and the sampling rate is set to greater than 60 seconds. Any other option would yield a value of either 0 or "greater or equal" to 1.

A value between 0 and 0.1 errors per minute can only occur if the graphing option is set to "Per minute" and the sampling rate is set to 10 minutes or more. I could be wrong, but I would have thought that anyone concerned about errors that low is likely to be sampling more frequently than every 10 minutes!

BTW, wouldn't changing the minimum on the logarithmic scale from 0.1 to 0.01 require two extra gradations on the graph?
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 19, 2018, 06:27:15 PM
I think you've put it in a nutshell there Westie. Thank you. Unless anyone has strong objections, I'm going to set the minimum value on all the error graphs to 0.1 when in logarithmic mode.

On your last point, I really meant factors of 10 when I referred to gradations.
Title: Re: DSLstats pre-release version 6.3.1
Post by: burakkucat on March 19, 2018, 07:41:28 PM
In my opinion, the ideal logarithmic scale minimum is less than 1 (to avoid missing single errors) but equal to or greater than 0.1. Anything smaller than this is relatively insignificant, and would be better analysed by reducing the sample period (which is set by the user).

Explanation
A minimum graph scale of 1 does not identify single errors.

I was asked for my feedback, earlier, and have only just got around to reading the latest posts. Westie has typed the words which exactly explain my point of view . . . so much so that I just quote the first two paragraphs, above.

The sentence "A minimum graph scale of 1 does not identify single errors." is what was troubling me when I mentioned the inconsistency of the logarithmic Y axis, when the three (error) plots are considered.
Title: Re: DSLstats pre-release version 6.3.1
Post by: jelv on March 19, 2018, 07:54:35 PM
So I set my sampling period to 10 minutes and I use per sample. How do I tell the difference between a period where one error occurred and one where there were no errors?
Title: Re: DSLstats pre-release version 6.3.1
Post by: V_R on March 19, 2018, 08:58:45 PM
I would guess that the executable marker on the DSLstats executable has been lost in unzipping and transferring the files. In the RPi, open the file manager and point it to the location of the DSLstats files. Right-click on the dslstats executable and select Properties from the popup menu. In the Permissions section, set the Execute option to Anyone.

Sorted it! Thanks very much Eric. I'm still finding my way round this Raspberry Pi lark! :D
Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 19, 2018, 09:57:43 PM
So I set my sampling period to 10 minutes and I use per sample. How do I tell the difference between a period where one error occurred and one where there were no errors?

You would not be able to tell the difference if the minimum scale value is 1, but you would if the minimum scale value is 0.1.
Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 19, 2018, 10:01:40 PM
Unless anyone has strong objections, I'm going to set the minimum value on all the error graphs to 0.1 when in logarithmic mode.

That gets my vote!  :thumbs:
Title: Re: DSLstats pre-release version 6.3.1
Post by: NewtronStar on March 19, 2018, 10:03:44 PM
No amount of logarithmic y scale is going to make my FEC's look any better than what the modem telenet data gives me.

one with it on and off

Title: Re: DSLstats pre-release version 6.3.1
Post by: Westie on March 19, 2018, 10:30:14 PM
No amount of logarithmic y scale is going to make my FEC's look any better than what the modem telenet data gives me.

I sympathise with you. :(

My first response to the linear scale (the lower picture) is "Wow! What happened at around 21:30?" My response to the logarithmic scale (upper picture) is "Wow! They look high ALL the time!"
On the other hand, at least it's FEC's not CRC's.  ;)
Title: Re: DSLstats pre-release version 6.3.1
Post by: NewtronStar on March 19, 2018, 10:41:34 PM
I sympathise with you. :(

It's not to bad during the day from 8am to 3pm it hovers around 3000 FEC per minute CRC spike errors used to shoot up to into the 48-68 have been able to get these down to 1-2 CRC's but still an errored second but no SES counts.
Title: Re: DSLstats pre-release version 6.3.1
Post by: les-70 on March 23, 2018, 11:38:40 AM
  I am using the pi version of 6.3 and more recently 6.3.1.  Sorry to add some bother but I can't get the time per page to extend past 24 hours when a number of days is set.  The time per page stays at 24 hours and the hours setting moves to 1 hour after being set to 0 hours. It looks like the graphs may save after the number of days as they don't save every 24 hours when more than day is set, however I have mainly used 7 days and then messed about before 7 days so I have not tested what happens after 7 days.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 23, 2018, 12:02:39 PM
@les-70: The total time per page is supposed to be the sum of the days and the hours. I haven't got this quite right yet.  The hours setting should be limited to 23, and the next setting above that should be one day,  0 hours. It goes wrong if the hours are set to 24 when there's 1 or more days set (which should be an impossible setting). I'll sort this out, but in the meantime, if you set the hours to less than 24 it should be OK.
Title: Re: DSLstats pre-release version 6.3.1
Post by: les-70 on March 23, 2018, 12:35:48 PM
  Thanks things are better if I do that.  Could the ES/hours also be set to reflect the longer time?  That still does not budge from 24 hours.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 23, 2018, 12:44:16 PM
The ES/hour graph is constructed differently, and it would be quite a large undertaking to have it follow the time per page setting. You can always check back through the last 10 days, using the "... days ago" setting.
Title: Re: DSLstats pre-release version 6.3.1
Post by: les-70 on March 23, 2018, 02:45:52 PM
  Given that ES are most significant for DLM watching that is a pity when viewing via the web server files but but I do understand.  If 48 hours were possible and easy on the ES graph that would be nice in allowing  a fuller view via the web server.  Please take that as a thought in ignorance of the program structure.
Title: Re: DSLstats pre-release version 6.3.1
Post by: j0hn on March 23, 2018, 02:58:10 PM
Perhaps simply showing the previous days ES graph on the webserver would be easier to implement than having to redo the whole ES code.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 23, 2018, 03:29:00 PM
OK, I understand the issue. I take it you still need the per-hour figures and not just the total values per day?
Title: Re: DSLstats pre-release version 6.3.1
Post by: les-70 on March 23, 2018, 04:48:43 PM
  The total per day for each day of the graph time scale would be fine but the even just the previous days total would be helpful in the web server.  Anything to help keep full track of the ES per day.  In MDWS I only looked at the ES plot on a regular basis and I guess others also focused on that. Other stats were for when things went wrong. Replicating the MDWS style of ES plot with the rising totals for each day over multiple days would be another ideal. However hopefully you can think what would be easiest.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 23, 2018, 06:24:50 PM
OK, I'll see what I can do.
Title: Re: DSLstats pre-release version 6.3.1
Post by: g3uiss on March 23, 2018, 08:26:59 PM
Hi Eric

I’m new to DSL stats, and thanks for your work. I’ve been on HG612 since we were able to change the modem firmware.

With your software, when you have an update, is it sufficient to just replace the .exe file ? Obviously having talyored certain bits to my requirements I don’t want to overright everything.

Thank

Tony
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 23, 2018, 10:43:54 PM
Hi Tony,

Unfortunately there isn't a general answer to that question because sometimes other files in the package are changed too. Also of course it may depend on what version you're upgrading from.

If you've left the configuration files in their default location, there shouldn't be a problem because the configuration files are not in the same directory as the program files. You can change the program files to the newly downloaded files (or save then in a different directory) and the new executable will pick up your program settings.

Title: Re: DSLstats pre-release version 6.3.1
Post by: g3uiss on March 24, 2018, 01:19:24 PM
Eric

Thanks, unfortunately I changed the location of the config files to the same directory. Essentially because I'm running it on Server 2012 using a very low privileged user, it was easier to configure. I guess I can work out which are the config files. I'm on 6.1.3 pre release, the one before you changed the MWDS page. I'm interested in getting the enhancements in the latest version.

I've also created custom .html pages, but I can save these as they are static.

Its great I'm running it on two servers monitoring 2 lines. Excellent

Tony
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 24, 2018, 02:14:55 PM
  The total per day for each day of the graph time scale would be fine but the even just the previous days total would be helpful in the web server.  Anything to help keep full track of the ES per day.  In MDWS I only looked at the ES plot on a regular basis and I guess others also focused on that. Other stats were for when things went wrong. Replicating the MDWS style of ES plot with the rising totals for each day over multiple days would be another ideal. However hopefully you can think what would be easiest.

What I've done for now is extend the range of the ES/hour chart to cover two days (the current day and the previous day). It looks like the image below. I'll release this as soon as I can build the different OS versions.

[I see that there's a small cosmetic error in the positioning of the central date marker at the bottom, but that's easily corrected]
Title: Re: DSLstats pre-release version 6.3.1
Post by: V_R on March 24, 2018, 06:45:29 PM
Looking good. :)
Title: Re: DSLstats pre-release version 6.3.1
Post by: broadstairs on March 25, 2018, 11:46:51 AM
Eric apologies if already mentioned but can you change the web server files location NOT to add an additional directory to the location of webserver? What I want to do is to use a directory called DSLStatsWeb in the xampp htdocs directory, by adding the extra directory it makes me use an additional directory in the path for the url when accessing it.

Stuart
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 25, 2018, 11:50:38 AM
Yes, I should be able to do that, Stuart.
Title: Re: DSLstats pre-release version 6.3.1
Post by: broadstairs on March 25, 2018, 11:57:42 AM
Yes, I should be able to do that, Stuart.


Thank you....

Stuart
Title: Re: DSLstats pre-release version 6.3.1
Post by: ktz392837 on March 25, 2018, 12:44:39 PM
Eric apologies if already mentioned but can you change the web server files location NOT to add an additional directory to the location of webserver?

I would also find this useful.

In a similar vein it is also a pity that snapshot and DataStore is hardcoded especially as you give the option to specify the directory but then change it :(

A minor oddity I have also come across is that the snrm graph on the webserver pages is smaller than the other graphs if that can be tweaked that would be great as it looks odd on the combined page.

Really looking forward to giving the next test version a try.  I for one are amongst many that value the time you invest.
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 25, 2018, 04:55:08 PM
Quote
In a similar vein it is also a pity that snapshot and DataStore is hardcoded especially as you give the option to specify the directory but then change it

Snapshots are saved in the directory specified by the user. It's not hard coded. But you're right about the DataStore and webserver directories. I'll deal with these when I've finished with the work on the ES/hour graph (which is currently not behaving itself).
Title: Re: DSLstats pre-release version 6.3.1
Post by: les-70 on March 25, 2018, 06:52:13 PM
What I've done for now is extend the range of the ES/hour chart to cover two days (the current day and the previous day). It looks like the image below. I'll release this as soon as I can build the different OS versions.

  Eric,

   Belated thanks for the change I will look forwards to it.
Title: Re: DSLstats pre-release version 6.3.1
Post by: kitz on March 25, 2018, 10:19:25 PM
Installed and working fine here.

Thank you for the Err Stats daily total  :thumbs:
Title: Re: DSLstats pre-release version 6.3.1
Post by: roseway on March 25, 2018, 10:35:41 PM
You're welcome :)