Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roseway on April 09, 2015, 07:18:38 PM
-
This version is intended as the basis for the next full release. There is now a new G.INP tab which becomes visible when a G.INP enabled connection is detected during sampling. This tab contains several graphs of G.INP related data and a table of the main G.INP settings.
http://www.s446074245.websitehome.co.uk/downloads.html (http://www.s446074245.websitehome.co.uk/downloads.html)
(Later)
The Raspberry Pi (Raspbian) version is now available for download.
Note about the Raspberry Pi: Some versions of the RPi may not have updated their clocks since we moved to summer time. If you're uploading data to MDWS you need to correct this. Open a terminal and type
sudo dpkg-reconfigure tzdata
You may be asked for your user password, then set the timezone to Europe/London. After completing this, reboot the RPi and it should be displaying the correct time. You must reboot before running DSLstats.
-
Many Thanks, appreciated.
-
Very nice indeed now i have more spikes to look at with g.inp.
Wonder how the rtx_tx and rtx_c will look like during a T/Storm this year :o
10 out of 10 for implementing g.inp stats into DSLstats.
-
Thanks :)
-
Just a quick note.......
The Attenuation Log mentions Signal Attenuation:-
Signal attenuation (dB) U0 U1 U2 U3 U4 D1 D2 D3
10 Apr 2015 07:26:39 8.1 53.2 N/A N/A N/A 21.7 65.3 N/A
However, it actually appears to be reporting the Line Attenuation values:-
xdslcmd info --pbParams
xdslcmd: ADSL driver and PHY status
Status: Showtime
Retrain Reason: 0
Last initialization procedure status: 0
Max: Upstream rate = 5266 Kbps, Downstream rate = 22024 Kbps
Bearer: 0, Upstream rate = 4999 Kbps, Downstream rate = 22399 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Discovery Phase (Initial) Band Plan
US: (7,32) (871,1205) (1972,2782)
DS: (33,859) (1216,1961) (2793,3970)
Medley Phase (Final) Band Plan
US: (7,32) (871,1200)
DS: (33,859) (1216,1783)
VDSL Port Details Upstream Downstream
Attainable Net Data Rate: 5266 kbps 22024 kbps
Actual Aggregate Tx Power: 6.6 dBm 12.8 dBm
====================================================================================
VDSL Band Status U0 U1 U2 U3 U4 D1 D2 D3
Line Attenuation(dB): 8.1 53.2 N/A N/A N/A 21.7 65.3 N/A
Signal Attenuation(dB): 8.1 53.1 N/A N/A N/A 30.4 65.1 N/A
SNR Margin(dB): 5.8 5.9 N/A N/A N/A 6.0 6.0 N/A
TX Power(dBm): -1.0 5.9 N/A N/A N/A 11.6 6.3 N/A
Some users, particularly those on longer lines - such as me, do see quite a difference between DS Line & Signal attenuation values.
Might it be worth reporting both in the Attenuation Log?
-
Thanks for that, I'll do what you suggest.
-
So it does - Same here,
Yet at one point Im sure it was correct.. or at least when I had the line fault in 2013 as my signal attenuation was changing frequently and why I asked if eric would add it to DSLstats for me.
-
Yes, I'm sure it was correct when I first implemented it, but I agree that it's not correct now. I'm not sure where it went wrong yet, but I'll sort it out.
-
The Raspberry Pi version is now available for download. See the opening message.
-
The Raspberry Pi version is now available for download. See the opening message.
Have been having issues with the RPi bluetooth mouse and keyboard over the last 4 days they work for a while and won't awaken the RPi from it's screen sleep mode even tried another bluetooth adapter and a fresh install of debian wheezy.
Sorry to ramble Roseway but i can't install you lastest RPi DSLstats at the moment :(
-
I don't mind you rambling :)
-
Should the latest test version for RPi have the correct version number displayed? It does not appear to be incrementing between versions? Thanks
-
It should show "DSLstats v5.4.5" in the title bar. It did when I tested it here. It should also show the same in the bottom right hand corner of the graphs.
-
It should show "DSLstats v5.4.5" in the title bar. It did when I tested it here. It should also show the same in the bottom right hand corner of the graphs.
Just downloaded this version on the RPi and it's a TAR file not a ZIP it won't unzipp it's ok must be going doolaaley in my old age, ps the new LXDE on the RPi is great it's very quick all round.
-
It should show "DSLstats v5.4.5" in the title bar. It did when I tested it here. It should also show the same in the bottom right hand corner of the graphs.
Just downloaded this version on the RPi and it's a TAR file not a ZIP it won't unzipp it's ok must be going doolaaley in my old age, ps the new LXDE on the RPi is great it's very quick all round.
I de-tar manually and then use SFTP to transfer the DSLstats files across, easier that way I find :)
-
The reason I changed the archive format from .zip to .tar.gz was because people were using Windows to unzip the archive, and in doing so the executable file was losing its executable permission. I thought that the .tar.gz archives would get extracted on the Raspberry Pi, which would preserve the executable permission.
In retrospect, all this achieved was to make it more difficult for people. I've now replaced it with a .zip file.
-
Does this version upload to MDWS as the RPi is up and running here as the event log says
unable to upload to MyDSLWebstats data (file "upload13.rpi" missing there are loads of these events.
Edit: found the problem i had used the Icon on the desktop to run DSLstats but this time i executed the program from the folder.
-
Yes, DSLstats expects to be run from the folder its files are in. If you want to run it from somewhere else you need to create a little launch script for it.
-
Yes, DSLstats expects to be run from the folder its files are in. If you want to run it from somewhere else you need to create a little launch script for it.
It's not so straight forward like ms windows but at least i'm starting to get the hang of editing files and scripts in Debian Wheezy.
Will tony be setting up new parameters on MDWS to show the new g.inp stats that are available ?
-
The script should take this form:
#!/bin/sh
cd <the folder where the DSLstats files are> [e.g. cd /home/pi/dslstats]
./dslstats
Some of the G.INP data will be added to MDWS, but it's not yet decided which data to include - there's an enormous amount of it. :)
-
there's an enormous amount of it. :)
I know it's like learning a new set of broadband rules all over again and thanks for the script that will help.
-
I upgraded to this new version of the program, so it can display my G.INP data, but I'm encountering a few oddities. Some are probably due to this being the first version for G.INP, but some might be because of problems I've been encountering with power here, which has killed the computer 3 or 4 times.
1. When starting the program, then hitting the "start recording" button, then waiting for the first set of samples, I'd cause a crash. The pop-up window would inform me, and ask if I wanted to continue with the potential of using corrupt data.
If I allowed the program to continue, I could see that the graphs for Hlog and QLN stopped at around tone 250 (and stayed that way forever). The bitloading graph stopped at around the same point (with all bars coloured blue), but would fix itself after the the next sample.
This happened a few times, but it has stopped now, so I can't get a screenshot of the pop-up window. But I do have snapshots of the initial graphs.
2. I always used to have the "SNR per band" graphs ticked, and they'd be included in the tabs. I'd see plenty of variation, too. On starting 5.4.5, this tab seemed to be not present after startup, and needed to be re-configured.
However, when it did appear, I seemed to get one of two behaviours:
a) I get a historical line (coloured for U2 or D3), as though all previous samples were an identical value ... but the new samples start to vary correctly
b) I get the same kind of historical line, but all future sample stay identical - so I get one single straight line (coloured for U2 or D3).
I don't know if (a) is expected, but (b) is definitely wrong ... the same values logged on HMS show variations that ought to be visible.
On my most recent startup, the graph was present as normal, and values were logged independently, correctly.
I'm now wondering if this got affected by a change I made in the "Items to Monitor" screen; by default, (under pbParams), the "autodetect" option is ticked but greyed out, and I have had the "SNRM per band" option ticked. I think that, on first starting 5.4.5, I added a tick to "Stats per band" before starting the recording.
IIRC, I couldn't see any new stats, and I reckon this triggered the "SNRM per band" setting to get mixed up.
3. The G.INP graphs don't follow the configuration of the other graphs
I have my graphs configured to show 8 hours, and a 3-pixel line. The G.INP graphs are 2 hours, with 1-pixel line width.
4. The G.INP graphs don't scroll back-and-forth correctly
I can use the mouse wheel to scroll left and right, but once I've sent the viewport towards the left, I can't get it all the way back to the right.
It does revert to the right at the next sample, though.
-
Taking those points one by one:
1. I'm just thinking aloud here. It would seem that something caused the first sample after pressing the green button to contain corrupted or incomplete data. So the connection mode was perhaps wrongly detected as ADSL, with various consequences. The reason that the HLog and QLN graphs didn't recover with subsequent samples is that they are only read once per recording session (or again after a resync).
The warning popup indicates that an untrapped exception occurred, and as it's not happening now it's going to be difficult to trace it. I assume that you're connected by ethernet, not WiFi, by the way. I'll see if I can come up with any ideas.
2. This looks as though it's another consequence of the first issue, as it's now working correctly. I've never known an instance of the SNRM per band tab failing to appear on a restart, and the other weird effects look like corrupted data.
3. I agree. That's a detail I forgot to implement. It's easily fixed.
4. I've been aware that scrolling has some imperfections, but as the position is restored with each new sample I haven't given it a lot of attention. It's on the list of things to do.
[Later] I've realised that scrolling depends on the hours per page value, so the main issue here is another result of item 3. With that fixed, item 4 will be fixed too.
-
Roseway with 5.4.5 i am deffo seeing odd stuff like unable to connect to modem this is on the PC and the RPI last night just stopped logging with the upload13.rpi error.
I pause it and then start recording again and all works again i
Here is the event log for the PC version
16 Apr 2015 17:13:45 Configuration files stored in C:\Users\Colin\AppData\Local\dslstats\
16 Apr 2015 17:13:45 Snapshot folder is C:\Users\Colin\Documents\dslatats graphs\
16 Apr 2015 17:13:48 Recording started
16 Apr 2015 17:13:55 IP address is now *.**.***.***
16 Apr 2015 17:27:50 Unable to login to modem/router
16 Apr 2015 17:33:50 Unable to login to modem/router
16 Apr 2015 17:44:50 Unable to login to modem/router
16 Apr 2015 17:53:50 Unable to login to modem/router
16 Apr 2015 18:00:50 Unable to login to modem/router
16 Apr 2015 18:06:50 Unable to login to modem/router
16 Apr 2015 18:53:50 Unable to login to modem/router
16 Apr 2015 18:58:55 Timeout while retrieving stats
16 Apr 2015 18:58:55 No stats received
16 Apr 2015 19:08:50 Unable to login to modem/router
16 Apr 2015 19:09:46 Recording stopped
16 Apr 2015 19:09:47 Recording started
16 Apr 2015 19:09:55 IP address is now *.**.***.***
edit: spoke to soon
16 Apr 2015 19:18:49 Unable to login to modem/router
16 Apr 2015 19:25:49 Unable to login to modem/router
16 Apr 2015 19:33:49 Unable to login to modem/router
-
A reboot of the HH5A (router) seems to have fixed the error's so i'll stand back and continue to monitor for any issues.
-
It's certainly one possible cause. I'll wait to hear how it goes.
-
@WWWombat:
2. This looks as though it's another consequence of the first issue, as it's now working correctly. I've never known an instance of the SNRM per band tab failing to appear on a restart, and the other weird effects look like corrupted data.
To expand on what I said above:
The "autodetect" option under pbParams is always ticked and greyed out. This was meant to indicate that the option is always enabled and can't be disabled.
The "Stats per band" option is now a pointless option. Originally it determined whether the pbParams tab was visible under "Telnet data", but it's clear that there's no reason to hide it (except on ADSL connections). I'll remove the option in the next release. Enabling it wouldn't have been responsible for the issues you had.
-
It's certainly one possible cause. I'll wait to hear how it goes.
No issues over the last 24 hours so i'll put this down to an odd router problem :fingers:
-
No issues over the last 24 hours so i'll put this down to an odd router problem :fingers:
Thanks for that. I'll cross my fingers as well. :)
-
1. I'm just thinking aloud here. It would seem that something caused the first sample after pressing the green button to contain corrupted or incomplete data. So the connection mode was perhaps wrongly detected as ADSL, with various consequences. The reason that the HLog and QLN graphs didn't recover with subsequent samples is that they are only read once per recording session (or again after a resync).
Ah yes - detecting it as ADSL would indeed explain a limitation of 250ish tones.
The warning popup indicates that an untrapped exception occurred, and as it's not happening now it's going to be difficult to trace it. I assume that you're connected by ethernet, not WiFi, by the way. I'll see if I can come up with any ideas.
Actually it is connected via WiFi at the moment ... either direct to the 8800NL's wifi, or via a Devolo powerline wifi adapter that then plugs into the modem via ethernet.
I do swap the connection around somewhat.
2. This looks as though it's another consequence of the first issue, as it's now working correctly. I've never known an instance of the SNRM per band tab failing to appear on a restart, and the other weird effects look like corrupted data.
It looks like I misspoke - the program now manages to display the SNRM-per-band graphs, and does an initial reading that varies by band, but then the graphs are just straight lines from that point onwards.
-
It looks like I misspoke - the program now manages to display the SNRM-per-band graphs, and does an initial reading that varies by band, but then the graphs are just straight lines from that point onwards.
There are two ways in which that can happen:
- the values don't change, so the graphs are correct (unlikely but not impossible)
- the pbParams data is only read once, immediately after the start of recording
The second possibility is what happens if none of the optional items under "pbParams" are ticked, but in that case it shouldn't be showing the SNRM per band graphs.
In the next full release, which I'm hoping to put out soon, I've removed those options. On VDSL2 systems the pbParams data will always be read. This should remove the possibility of the second explanation occurring. I can't really see how it's happening now, but I would theorise that DSLstats is misbehaving in some way when a sample is not collected correctly, and perhaps this is a consequence of using a WiFi connection.
-
In the next full release, which I'm hoping to put out soon, I've removed those options.
Were you intending to include uploading any of the G.INP related data to MDWS in your next full release?
If so, perhaps we could confer with tbailey2 & agree a) exactly what data to upload & b) which fields to use for both DSLStats and HG612 Modem Stats.
-
I'm ready to include G.INP data in the MDWS uploads when we have an agreement on the details. Let's confer with tbailey2 ASAP to establish what fields are available, then we can decide what to include.
-
I'm ready to include G.INP data in the MDWS uploads when we have an agreement on the details. Let's confer with tbailey2 ASAP to establish what fields are available, then we can decide what to include.
I've e-mailed you both with some ideas...
-
It looks like I misspoke - the program now manages to display the SNRM-per-band graphs, and does an initial reading that varies by band, but then the graphs are just straight lines from that point onwards.
There are two ways in which that can happen:
- the values don't change, so the graphs are correct (unlikely but not impossible)
- the pbParams data is only read once, immediately after the start of recording
The second possibility is what happens if none of the optional items under "pbParams" are ticked, but in that case it shouldn't be showing the SNRM per band graphs.
In the next full release, which I'm hoping to put out soon, I've removed those options. On VDSL2 systems the pbParams data will always be read. This should remove the possibility of the second explanation occurring. I can't really see how it's happening now, but I would theorise that DSLstats is misbehaving in some way when a sample is not collected correctly, and perhaps this is a consequence of using a WiFi connection.
Up until last week, the values were always changing in the per-band graphs; when something changed in the overall SNRM graph, there would often be a similar change in at least one of the bands in the per-band graph. It would be strange for there to now be changes in the overall graph, but no corresponding change to per-band.
The WiFi connection was being used back then too. Things do go wrong with it - but I then see a thin line in the speed and overall SNRM graphs, to denote the failure to sample. Looking back, the per-band graphs never seem to show the same thin lines.
In my configuration, the "SNRM per band" item is ticked.
Note: In adding graphs as attachments, I just realised that DSLstats might well be displaying the per-band graphs, but it isn't saving them as part of the automatic snapshots. However, they are saved when I hit the button in the GUI. Lo and behold, the option to auto-save them had turned off on the config screen.
These graphs were being auto-saved before, so whatever change I made (or happened anyway) means this tick has been lost. I've just turned it back on.
-
Just a post to let you know if i change the snapshot to a different directory like my pictures on the PC version i'll get a Division by Zero error the next time i run DSLstats from a cold reboot.
The way i fixed this was to stop recording stats and press ok when the error message pops up and change the snapshot directory is was originally using and close down dslstats and press start recording.
-
Thanks, I'll check that out shortly.
-
Sorry, but I don't use Windows myself. What "Scheduled task" is this?
[Edit] The message to which this refers seems to have been deleted, but I'll leave this and the subsequent messages.
-
Sorry, but I don't use Windows myself. What "Scheduled task" is this?
It's the task that runs the stats every minute to update the GUI.
-
I think you're confusing DSLstats with HG612 Modem Stats.
-
I am afraid to say i am still getting the division by zero error on pc going have uninstall the 5.4.5 and purge the user\appdata\local and re-install dslstats 5.4.5 of will lose all setting.
-
I'm sorry to hear that. I've spent some time today trying to reproduce the problem without success. The only way I could get the program to misbehave (using a Win 8.1 laptop) was if I changed the snapshot folder to one where ordinary users don't have write permission, for example C:\windows.
If it's not too late, you can avoid losing your configuration settings by editing dslstats.ini manually before running it. Find the section headed "[snapshots]" and delete the line starting "Snapshotdir=" (or just delete everything after the =). The next time you run DSLstats the snapshot folder will revert to its default.
-
It's to late and i'll know tomorrow if what i've done stops the error coming back.
-
It's to late and i'll know tomorrow if what i've done stops the error coming back.
No errors to-day and PC and DSLstats & HG612_Modem_stats have been working away for over 11+ hours :thumbs:
-
Thanks for confirming that. :)
-
Hi Eric,
I upgraded to 5.4.5 and I migrated DSLstats to a new desktop environment (LXQT) on my NAS. Previously I was using it under Debian chroot environment.
Everything's working perfectly, data is collected correctly from the modem, graphs are working, but I have a problem with MDWS upload (Unable to check at present error).
Here are a couple of screenshots:
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FtLSg8mU.png&hash=609ee4449dc108e6127e2e7ec3a93540d6736098)
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FOFF95j8.png&hash=fecc227c6c9eb4508715413d06b458dda9a334e4)
I checked if curl is working from shell, the path is correct and curl works.
I don't know what else to check, so I though to ask for support here.
Hope we'll be able to solve this one. :)
-
Hi Alex,
I've done some tweaks to the validation check, although I can't really explain your issue. If you'd like a test version to try, please let me know. Is your system 32-bit or 64-bit?
-
64bit. I'll test it immediately. Also: I tried version 5.4 and I have the same issue, so I don't think it's something introduced in 5.4.5.
In the test version, could you add some sort of debug writing (in the event log tab) to the validation check routine so we can see what's happening and where it fails exactly? I'm thinkg about some components missing at the OS level.
-
I'll add some debugging entries and give you a link a bit later.
-
You can download test version 5.4.7 (Linux 64-bit) here:
http://www.s446074245.websitehome.co.uk/files/dslstats64L-5.4.7-debug.tar.gz (http://www.s446074245.websitehome.co.uk/files/dslstats64L-5.4.7-debug.tar.gz)
It will add a few lines to the event log to indicate progress with the validation key check. If the check is successful it will only do this at the start of the recording session, or if you press the "Check now" key.
-
Same problem, here's the screenshot with the debug lines.
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FTCioC0S.png&hash=589be53dae1735dcfbb95df42ef508795648de54)
It looks like the communication with the MDWS site is working, but I don't know what happens after it receives the response.
Question: if I input a wrong validation key, is the error msg specific or it's always the same? I just tried a wrong validation key and the error msg is the same.
Let me know if I need to try else.
-
Just noticed another error in the event log:
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2Few0OH3C.png&hash=53ca357b751516acfcb518d751a493e84ffe35df)
SSL error: hope it is related to the validation check so we can understand what's going on.
Something at the OS level (ssl libraries) as a prerequisite?
-
Ok, I solved the "SSL error: TSendMail" error by installing the libssl-dev package on ubuntu. I didn't have the SSL/TSL option checked in the Alert tab, so it seems libssl-dev is needed for TSendMail even if SSL/TLS is not used.
The validation check issue is still there, so the 2 issues where not related.
-
OK, thanks, I'll investigate.
-
I think I've resolved it now. I was unnecessarily saving the result of the validation check to disk, and it was this which was triggering the exception. I'm not sure why this was, but I'm guessing some sort of permissions problem. This version doesn't make that unnecessary file save:
http://www.s446074245.websitehome.co.uk/files/dslstats64L-5.4.8-debug.tar.gz
-
Ok, just tested and it works now. :)
About the ssl issue with TSendMail, is it ok that the libssldev is required even if SSL/TLS is not checked in the options?
And the last issue that I have is the following: I noticed that many working files are not created under $HOME/.dslstats but in $HOME.
Check the image below, considering that I'm in the $HOME directory of the user I'm running dslstats with (user: ubuntu, $HOME: /home/ubuntu).
Some files are created under "$HOME/.dslstats" but many others (I counted 16) are created under $HOME.
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FmaKYPan.png&hash=8b0a5ff9ee93223757a273cf03e88f11299b3fcb)
-
Ok, solved it. It was a directory permission issue. DSLstats created the .dslstats directory and the snapshosts subdirectory, but I don't know why, it wasn't able to write to them. I changed all permissions on .dslstats (and subdirs) to 775 and restarted. Now it seems to be able to write correctly.
This was the real problem also for the MDWS validation check I think. ;)
It could be useful to do a write-check on start to see if you are able to write to .dslstats and relevant subdirs, and write the outcome in the event log.
Eric, thanks for everything. I'll wait for the final version. :D
-
Ok, just tested and it works now. :)
Thank you :)
About the ssl issue with TSendMail, is it ok that the libssldev is required even if SSL/TLS is not checked in the options?
I think it's a dependency of the third party library which I use for the email functions.
Ok, solved it. It was a directory permission issue. DSLstats created the .dslstats directory and the snapshosts subdirectory, but I don't know why, it wasn't able to write to them. I changed all permissions on .dslstats (and subdirs) to 775 and restarted. Now it seems to be able to write correctly.
This was the real problem also for the MDWS validation check I think. ;)
It could be useful to do a write-check on start to see if you are able to write to .dslstats and relevant subdirs, and write the outcome in the event log.
Yes, your analysis is right. On the occasional system, yours being an example, DSLstats fails to set up the .dslstats directory properly, and the various symptoms devolve from that. I'll give this some attention straight away.
-
Eric, what would be needed to support the ECI modems? I don't know whether to request the source so I could add support or add a script file to the modem to supply data in a format DSLStats can easily parse? In that case DSLStats would need to know the name & location of the script to invoke. I'm assuming it currently only supports Broadcom chipset modems that provide XDSLcmd --info and cousins?
Steve
-
Eric, what would be needed to support the ECI modems? I don't know whether to request the source so I could add support or add a script file to the modem to supply data in a format DSLStats can easily parse? In that case DSLStats would need to know the name & location of the script to invoke. I'm assuming it currently only supports Broadcom chipset modems that provide XDSLcmd --info and cousins?
Steve
It would require a total rewrite of the data collection and processing parts of the program, i.e. nearly everything except the graph drawing parts. That's currently about 7000 lines of code.
The alternative of adapting it to import data from another program or script instead of doing its own data collection could work, but the task for the other program to convert the data would be enormous. It may not even be possible (do the ECI modems provide the equivalent of pbParams, for example?).
In my opinion it would be quicker and cleaner to start from scratch. I don't personally have the time or the energy to undertake the job either way, but I would try to help if another programmer took it on.
-
OK thanks for that. In that case it's probably best if I drop the idea given the comparative lack of interest in this modem and it's awkwardness. I think I'll just collect my own line data for now and stuff it into a SQl db while I tinker with a local web display app.
Thanks again,
Steve
-
Anyone know what might cause this in latest DSLSTATS?
Error code 2 while uploading MyDSLWebStats data
Was fine yesterday but today it's not playing ball.
Update: Upgraded to preview version and all is fine.
-
That error code will have come from the curl program which performs the upload. Unfortunately error code 2 is an indeterminate error which doesn't say anything helpful about what caused it. It could even have been a corrupted curl.exe, and installing a newer version of DSLstats will have brought in a clean copy of that executable.