Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roseway on May 31, 2018, 07:50:43 PM
-
Changes since v6.5.8:
- fixed bug which resulted in all uploaded snapshots being copies of the ES snapshots
- text snapshots are now included in the uploads
- corrected issue whereby auto deletion of older snapshots sometimes happened too soon (?)
I'm unsure about the last item, because I've been unable to reproduce the problem. I've made a couple of changes which might fix it.
Download from: http://dslstats.me.uk/downloads.html
-
First snapshots upload.... perfect. :)
Thanks again Eric, don't know what we would do without you.
-
Thank you :)
-
Just installed OK, will check snapshots in morning as I only do them every 2 hours but I expect them to be OK. Also will check out the delete tomorrow.
Stuart
-
Just checked and as expected the snapshot uploads are fine now. As for the delete I'll leave it another day to allow enough daily files in excess of my original 10 days to be created and I'll reduce the 60 back to 10, allow it to do the deletes and then up it again to see what happens.
Stuart
-
Eric - Installed and working.
Forgive me, how do I get Hlog and QLN in the Web Server directory ?
Tony
-
Forgive me, how do I get Hlog and QLN in the Web Server directory ?
Tony
Found the option to include all active graphs on Snapshot page. So all sorted
Tony
-
The 64-bit Linux version has been downloaded and is undergoing testing.
One oddity has been observed. I have a symbolic link to the binary file on my "Desktop" as an easy method of invoking the utility. Upon starting it running, a file named "es_data-endofmonth.txt" is created on the "Desktop". My first thought was that it is a product on my mis-configuration of the utility but a checking of all the files in the ~/.dslstats/ directory shows that is not the case.
-
I'd forgotten all about that file. When I first extended the ES/hour chart to cover two days, it was found to have a problem at the end of the month. In fixing that problem, I saved a special version of es_data.txt with that name, just to preserve the data in the event that my fix didn't work. I failed to add a pathname to the filename, so the file is saved in the current directory. I see on my system that it's saved in my home directory. In your case, the desktop has become the current directory.
In any event, the fix did work, so that precautionary file isn't needed. I'll remove that line of code for the next release.
-
Ah, I see. Thank you for the prompt analysis. :)
-
Eric I have just taken a look at the snapshot delete again and found that having reduced my value to 20 days (2 days ago) it still has not removed any of the 34 folders in the graphs directory. However I think I might know why. The daily folders are not for consecutive dates, there are some current May June consecutive ones but less than 20, but then some from March and February making up the total. I suspect this is the issue and that the code simply counts back 20 days from the most current date and tries to delete the 21st which of course does not exist so stops.
Now this is not a big issue and to be honest I dont know how I got those old folders. So if I'm correct in my assumption I dont believe it is worth bothering with.
Stuart
-
If your diagnosis is correct I'd disagree with your conclusion.
If someone goes away on holiday for a time and while away turns everything off including the router and what is running DSLstats, on return and starting up again there will be a number of directories that are not deleted.
-
I'll take another look at it.
-
Is there any possibility of adding a couple of the G.INP graphs to the webserver? Would be handy to have rtx_tx and LEFTERS, and possibly the Data Summary tab as part of the text pages.
Sorry if all or some of this already exists and I've just configured things incorrectly.
-
Yes, that can be done. It will take a little time.
-
Apologies for the following questions if they have already been answered elsewhere, but despite looking through the old 6.5.8 thread I would appreciate some clarification on a couple of queires please.
I was aware that DSLstats was aiming to upload the contents of the webserver folder but what I would like confirmation about is "Snapshots".
Are these the contents of the folders that DSLstats saves to ie CRC, ConnSpeed, FEC etc or is it something simpler than that?
When/if it uploads 'snapshots' how does it store them directorywise. ie does it mirror the local directory configuration or are all files stored in one directory. Reason I'm asking is that I may need to alter my code in history.php which relies on the graphs being in sub-directories on the remote server.
Does the integrated ftp in DSLstats do a synchronise before uploading any files or does it do a rewrite of all files.
I know that I'm a bit late discussing this topic, but as I've been successfully been using WinSCP and pulling live and history stats for the past few months with my scripts, I've not been taking much notice as to whats been going on with DSLstats. For the past 2-3 months my head has been totally tied up with converting the site to https and then straight on to GDPR and other site admin its only now that I've had any time to revisit anything to do with linestats.
-
There's nothing to apologise for. I haven't properly documented the latest features yet.
All the FTP uploads (Data store, Webserver, Snapshots) are performed immediately after sampling, so they contain the latest data at the time of uploading.
Data store: the main stats are uploaded after every sample, the four per-tone zips are uploaded once an hour (i.e. the same timing as MDWS used to use).
Webserver: the frequency of the uploads is defined in the upload configuration.
Snapshots: all the local snapshots are processed according the Snapshots configuration, and if they're configured to be uploaded then this is done immediately after saving them locally. The way it works is that every snapshot is temporarily copied to an "Uploads" folder (including any manual snapshots which have been made since the last upload), then the entire contents of the Uploads folder is uploaded to daily folders on the server, and afterward the files are all deleted from the Uploads folder. The filenames of the uploaded snapshots are the same as the locally saved snapshots.
-
Thank you for taking the time to answer :)
It was the snapshots folders I was wondering most about and now that you have explained to me how they work, then I could probably turn off the WinSCP scripts and do everything using DSLstats and it should hopefully still work with history.php without any modifications.
With DSLstats passing the snapshots to a temporary folder that avoids a lot of the issues I was having developing the WinSCP script for snapshots which had been my sticking point for several months with my FTP script when my graph type folders contained many months/years of data. If I had been on the ball and realised this earlier it would have saved me messing around with WinSCP trying to devise a more robust FTP script which was the only sticking point I had with history.php. I must have totally missed that fact whilst I was somewhere in GDPR land.
However, if I have understood correctly, it has made me aware that I perhaps need to develop history.php further to give an option for those who save folders in date ordered folders rather than categories.
Thanks again for the excellent work you do with DSLstats.
-
Hi Kitz
Snap shots folder does upload by date order. I exported all my old folders so my old stuff is present.
-
Cheers. All my DSLstat folders are saved as categories rather than date. I'll switch off my Winscp scripts later and have a play with DSLstats. :)
-
When I started my scripts I decided on sub-directories in date order so as to reduce the searching needed to find stuff, it does not matter whether the files are in name or date order within the date directory. I thought most folk would not want to look at more than a days worth of files. The historical data (in Datastore) is by date anyway so again my javascript graphs only show 1 day of data maximum. Hopefully this does not put too much strain on an RPi running a webserver, obviously using a hosted server the load is less problematic. What I refer to as Live files are the DSLStats own webserver files uploaded and they get replaced on each upload so there is only ever 1 file for each graph etc.
Stuart
-
With Webserver uploads, is there an option to just update the .png files, or does the whole set of files html/htm etc get uploaded every time.
Not sure if that's an option in the existing version. I'm sort of thinking bandwidth ?
Tony
-
At present all the webserver files are uploaded. But if you don't need the webserver .html files you can configure it to just save snapshots, in which case only these will be uploaded. Or you could use snapshots directly of course, ignoring the webserver setup.
-
Eric
Thanks, useful options there
Tony
-
Using the notification option when ES exceed a entered figure, is it possible to stop the repeated emails each time the number increments. I assume it would stop anyway at the end of the current hour ?
Thanks. I might have missed a setting.
-
I've never actually considered that issue, but I see what you mean. I'll see what I can do about it, but I'm afraid it will take some time.
-
Eric
Thanks.
-
Just a tiny, tiny thing I noticed:
I've been playing around with Stuart's web scripts and tried using the snapshots FTP upload feature on DSLstats 6.5.9. I then noticed that the FTP uploads file name for SNR Margin graph doesn't seem to be the same as the "local" file name e.g. the local file is saved as "SNRMargin-xxxx-xx-xx-xx.xx.xx.png" however the FTP uploads file is named "SNRM-xxxx-xx-xx-xx.xx.xx.png"
Therefore Stuart's scripts weren't picking up the SNR Margin graphs. I had a quick look at changing the script but because the strings aren't distinct, it's going to pull the SNRMperbandUpstream and SNRMperbandDownstream graphs as well as the SNRM graph.
Might be me doing something wrong? I probably didn't explain it all that well, sorry :-[.
Otherwise DSLstats 6.5.9 seems to be working perfectly on the Pi I have it running on. So thank you for the wonderful program.
-
Thank you for your comments, and I don't think it's you doing anything wrong. I'll take a look at it as soon as I can.
-
One thing I've noticed, is there is a 4 retries and your out for FTP uploading.
I've noticed if my FTP server (NAS) is doing it's weekly virus scan it seems to lock up the FTP port and the transfer fails
DSLStats tries the 4 times and aborts. Could this be made user configurable?
#Tries and time between retries?
-
I'll add it to the list. Hopefully I'll be able to resume work on DSLstats soon.
-
I have this tool (v6.5) running now for some weeks and I found some bugs and issues. I hope I am allowed to report some of them in this thread.
- if there is no DSL connection then time runs fast forward on the charts and it creates some weird graphs* after DSL connection is back
for example see https://i.imgur.com/1xaNS48.png - my line uses vdsl with profile 35b and when I activate "include with bitloading" in the Items to monitor options (the other options "SNR per Tone" and "show as separate graph" activated too), then I get an access violation error
I have these issues also with v6.5.9.
-
Eric has mentioned before he doesn't have the time to add full profile 35b support right now. It isn't used in the UK either so probably not a priority.
-
Thanks j0hn, that's quite right. Also, my computer time is very limited at present because of eye strain.
-
Quick question and maybe I'm just missing something obvious in the configuration, does or will DSLstats support bonded connections? I noticed there are options for the command to use xdslctl, xdslctl0, xdslctl1 and my modem (ZyXel C3000z / Century Link) supports all of the commands. Looking at the output it appears xdslctl and xdslctl0 produce the exact same output while xdslctl1 shows different stats for everything above 'VDSL2 framing' portion of the response (everything after that appears to be the same as the other two commands as well).
The reported speed in the status bar only shows one half of the connection along with most of the other stats, if I change the command to use xdslctl1 then I'll see stats for that line and the graphs change accordingly. Not sure if there's a way to run two instances of DSLstats at once, but I suppose that's a way to keep track of each line at least.
Thanks for the app as well! It's a great utility to keep track of line status and it works perfectly for me at home. Even if I lose internet my mail server is in the basement so I can still get alerts. I know my modem isn't supported but it seems to work well with DSLstats as long as I don't enable certain monitoring features.
-
DSLstats wasn't designed to support bonded connections, and to add that support now would be beyond me, I'm afraid. You can't run two different instances of the program on the same machine because they would both be accessing the same set of configuration files. A Linux user could run a second instance by using Wine to run the Windows version on the Linux machine. Or you could run a second instance in a virtual machine or on a different machine, for example a Raspberry Pi.
Sorry I can't be more helpful.
-
Actually that just gave me an idea to run the instances in sandboxie and I can give them both their own config files and workspace. Not sure how it will work, but it's an idea. For now I just installed a second instance of it on the other server and I've got each one monitoring a line.
Completely understand that you don't have the time to dedicate to that change, I think it'd probably be a pretty big overhaul in some areas. Thanks anyway for the application itself!
-
Good idea, just fire the application up inside a VM.
-
I have 3 VM's monitoring 3 different DSL modems.
I monitor both of my lines, and my fathers via an SSH tunnel - because he doesn't have a machine he runs all the time.
You only need very tiny VM's to achieve this.
-
If roseway ever does get the time to make the change, the app could always take a command line parameter to tell it where it to find the config file. It would also have to stop detecting existing running processes of itself, or windows with the same name, or any other active self-detection methods.
-
It's not a matter of available time, I'm afraid. I can't concentrate on a screen for long before my eyes start going funny (like a migraine without the headache), and I'm not prepared to take risks with my eyesight. Any changes I make to DSLstats from now on will be small ones.
-
Oh no, I am very sorry to hear that. My eyesight is shot and I sympathise.
-
(like a migraine without the headache)
I get that from time to time particularly if I'm tired and spending too much time at the computer. I get a zig-zag flashing halo pattern usually in one eye only which thankfully only lasts for about 20 minutes. My optician described it as a painless migraine. It is very disconcerting while it lasts.
-
Same here: my optician (and doctor) called it "silent migraine". https://migraine.com/migraine-types/silent-migraine/.
It usually comes with stress and overwork. (Rare since I retired.)
For me, what works best is to lie down somewhere dark for half-an-hour.
My sight's not great, but my feeling the "light-show" is a "wiring" rather than a "sensor" problem.
I've had some blinding/disabling headaches, but I don't think I've ever had a migraine.
-
Yes, "silent migraine" is the term my doctor used.
-
Very good description for the ailment, I have never heard this description before from my optician or GP, as a long term migraine sufferer I can certainly vouch for it's accuracy.
Definately appears to be a common factor, too much time in front of the PC's VDU.
-
I can sympathize on the migraines, my trigger is fluorescent light for an extended time or any type of artificial sweeteners. If it's a full on migraine I'm down for the next 24 hours and a dark quiet room is my only reprieve.
My long code review sessions used to have my eyes bugging out where I used to get a 'focus circle' in the middle of my vision after hours. Kind of where the center of my vision is clear but the outer ring is a bit blurry and it makes reading things quickly a pain as it's easy to lose what line I'm on.
Something I'll throw out there that's really helped me a lot with spending a lot of time in front of the computer screen, Gunnar glasses. They're specifically meant for viewing computer screens and help with eye fatigue & headaches associated with viewing monitors. Might be worth a shot to see if it helps, I picked my pair up for around $40 from Amazon about 2 or 3 years ago. Here's a link to their site: https://gunnar.com/
-
Thanks for the hint. I'll take a look at them.