Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on May 30, 2014, 03:25:04 PM

Title: DSLstats pre-release version 4.54.4
Post by: roseway on May 30, 2014, 03:25:04 PM
This version is functionally the same as v4.54.2 described below, but with two internal changes:

- the section which handles changes of the snapshot directory has been tidied up to remove the risk of deleting the wrong files (see http://forum.kitz.co.uk/index.php?topic=14035.0 (http://forum.kitz.co.uk/index.php?topic=14035.0) for details)
- the HG622 login code has been modified to resolve the timing issue with some systems

This version uses the faster networking library, and hopefully resolves the remaining issues with it. I would appreciate it if the long-suffering testers could give it a try and report their findings. Thank you.

http://www.s446074245.websitehome.co.uk/downloads.html
Title: Re: DSLstats pre-release version 4.54.4
Post by: burakkucat on May 30, 2014, 04:21:00 PM
PRVersion 4.54.4 has been downloaded and is now running at The Cattery.

So far, everything looks good. There was no sign of any attempts to delete incorrect files or a directory.  :thumbs:
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 30, 2014, 04:23:33 PM
Many thanks :)
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 30, 2014, 05:27:20 PM
Still having problems here Eric. First time I started it I got a crash List index out of bounds (I think) and now it says no data after a restart. Linux 64 bit version. It does seem to get data after a while though and it has not crashed again (yet!).

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: burakkucat on May 30, 2014, 05:33:49 PM
Which 64-bit Linux-kernel based OS are you using, Stuart? I'm using RHEL 6.5 and have no trouble whatsoever . . . (Huawei HG622 modem/router.)
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 30, 2014, 07:17:14 PM
It's Fedora 20 64bit fully up to date. I'll run a wireshark trace and see what I get.

Just had a thought about the crash, I am running the stable version on my W2K PC and did not stop it before I started this new one on Linux and I'm wondering if the crash could be related to a problem when the HG622 is busy on the other PC from DSLStats.

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: NewtronStar on May 30, 2014, 07:24:08 PM
Dslstat V4.54.4 has been working hard for one hour at my end with zero issues and I have to say it's very nippy indeed, the older versions took ages to sample 6 - 7 seconds now it's a blink of an eye.

Thankyou very much Roseway.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 30, 2014, 07:27:07 PM
It's Fedora 20 64bit fully up to date. I'll run a wireshark trace and see what I get.

Just had a thought about the crash, I am running the stable version on my W2K PC and did not stop it before I started this new one on Linux and I'm wondering if the crash could be related to a problem when the HG622 is busy on the other PC from DSLStats.

That's certainly a possibility if their sampling periods overlap. Depending on how the modem responds to that situation, DSLstats may not like it.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 30, 2014, 07:27:42 PM
Dslstat V4.54.4 has been working hard for one hour at my end with zero issues and I have to say it's very nippy indeed, the older versions took ages to sample 6 - 7 seconds now it's a blink of an eye.

Thankyou very much Roseway.

Thank you NS.
Title: Re: DSLstats pre-release version 4.54.4
Post by: burakkucat on May 30, 2014, 08:34:51 PM
I will agree with N*Star's description of very nippy. With earlier released versions, when I saw the Sampling tell-tale I could scan across the entire width of the displayed graph for a mental snap-shot and re-focus on the tell-tale before it disappeared. Now using PRVersion 4.54.4 the tell-tale has been displayed and gone almost before I have a chance to react.  :)
Title: Re: DSLstats pre-release version 4.54.4
Post by: HighBeta on May 30, 2014, 10:09:02 PM
Its excellent.

Thanks roseway! :)
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 30, 2014, 10:36:12 PM
Thank you all :)
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 31, 2014, 08:35:23 AM
Been doing a bit more testing and it seems so far I cannot recreate the crash. One thing though is that I always get a no data message on the first sampling after starting the program and the second one works. Now I'll leave it running to see if it still fails periodically. This is still Linux 64bit, I've not yet tested it on W2K.

Stuart

Update: after an hour it got the no data message again so at least for me it is not fixed.
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 31, 2014, 09:41:18 AM
Eric I have recreated the no data on first program start and captured a trace with wireshark which is attached. Hope this might help decide what the issue is. Should say that the IPs you need are 192.168.0.6 (Linux PC) and 192.168.0.1 (HG622).

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 09:57:16 AM
Thanks for that, Stuart. I'm not familiar with Wireshark, and there are a lot of unreadable characters in that file. What do you use to read it?

Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 31, 2014, 10:10:28 AM
Thanks for that, Stuart. I'm not familiar with Wireshark, and there are a lot of unreadable characters in that file. What do you use to read it?

Sorry Eric I just assumed you would have it installed. I use Wireshark, at least for Fedora it is in the standard repositories. I believe it is available for Windows as well.

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 10:17:29 AM
I'll install it and see where that takes me.
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 31, 2014, 11:26:46 AM
BTW I'm now running the 32 bit windows version on W2K and that fails the 1st sample as well. I'll leave it running and see if it also fails periodically.

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 11:46:57 AM
One question, Stuart: do you have any custom commands set to run at various times? I've realised that custom commands don't work in this version, and I've fixed it now.
Title: Re: DSLstats pre-release version 4.54.4
Post by: broadstairs on May 31, 2014, 12:03:03 PM
One question, Stuart: do you have any custom commands set to run at various times? I've realised that custom commands don't work in this version, and I've fixed it now.

Yes I do so I guess that's why I still have a problem. It gets the IP address once per hour and on start up.

Stuart
Title: Re: DSLstats pre-release version 4.54.4
Post by: les-70 on May 31, 2014, 12:06:24 PM
  All has been fine for me but I can confirm that the custom commands don't work in this version. 

One thing puzzles me, but it is nothing new, and been like it for a while although I am not sure when it first appeared.  My CRC and FEC errors plots seem to have acquired spikes/vertical lines which may be associated with some but not all standby's.  They appear over the full high of the plots but fortunately they seem to be ignored by the scaling.  A result of the scaling ignoring them is that they don't worry me much.

 Likewise I keep puzzling over whether it might be better to have errors per sample rather than per minute.   Both have interpretation issues with high sampling rates but errors per sample might be better on the graphs.

   As usual many thanks for all the tremendous work.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 12:55:32 PM
@Stuart: Yes, that would seem to be the problem. It's fixed now, so the next version should be OK.

@les-70: As you know, I've been very unsure about standby ever since you first raised it. I suspect that the reason the large spikes don't cause a rescale is that they are actually negative values being wrongly generated and displayed. If I'm right, I should be able to skip them altogether. Concerning the per-sample or per-minute question, I'll look at the possibility of making it an option.
Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on May 31, 2014, 01:06:18 PM
Still having problems here Eric. First time I started it I got a crash List index out of bounds (I think) and now it says no data after a restart. Linux 64 bit version. It does seem to get data after a while though and it has not crashed again (yet!).

Stuart

I dont know if this info helps Eric

Ive had dslstats crash on me a few times now (I think I mentioned it in another thread somewhere).   The times it occurs is when I do a manual run on hg612 modem stats (eg do a 7day graphpd).   If dslstats is running at the time modemstats doesnt collect any data, but dslstats will just 'die'.   You wont notice that dslstats has died at first because the icon still looks ok in the system tray.   Its only later when you open dslstats that you realise there is an error message and its lost any data.

More recently its been discussed via email to BE because I thought it was perhaps only happening to me whilst I was doing rather a lot of testing for modemstats and force running the prog.  I told BE not to worry about it and Id try to ensure that for future test runs Id pause off dslstats first.

Anyhow the crux of what Im trying to say... is that perhaps its possible that dslstats crashes if another program tries to harvest stats at the same time (ie you have 2 versions running on different machines).

I'll try to see if I can force a crash again... and if so I'll attach any screen caps.
Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on May 31, 2014, 01:22:25 PM
Ive just tried repeatedly to force a crash but cant.   However I did notice that during one of the cmds windows I briefly saw something like


*********************************
DSLstats is running - waiting

**********************************

However it flashed by so quick I couldnt see properly and it may have even said something else - maybe even HG612 stats is running.   I have deliberately have error logs turned off atm so cant find it in the logs to check. 

I think it must depend on split second timing and catching dslstats at the wrong second somewhere around 35 secs past. 
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 02:50:38 PM
It is true that a crash can occur when another program tries to gather data at the same time, and this does depend on critical timing. I'm not sure if I can totally avoid it happening, but I'll have another look at it.
Title: Re: DSLstats pre-release version 4.54.4
Post by: rhohne on May 31, 2014, 03:05:21 PM
Can any one confirm that the Target SNRM offset is actually dropping the ADSL connection as when I try it on a DG834GT the ADSL connection is not dropped

It appears that "adslctl configure --snr 100" is sent but the telnet connection is immediately terminated. Manually sending the command results in the expected disconnection.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 04:32:13 PM
You're right, it doesn't work. It will be fixed for the next version.
Title: Re: DSLstats pre-release version 4.54.4
Post by: NewtronStar on May 31, 2014, 07:10:51 PM
You're right, it doesn't work. It will be fixed for the next version.

I do hope you can still keep the sampling rate as it is in this version after fixing stuff for the next version as I am impressed by the speed, I also have HG612_Modem_stats running in the background and have yet to see any crashes/errors in 14 hours.

Here are my specs E6700 (DualCore Intel Pentium, 3200 MHz (12 x 267)
Vista Home premium 32bit and Windows 8.1 Pro 32bit
4GB of DDR3 Ram
HG612 and BT Home Hub 3 Version 4.7.5.1.83.8.94.1.37 (Type A)
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 08:11:43 PM
Yes, my plan is to base all new versions on this series. I just need to tidy up the remaining issues reported by testers.
Title: Re: DSLstats pre-release version 4.54.4
Post by: burakkucat on May 31, 2014, 08:24:23 PM
The only thing that still "bugs" me is the legend on the y-axis of the QLN graph.  ::)

The graph, as an entity, shows dBm versus Hz (where Hz is just an alias for Tone). The y-axis is dBm and the x-axis is Hz (alias Tone).
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on May 31, 2014, 08:32:03 PM
When I checked this before, I managed to convince myself that the reported QLN values are dBm/Hz, i.e. the noise power in dBm divided by the frequency. Most of the sources I found do plot it that way. I would really like to see a definitive reference.
Title: Re: DSLstats pre-release version 4.54.4
Post by: burakkucat on May 31, 2014, 11:07:53 PM
Hmm . . . I shall have to put it on my "search" list.

Signal power (as dB relative to 1 mW) divided by the frequency (Hz) versus frequency (Hz) . . .  :-\  ???
Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on June 01, 2014, 05:19:27 PM
I just went to look and dslstats, and I noticed that it had crashed.
 
I attach the relevant screen caps.   Because the icon sits in the system tray I hadnt noticed it had crashed, but judging from the last stats gathered, it last reported the 2hr stats yesterday at 13:06:33 meaning that it crashed some point inbetween then and 2 hrs later.

I dont know if it makes any difference but those graphs at 13:06:33 would have been snapshots that I manually captured knowing that it may crash if I manually ran HG612modem stats.  The last scheduled 2 hr run I have is at 11:36:38.

Not sure if the next schudule would have been due to run therefore at 13:36 if this narrows the time frame down further.
Looks like I may have possibly managed to make it crash.   I had the DSLstats window open when I was running, but minimised it to system tray at about 13:22 when I made the above post. I didnt look at dslstats again until now, which is why I hadnt noticed earlier.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 01, 2014, 07:05:00 PM
It does look as though the crash was probably the result of a conflict. I will try to find some way to make it less susceptible. I suppose I could check for the presence of HG612-Modem-Stats before each sample. I'll give it some thought.
Title: Re: DSLstats pre-release version 4.54.4
Post by: NewtronStar on June 01, 2014, 08:05:37 PM
Its strange Kitz your system is having issues with the pre-releace version of Dslstats as it's working very well here on my system, to me it sounds like some configuration settings with Both Dslstats and HG612_Modem_stats or OS, you could use the Event Viewer and look at the windows logs application and system to see if there is any failures at the time range you think the crash happened.

the only time I get a Division by Zero is when i adjust the sampling rate from the default of 1 minute to say 20 seconds but it sorts itself out in on the next sample of 20 seconds.
Title: Re: DSLstats pre-release version 4.54.4
Post by: Bald_Eagle1 on June 01, 2014, 08:56:59 PM
I don't think that plotting 7 days of ongoing stats via HG612 Modem Stats is the issue as the modem isn't accessed.
The graphing program simply obtains data from the modem_stats.log file.

Scheduled stats harvesting for ongoing stats and/or current (snapshot) stats is usually completed well before 35 seconds past the minute.

However, running HG612_current_stats.exe manually, either directly or via the GUI to harvest & optionally plot snapshot data could cause a clash if the timing coincides with DSLStats harvesting data at the same time.

There's a bit of a lead in period, creating folders etc., before the start of the snapshot stats harvesting & on my 'slow' system, harvesting itself takes around 3 seconds.

It may take longer if 0.25 second or 0.50 second pauses have been introduced to slow down faster PCs.


So, if manually running HG612 Modem Stats' snapshot harvesting/graphing whilst DSLStats is running, it might be wise to do it before say 25 seconds past the minute or after 45 seconds past.


Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on June 01, 2014, 09:18:02 PM
>>> Its strange Kitz your system is having issues with the pre-releace version of Dslstats

This isnt limited to the pre-release version.  Its happened with previous versions.  The only reason I mentioned it in here because it may have been relevant to what broadstairs was reporting in this thread.  It happens if DSLstats co-incides with HG612 modem stats snapshot.  So it could be happening because broadstairs is running dslstats on 2 PCs, thereby they could be clashing too.

>>> to me it sounds like some configuration settings with Both Dslstats and HG612_Modem_stats or OS

Everytime its happened I know its been a result of a conflict with both progs running.
That is...aside from today/yesterday .... because I dont know when that happened other than it was some time after 13:06 yesterday.


>>> you could use the Event Viewer

Unfortunately, that doesnt help.   The time reported in Event viewer isnt until I actually attempt to open dslstats from my sys tray.   Its not until I re-open it from minimised will it be recorded in event viewer.


So this is what I currently see

Quote
01/06/2014 17:02:45

Application Error Event ID 1000 dslstats.exe 00000000   unknown   0.0.0.0

I actually had nothing at all in Event viewer yesterday other than gupdates, notification of backup complete, and sys retore point created.


Yet dslstats actually stopped running and recording any data more than 24hrs previously. The last data recorded by dslstats was 31/05/2014 13:06:33
 
Although a conflict causes dslstats to stop recording.. it wont throw an error until you re-open the dslstats window.
Its caught be out several times now as I think its running fine, when the icon is still there minimised in sys tray.


Bear in mind I was trying to deliberately make it crash yesterday at around 1pm..  but because DSLstats window remained open I thought all was ok.  I then minimised dslstats to sys tray and made the post at 01:22:25.   It wasnt until 5pm today that I realised dslstats hadnt been recording for the past 24 hrs.

I'd been doing some additional monitoring for BE, which is why Id been running graphpd & snapshots more often than the average user would be... and thats why Im seeing more crashes.   Ive just not mentioned it to eric before, because I know its a manual run of dslstats when the time co-incided that causes the crash.  If broadstairs hadnt have reported similar, I probably wouldnt have said anything to Eric as he has enough to do & I was just attempting to help with broadstairs issue.

In fact.. this is what I said to BE the last time I caused dslstats to crash & HG612 modem stats to throw errors.

"I wouldn’t worry too much about it BE,  It was my fault I was testing and forgot to disable dslstats... or check the time.   I realised straight away what Id done. It does crash DSLstats without warning though."

So the crux is, if DSLstats is running and another prog tries to access your router for linestats, then dslstats crashes without warning, and you wont know its crashed until you next try to restore it from sys tray.


---

ETA

BaldEagle posted whilst I was doing, but Ive left the post as was.
Title: Re: DSLstats pre-release version 4.54.4
Post by: NewtronStar on June 01, 2014, 11:18:14 PM
Kitz would it not be a good idea to focus your beta testing for BE1 HG612_Modem_Stats and disable Dslstats until BE1 is satisfied that Modem_Stats is working as it should on your system as all your doing is confusing each programers program and they don't know if it theres side or the others thats causing your issue (1 step at a time)  :)

Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on June 02, 2014, 12:03:00 AM
Ummmmm, the testing for HG612 stats is now complete..   I was trying to avoid clashes.

Quote
It was my fault I was testing and forgot to disable dslstats... or check the time.   I realised straight away what Id done. It does crash DSLstats without warning though.

Shouldnt have said anything in this thread..  I was only trying to help point out what could have been the cause of broadstairs problems from what Id noticed in the past. :(

I wouldnt have said anything in this thread apart from the fact Id seen broadstairs post..   thought I was trying to help, because there will be others that run into the same problem at some point.   I was trying to identify what happens so that perhaps in future it could be avoided.   There are many people who run both progs.


---
Edited to clarify

<< It does crash DSLstats without warning though.>>

What I meant by that is that DSLstats crashes... and there is no warning that it has crashed.
Was going to attempt to do some further tests tonight, but obviously if no one is bothered that the 2 cause conflicts... then theres not much point me wasting my time.


Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 02, 2014, 10:09:35 AM
Just to be clear, I am interested in the problem of conflict crashes, and I'm beefing up the error trapping and logging to try to resolve it. Please do continue to report these events.
Title: Re: DSLstats pre-release version 4.54.4
Post by: Bald_Eagle1 on June 02, 2014, 10:22:02 AM
I'm also interested.

Perhaps we should jointly confirm/agree how best to deal with this, otherwise we may unwittingly introduce a problem.

I'll provide some suggestions tonight as I'm setting off to work now.


Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 02, 2014, 10:26:11 AM
I'll be interested to hear your thoughts.
Title: Re: DSLstats pre-release version 4.54.4
Post by: krypton on June 02, 2014, 07:36:08 PM
...
- login should now work with modem/routers which don't require username and password to log in
...

I can confirm this works now. Thanks.

I tried the new pre-release versions several times. It is truly much faster, on my vdsl2 line one sampling period takes between 1 and 1.5 seconds with all items enabled. The old version needs about 10 seconds.

One time it worked for about 10 minutes without an error on linux 32 bit.

But almost always it does not work and this happens: After start "sampling" stays permanent on during the configured sampling time and "Previous sample still being processed" is displayed. No graphs or status bar informations is visible. This repeats over and over.
With wireshark I can see that every sampling period the --stats data is successfully transmitted but after this, nothing more is sent or received.
If this happens dslstats can't be closed properly. It doesn't freeze but the close buttens don't react or "closing..." stays on forever. It's the same on linux and windows versions.

Some people seem to have problems with multiple graphing programs running at the same time. I tried this an let 5 instances of dslstats (4.53.3) sampling simultaneous. They worked perfectly with no errors.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 02, 2014, 08:05:43 PM
Thanks for that report. I think I can see what's causing that, and I'll fix it.
Title: Re: DSLstats pre-release version 4.54.4
Post by: Bald_Eagle1 on June 03, 2014, 01:52:15 AM
I'll be interested to hear your thoughts.


Please correct me if any of this is incorrect, but this is how I think DSLStats works when HG612-Modem-Stats co-operation is selected:-

DSLStats will always start sampling at 35 seconds past the minute.
If any of my programs are running at that time, DSLStats will abandon that sample & try again at 35 seconds past the next minute.

Depending upon the PC's speed, sampling with your new method can take between 3 & 5 seconds.

if my programs (or any other program) attempt to log in to the modem during DSLStats' sampling period, DSLStats will currently crash (or continue looping).

I assume your new sampling method is similar to the method employed by my programs. i.e. a login/password/ATP> prompt or the '#' or '>' characters are awaited & then login details or a stats command are sent to the modem with the resulting output used for the next stage of logging in or for the stats data.

If another program 'butts in', it could  affect the expected output, which is the root cause of the problem.


I can think of 3 possible options at the moment to hopefully avoid this issue:-

1) DSLSTats could run a 'dummy' process immediately before sampling & end it immediately after sampling has completed.

My programs could check for that process & if it is present, they will keep checking until the process has ended, at which time my programs will continue to log in etc.

If 'too much' time has elapsed, my program could try to log in & if it fails, my program will exit rather than keep trying indefinitely.



2) My programs could check for DSLStats running & assume that sampling will start at 35 seconds past the minute & complete by 40 seconds past.

If my programs are started just before 35 seconds past the minute they could simply wait until 40 seconds past before continuing.

If they are started as say 36 seconds past the minute i.e. after DSLStats has started sampling because it hadn't detected my programs, they could again wait until 40 seconds past before continuing.



3) In the extremely unlikely event that DSLStats sampling & my programs' sampling start at exactly the same time, they could both fail to log in or obtain valid data.

Kitz & I think that happened once while my program was at the login password prompt stage, but she was intentionally attempting to cause a clash at the time in order to provide us both with useful feedback.
My program caught that event & after pausing & trying again a few time it eventually exited accordingly, with a logged error message:-

31/05/2014 13:15:36.75   - CURRENT-ISRUNNING-131536-710 - Start of [HG612_current_stats.exe] *** Version 2.11 Beta - 26/05/14 ***
31/05/2014 13:15:45.01 - CURRENT-ISRUNNING-131536-710 - **** expect("ssword:") FAILED!!! - exiting. Status = 1.

I think the output from the modem for DSLStats's data harvest interfered with the 'expected' password prompt for my program.


There have also been a couple of times when my program continued unhindered, but it caused DSLStats to fail.

We are dealing with minute but critical fractions of a second here so accidental clashing is very unlikely in normal scheduled logging from both sets of programs.

I recently improved error handling in my programs so that if a preset number of attempts to log in or obtain valid data fail, they will now exit rather than continue vainly looping indefinitely.

Previously if one of my programs ended up 'stuck' DSLStats would detect its presence & after 5 continuous instances where sampling would be abandoned it would wait for 30 minutes before trying again, to no avail.

Also previously, if DSLStats caused any of my programs to 'stick' none of our programs would manage to obtain data until the 'stuck' processe(s) were ended.



As mentioned above, please feel free to corect me if I have misunderstood how DSLStats operates.


Do you have any other thoughts on how we could avoid this issue or at least deal with it more robustly than at present?

 
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 03, 2014, 08:40:05 AM
Quote
DSLStats will always start sampling at 35 seconds past the minute.
If any of my programs are running at that time, DSLStats will abandon that sample & try again at 35 seconds past the next minute.

It actually can start between 35 and 40 seconds past the minute. If you press the green button between those times it will start sampling immediately, otherwise it will wait until the next 35 seconds past the minute. It will skip a maximum of 5 samples in succession, after which it will bypass the check for 30 samples. With the new networking library, sampling takes between 1 and 5 seconds (generally less than 3 seconds).

Quote
I assume your new sampling method is similar to the method employed by my programs. i.e. a login/password/ATP> prompt or the '#' or '>' characters are awaited & then login details or a stats command are sent to the modem with the resulting output used for the next stage of logging in or for the stats data.

That's what it does. After each message is sent to the modem, it waits until the prompt returns before sending the next message. It automatically detects what the prompt is.

Quote
If another program 'butts in', it could  affect the expected output, which is the root cause of the problem.

There is some sort of critical timing issue associated with this, and no doubt other factors as well, so the best solution will be to try to avoid clashes altogether.

Quote
1) DSLSTats could run a 'dummy' process immediately before sampling & end it immediately after sampling has completed.

Yes, I think I could do that. I'll have to think about what form the dummy program will take (it needs to sleep while running so it doesn't use CPU cycles) but I should be able to do that.

Quote
2) My programs could check for DSLStats running & assume that sampling will start at 35 seconds past the minute & complete by 40 seconds past.

If my programs are started just before 35 seconds past the minute they could simply wait until 40 seconds past before continuing.

If they are started as say 36 seconds past the minute i.e. after DSLStats has started sampling because it hadn't detected my programs, they could again wait until 40 seconds past before continuing.

From my viewpoint, that would be the best solution. When the "co-operation" option is set in DSLstats, I can almost guarantee that sampling won't be happening outside the times of 35-45 seconds after the minute (assuming a maximum of 5 seconds for sampling). The only exception to this would be some unexpected error condition in the program, and I'm currently enhancing the error trapping to try to eliminate this possibility. So if your programs could restrict their activities to the period from 45 seconds after the minute to the next 35 seconds after (including processing time) that would largely eliminate all clashes.

I expect I'll have some more thoughts, but I have to go off and do things now.
Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 03, 2014, 11:07:16 PM
(Continuing)

Quote
3) In the extremely unlikely event that DSLStats sampling & my programs' sampling start at exactly the same time, they could both fail to log in or obtain valid data.

Kitz & I think that happened once while my program was at the login password prompt stage, but she was intentionally attempting to cause a clash at the time in order to provide us both with useful feedback.
My program caught that event & after pausing & trying again a few time it eventually exited accordingly, with a logged error message:-

31/05/2014 13:15:36.75 - CURRENT-ISRUNNING-131536-710 - Start of [HG612_current_stats.exe] *** Version 2.11 Beta - 26/05/14 ***
31/05/2014 13:15:45.01 - CURRENT-ISRUNNING-131536-710 - **** expect("ssword:") FAILED!!! - exiting. Status = 1.

I think the output from the modem for DSLStats's data harvest interfered with the 'expected' password prompt for my program.
..
I recently improved error handling in my programs so that if a preset number of attempts to log in or obtain valid data fail, they will now exit rather than continue vainly looping indefinitely.

The changes I've recently made should result in DSLstats behaving in a similar way: while in the telnet interface, every command sent to the modem is error trapped, a timeout is included in case the command fails to return, and the results are logged. So, hopefully, if a clash does occur, the programs should be able back out without misbehaving.
Title: Re: DSLstats pre-release version 4.54.4
Post by: kitz on June 04, 2014, 12:28:36 AM
Note for BE.

Ive given Eric a copy of the steps I sent to you via email the other night of how I can repeatedly cause a clash, which causes DSLstats to display strange behaviour.   In particular the Incorrect login name or password, and not being able to close it until it displays the div by 0 error.
 
Title: Re: DSLstats pre-release version 4.54.4
Post by: Bald_Eagle1 on June 04, 2014, 12:44:46 AM
Thanks, Kitz.

I think we are now very close to ensuring 100% compatibility between the programs.

We just need to be careful to avoid fixing one issue that could introduce a different issue.

As long as Eric & I are fully aware of what we are doing with our respective programs, everything should be just fine & dandy  :)


Title: Re: DSLstats pre-release version 4.54.4
Post by: Bald_Eagle1 on June 05, 2014, 07:47:35 AM

Quote
1) DSLSTats could run a 'dummy' process immediately before sampling & end it immediately after sampling has completed.

Yes, I think I could do that. I'll have to think about what form the dummy program will take (it needs to sleep while running so it doesn't use CPU cycles) but I should be able to do that.

Quote
2) My programs could check for DSLStats running & assume that sampling will start at 35 seconds past the minute & complete by 40 seconds past.

If my programs are started just before 35 seconds past the minute they could simply wait until 40 seconds past before continuing.

If they are started as say 36 seconds past the minute i.e. after DSLStats has started sampling because it hadn't detected my programs, they could again wait until 40 seconds past before continuing.

From my viewpoint, that would be the best solution. When the "co-operation" option is set in DSLstats, I can almost guarantee that sampling won't be happening outside the times of 35-45 seconds after the minute (assuming a maximum of 5 seconds for sampling). The only exception to this would be some unexpected error condition in the program, and I'm currently enhancing the error trapping to try to eliminate this possibility. So if your programs could restrict their activities to the period from 45 seconds after the minute to the next 35 seconds after (including processing time) that would largely eliminate all clashes.

I expect I'll have some more thoughts, but I have to go off and do things now.


As a temporary measure, the programs that harvest stats (HG612_stats.exe & HG612_current_stats.exe) now check if DSLStats is running.

If not, they will continue as normal.

If DSLStats is running & the time is greater than 32 seconds past the minute & less than 45 seconds past, they will wait until 45 seconds past before moving on to login to the modem & harvest the stats.

e.g. if the time is 32 seconds past the minute, the programs will wait for 13 seconds.
      if the time is 36 seconds past the minute, they will wait for 9 seconds
      if the time is 42 seconds past the minute they will wait for 3 seonds (just in case).


My preference would still be to check for a dummy process started by DSLStats that only runs during the sampling period.
My programs would then only need to wait until DSLStats sampling had completed rather than have to wait until 45 seconds past the minute.


If my programs start just before 32 seconds past the minute, they will continue as normal as DSLStats checks for them running & either waits for them to complete or aborts sampling for that minute.

If my programs do end up stuck in a loop, failing to obtain any valid data from the modem, they should exit cleanly, also freeing up the modem to be ready for the next attempt.




The only potential problem area I can think of at the moment would be if a user has sufficiently & permanently delayed the start of my programs or perhaps has 2 instances running for testing purposes or as a belt & braces back up system, one copy starting on the minute & the other copy starting at say 30 seconds past.

In those cases from a very slow PC, it is possible that one of the programs could still be running when DSLStats is due to start sampling.



For any of these checks & clash avoidance measures to work, DSLStats & HG612 Modem Stats programs do have to be running on the same machine in order to check for each other.

Using seconds past the minute alone as the criteria for clash avoidance measures would not always work if running the programs on different machines either as each machine's system time can & occasionally does drift out of sync with system time on the other machine, leading to each machine attempting to access the modem at the same time.

This can also cause clashes when running instances of the same program on separate machines.


I have attached copies of HG612_stats.exe & HG612_current_stats.exe if anyone wishes to test the clash avoidance measures.

Evidence of avoidance action should be visible in CURRENT_ERROR.LOG_file_ERROR.TXT & ONGOING_ERROR.LOG_file_ERROR.TXT

e.g:-

**** From IsRunningVB.exe, DSLStats.exe *IS* running 1 instances - seconds past the minute = 34
**** Sleeping for 11 seconds


Title: Re: DSLstats pre-release version 4.54.4
Post by: roseway on June 05, 2014, 08:02:00 AM
Quote
My preference would still be to check for a dummy process started by DSLStats that only runs during the sampling period.
My programs would then only need to wait until DSLStats sampling had completed rather than have to wait until 45 seconds past the minute.

Later this morning I'll be uploading a new test version which does this. At the start of sampling it runs a dummy program called dslstatssampling.exe, and when sampling is complete it closes this program. As a backstop in the event of something going badly wrong, the dummy program times out after 20 seconds (perhaps I should make this 10 seconds?).