Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[NewtronStar]

Does that make any sense?

[Edit]PS: the thread I mentioned in reply #25 above has some replacement modules provided by BE to try to overcome the harvesting issue with v1.1/4.1.  Try those.  They work for me.

It does make sense the way you put it but now i'll have to put it into practise but I am still using V2.0 or upload version 0.4 and I am happy to go back to V4.1 if you have link to those replacement modules as upbove and your posts are very helpfull indeed 

[ColinS]

If you're willing to try BE's v4.1 again, apply the 3 files below in order to the Scripts subdirectory.  Files 2 & 3 replace earlier ones in File 1.  Afterwards you should have:
Set_HG612_Date_and_Time.exe dated 30/08/2013_17:11
HG612_Current_Stats.exe dated 04/09/2013_06:19, and
HG612_Stats.exe dated 04/09/2013_22:01
If you also run DSLStats, you should use v3.95 (see reply #112 in http://forum.kitz.co.uk/index.php/topic,12844.msg244301.html#new), and the synchronise with HG612 option, or (using the Windows Time Clock), start v3.94 manually at about 35s past the min.
[edited] to reflect Roseway's latest changes.

In my last post about copying the modem_stats.log:  The copy is to preserve your data for restoration later.  You will then have to edit the modem_stats.log in situ as this is where graphpd.exe expects to find it.  Alternatively, I believe that you can edit the copy elsewhere and then manually drag & drop it onto graphpd.exe, which should then open in a command windows and ask you about the desired plotting interval.

Good luck if you decide to try it, but I would understand if you decided to wait for BE's return instead.

[kitz]

The way that Ongoing stats plots works (graphpd.exe) is that it translates the requested plotting interval into the equivalent number of 1 minute samples.  So, with the 1day default, it is looking for 1440 samples.

Thanks for that info.. I wasnt aware thats how it calculated the daily stats. 
That would explain why I saw some weirdness plotting graphs over the periods when HG612_stats hadnt been recording for periods due to the stuck multiple instances in task manager.

I wonder if a slight default modification could be made to graph.exe by calling the equivalent of Date.getTime() to work out the periods to graph.

Obviously I dont have access to the source to say for sure -nor know which language its programmed in - but perhaps a small function could be added so that each day could be graphed correctly.  I suppose the hardest part would be finding the correct date/time to read back from the file, and if theres a lot of dates with missing data, would this slow things down?

[ColinS]

I wonder if a slight default modification could be made to graph.exe by calling the equivalent of Date.getTime() to work out the periods to graph.

I had already decided to email BE to suggest something similar.   He'll love that to come back to!

I suppose the hardest part would be finding the correct date/time to read back from the file, and if theres a lot of dates with missing data, would this slow things down?

I'll leave that to BE to decide, but I know that he is already concerned that a 'normal' daily plot takes almost 2mins on my (admittedly slow) server, which has caused 1-2 of the normal harvesting samples to be missed at that time.  While it does seem simple in principle, no doubt it will be more tricky with large logs (say months & months), and any arbitrary plotting interval (of days, hours or minutes).  I think you could say that this part of it is still a work in progress (since e.g. you can't give a date range, and have to make an extract from the modem_stats.log).

[kitz]

He'll love that to come back to!

I bet!  Wonder if we will see him for a while or if he'll hide 

While it does seem simple in principle, no doubt it will be more tricky with large logs (say months & months), and any arbitrary plotting interval (of days, hours or minutes).

Yes thats what concerned me..  the theory is simple, but the downside of going through a large log with several months of data to slow things down may not be such a good idea.   Perhaps a half way house solution that existing method is used for the daily ongoing stats so it doesnt clog things up.  But with a more detailed get.Time option for say up to so many days, that is disregarded for periods over a month?   I dunno..  BE knows how his prog works, so is in the best position to say what he can and cant do.   At the end of the day he puts a lot of time into it as it is..  and Im grateful for what we do get now.

[NewtronStar]

If you're willing to try BE's v4.1 again, apply the 3 files below in order to the Scripts subdirectory.  Files 2 & 3 replace earlier ones in File 1.  Afterwards you should have:
Set_HG612_Date_and_Time.exe dated 30/08/2013_17:11
HG612_Current_Stats.exe dated 04/09/2013_06:19, and
HG612_Stats.exe dated 04/09/2013_22:01
If you also run DSLStats, you should use v3.95 (see reply #112 in http://forum.kitz.co.uk/index.php/topic,12844.msg244301.html#new), and the synchronise with HG612 option, or (using the Windows Time Clock), start v3.94 manually at about 35s past the min.
[edited] to reflect Roseway's latest changes.

In my last post about copying the modem_stats.log:  The copy is to preserve your data for restoration later.  You will then have to edit the modem_stats.log in situ as this is where graphpd.exe expects to find it.  Alternatively, I believe that you can edit the copy elsewhere and then manually drag & drop it onto graphpd.exe, which should then open in a command windows and ask you about the desired plotting interval.

Good luck if you decide to try it, but I would understand if you decided to wait for BE's return instead.

Have done all that to-day but V4.1 is still missing 4 mins samples per 10 mins I doubled checked and triple checked the task manager and it only shows 1 task called HG612 stats program and it repeats every 1 minute but some samples don't get through using Microsoft Windows security essentials but I very much dought thats the cause, CPU is dual core 3.20GHZ so thats not the problem but V2.0 harvest's every minute but V4.1 keeps missing 4 mins out of 10

I have to point the finger at the software also have a fresh install of windows 8 and it does the same or could it be my hardware ?

This is a fresh log using V4.1 ->

[ColinS]

Thank you for trying that for us NewtronStar.

Could I possibly ask you to zip up the following files & post them here for us, if possible.  I will then, in BE's continuing abscence, attempt to take a look at them to see if I can spot anything that might give us a clue about what's going on.

First of all, all of these files will be in use, so please disable all logging using the settings editor while you make copies of them.  Thank you.

the files that would be useful to see are:
In the Ongoing Stats subdirectory -
ERROR.LOG_file_ERROR.TXT
ERROR.LOG
and an extract of the modem_stats.log file from when you started running with these debugging modules earlier today
In the Scripts subdirectory,
Login_events.txt

Do you ever have any files in the Scripts subdirectory with filenames like ONGOING-ISRUNNING-172044-106.TXT?
If so could you zip those too please?

Can't promise to spot the problem, but I can promise to look carefully at it for you.

HTH 

[NewtronStar]

Thank you for trying that for us NewtronStar.

Can't promise to spot the problem, but I can promise to look carefully at it for you.

HTH 

OMG that's a big list  I'll do my best !

1. logging disabled
2. Error.Log file from Ongoing stats attachment below
3. Modem_Stats.log file/login_events.txt attachment below

and thanks for looking into this for me as I have nearly given up hope after 3 months.

edit here is the Error.txt file below

[ColinS]

Hi NewtronStar,

I have sent you a PM with the full analysis of what I could see, but briefly, here are the conclusions:

It seems that you are missing samples because HG612_modem_stats is occasionally failing to read data from the modem leading it to believe that the modem has resync'ed.  It's not possible yet to tell why that is happening, and it may require BE's return to resolve it fully, but I have asked you for some additional information in case I can spot anything else.

I attach your plot.

[NewtronStar]

It seems that you are missing samples because HG612_modem_stats is occasionally failing to read data from the modem leading it to believe that the modem has resync'ed.  It's not possible yet to tell why that is happening, and it may require BE's return to resolve it fully, but I have asked you for some additional information in case I can spot anything else.

I attach your plot.

Hello ColinS

Many thanks for the PM and read every word it looks like I am in good hands here 
to-day I have disabled firewalls and security programs to see if it made any difference but no change, and it still remains a fresh log as I don't see any point merging old & new logs until we find the cause of the missing data.

Now it was my worry that for some reason the PC to Router (HH3) to Modem (HG612) may be causing some samples to be ignored , but it did work fine in V2.0 (older stats software) and nothing in that Hardware department has changed in 14 months.

Very much appreciated for looking into this odd issue.

[NewtronStar]

Very much appreciated for looking into this odd issue.

Edit here is the other Plink log sorry had to split it up due to attachment limit

[ColinS]

Thank you for that additional information NewtronStar.

Again, I have PM'ed you with the full analysis, but here is a summary:

As I concluded above, (far too often to be readily explainable) when hg612_stats.exe is calling xdslcmd info --stats, no data is being returned.  This causes BE's code to detect a false positive resync event, which in turn causes it to take a post-resync snapshot ~7secs later in which it issues exactly the same xdslcmd info --stats, which this time works!
See here for an example:

Code: [Select]

14/09/2013 20:02:01.829 - About to reply(xdslcmd info --stats)
14/09/2013 20:02:01.829 - reply(xdslcmd info --stats) O.K.
14/09/2013 20:02:01.843 - ERROR - get_data() FAILED!!!! - exiting the program   <<<<<---------- the smoking gun! from hg612_stats
14/09/2013 20:02:01.855 - Current  sync speeds are     DS 0 kbps      US 0 kbps <<<<<---------- no data
14/09/2013 20:02:01.855 - Previous sync speeds are     DS 0 kbps      US 0 kbps
=~=~=~=~=~=~=~=~=~=~=~= Plink log 2013.09.14 20:02:09 =~=~=~=~=~=~=~=~=~=~=~=
xdslcmd info --stats     <<<<<----------- exactly the same modem command from graph6
xdslcmd: ADSL driver and PHY status
Status: Showtime
Retrain Reason: 0
Max: Upstream rate = 6546 Kbps, Downstream rate = 32116 Kbps
Path: 0, Upstream rate = 6569 Kbps, Downstream rate = 27337 Kbps
The only time BE and I have seen this happen previously, we concluded (or assumed, as we couldn't prove it) that perhaps overlapping instances of xdslcmd info --stats calls had occured, and so only one of them got the data.  ATM that's all I can think it is, but I can't quite see why this is happening, unless (and I have absolutely no idea and I am now clutching at straws) perhaps the HH3 queries the HG612 as well?  Is that a possibility does anyone else know? 

Apart from that I don't think I can extract any more information from the available diagnostics, and we shall have to await the return of our much missed feathered friend.

[waltergmw]

@ Colin S,

Speaking as a novice in these matters, perhaps we should note that dual-port access to a single resource is unlikely to have been fully tested by the manufacturers. It is conceivable that a "busy flag" check has been omitted.

Perhaps we should also record the types of router involved as well ?

Kind regards,
Walter

[ColinS]

perhaps we should note that dual-port access to a single resource is unlikely to have been fully tested by the manufacturers. It is conceivable that a "busy flag" check has been omitted.

I suppose that's entirely possible Walter.  I'll pass on the observation to BE if he doesn't get round to reading it here.  Working around it is the key.  Coexistence between HG612_modem_stats and DSLstats seems to be there now, so that has been >90% of the problem for both him & Roseway.  People just may need to be aware if they are manually telnet-ing into the modem.

Perhaps we should also record the types of router involved as well ?

Yes, that is a very good suggestion.  If people reporting issues similar to NewtronStar's (regular missing samples) could do that, then perhaps a pattern could be established, yes.  ATM, in the absence of anything else to be suspicious of, I have a mental query about the extent to which the BTr HH is aware of the modem. 

[NewtronStar]

Perhaps we should also record the types of router involved as well ?

Yes, that is a very good suggestion.  If people reporting issues similar to NewtronStar's (regular missing samples) could do that, then perhaps a pattern could be established, yes.  ATM, in the absence of anything else to be suspicious of, I have a mental query about the extent to which the BTr HH is aware of the modem. 

I could directly connect the PC to HG612 and bypass the BT Home Hub 3 and see if it fix's the missing samples, PC is connected to lan port 4 on HH3 and Wan from HH3 is connected to HG612  lan port 1 and Hg612 Lan port 2 is then connected to HH3 lan port 3
So will it work if I connect PC directly to Lan port 2 on the HG612 ?