Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: 1 2 [3] 4 5 ... 8

Author Topic: HG612 modem stats - multiple instances.  (Read 38617 times)

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: HG612 modem stats - multiple instances.
« Reply #30 on: August 26, 2013, 03:11:05 PM »

Ok will download it.  Thank you vm :)

btw previous fix has been running for the best part of an hour and touch wood seems to work fine.

The error.log_file_ERROR.txt reports quite a few of the trying (sh) again.   The maximum it got up to was 34, but as expected it did decrease on each subsequent run.



Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: HG612 modem stats - multiple instances.
« Reply #31 on: August 26, 2013, 03:21:15 PM »

I think Paul has put his finger on the critical issue. It's not just the simultaneous logins which are the real problem, but their timing. I don't think I can make the timing of DSLstats' sampling any more accurate than it is unless I somehow synchronise to the Windows scheduler, and I have no idea how to do that. In any event, it's probably not really practical to have two independent programs trying to sync together.

The workaround of repeatedly trying the sh command until it gets a response may deal with the problem, but it would be much better if we can find a way of avoiding it before it happens. In general, DSLstats should handle situations where it can't log in for any reason - it should report the event in the event log and wait until the next sample is due. Perhaps what I should also do in this circumstance is delay the sampling by a few seconds so it doesn't get a succession of failed logins.

Although the total 'Sampling' period appears to be between 8 and 15 seconds, at least some of that time is in processing the data after logging out. For the purposes of what's being discussed here, only the telnet login - logout period matters. I should be able to raise some sort of flag during this period, which HG612-modem-stats could read and use to avoid a clash. The simplest flag would be a temporary file saved somewhere; we would need to agree on a suitable location for saving it, and also a protocol for dealing with the possibility that the temporary file doesn't get deleted at the end of sampling as a result of some unexpected event.
Logged
  Eric

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: HG612 modem stats - multiple instances.
« Reply #32 on: August 26, 2013, 03:46:47 PM »

I think Paul has put his finger on the critical issue. It's not just the simultaneous logins which are the real problem, but their timing. I don't think I can make the timing of DSLstats' sampling any more accurate than it is unless I somehow synchronise to the Windows scheduler, and I have no idea how to do that. In any event, it's probably not really practical to have two independent programs trying to sync together.


Maybe using Windows system time would work as that's what drives the Windows Task Schedules (and any other user defined schedules within HG612 Modem Stats).

So, as my programs run on the minute by default, perhaps DSLStats could always start sampling at say, 15 or 30 seconds past the minute at 1 minute intervals?

That should also allow for any additional time when HG612_current_stats occasionally harvests data on its few hourly schedule (the most user definable frequency is one hour).

The logging element of my programs CAN be delayed via user settings, to avoid clashes with anything else, but that depends on fixed timings for anything they wish to avoid.



 

Quote

The workaround of repeatedly trying the sh command until it gets a response may deal with the problem, but it would be much better if we can find a way of avoiding it before it happens. In general, DSLstats should handle situations where it can't log in for any reason - it should report the event in the event log and wait until the next sample is due. Perhaps what I should also do in this circumstance is delay the sampling by a few seconds so it doesn't get a succession of failed logins.


It does seem to do that:-

26 Aug 2013 12:08:03   Second stage login failed

I think that happened a couple of seconds into DSLStat's sampling period.


My programs, especially the current stats can be run on demand at any time a user chooses.
Retaining a workaround for those occasions is probably wise.

Quote
Although the total 'Sampling' period appears to be between 8 and 15 seconds, at least some of that time is in processing the data after logging out. For the purposes of what's being discussed here, only the telnet login - logout period matters. I should be able to raise some sort of flag during this period, which HG612-modem-stats could read and use to avoid a clash. The simplest flag would be a temporary file saved somewhere; we would need to agree on a suitable location for saving it, and also a protocol for dealing with the possibility that the temporary file doesn't get deleted at the end of sampling as a result of some unexpected event.

TBH, adding a slight delay to DSLStats if either of my programs are already running seems a simpler/tidier solution to me as users may have installed DSLStats in any folder of their choosing.


If, for whatever reason my programs still can't log into the modem, they now exit gracefully, freeing memory & closing open sockets etc.

Worst case scenario seems to be that 1 sample is missed, similar to how DSLStats handles a failed login.
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: HG612 modem stats - multiple instances.
« Reply #33 on: August 26, 2013, 04:20:26 PM »

Quote
users may have installed DSLStats in any folder of their choosing.

Sure, but that doesn't inhibit DSLstats from writing a temporary file to any specified location.

However, if you're happy with my adding a delay to DSLstats sampling if it detects a clash, then that's easy to implement, and I'll add it to the next version. There are a couple of minor tweaks and bugfixes which are already in place, so I'll add this one and release it as a minor version upgrade.

I'll investigate using the Windows timer for the Windows version of DSLstats, but as I know nearly nothing about Windows programming that will take a little longer.
« Last Edit: August 26, 2013, 04:26:02 PM by roseway »
Logged
  Eric

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: HG612 modem stats - multiple instances.
« Reply #34 on: August 26, 2013, 04:38:26 PM »

Quote
users may have installed DSLStats in any folder of their choosing.

Sure, but that doesn't inhibit DSLstats from writing a temporary file to any specified location.


That option may indeed turn out to be necessary, but in attempting to automate as much as possible, I'd rather treat it as a maybe rather than essential for the timebeing

Quote
However, if you're happy with my adding a delay to DSLstats sampling if it detects a clash, then that's easy to implement, and I'll add it to the next version. There are a couple of minor tweaks and bugfixes which are already in place, so I'll add this one and release it as a minor version upgrade.

Shall we do that for now & see how it goes?

For curiosity, would you be aiming to detect my programs' processes or modem logging in process?


The 2 main programs that access the modem are HG612_stats.exe & HG612_current_stats.exe.

I have also just remembered that whenever a resync is detected, SET_HG612_DATE_AND_TIME.exe also runs as soon as DSL is up, in order to reset the modem's internal clock.
That one is only necessary for VDSL2 connections when the modem is in its default bridge mode.
It's not really needed for ADSL connections when in router/modem mode as it synchronistes with a choice of time servers.

FWIW, there have been a few calshes since I started DSLStats sampling again at 09:20 this morning & only a couple of them resulted in Second stage login failed.

Maybe a delay in DSLStats would adequately address that minor issue.



 
Logged

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: HG612 modem stats - multiple instances.
« Reply #35 on: August 26, 2013, 04:46:11 PM »


I'll investigate using the Windows timer for the Windows version of DSLstats, but as I know nearly nothing about Windows programming that will take a little longer.


You probably know just as much as (or more than) I do then  :)

With assistance from others, I did work out how to use the PC's system time.
There are a number of options for that - rough & ready or accurate to 1/1000 of a second.

Let me know if you would like me to pass on any pointers that I have stumbled onto along the way.

Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: HG612 modem stats - multiple instances.
« Reply #36 on: August 26, 2013, 06:50:41 PM »

Quote
For curiosity, would you be aiming to detect my programs' processes or modem logging in process?

At present I'm keeping it simple and just detecting any circumstance in which DSLstats fails to log in. I'll think about detecting specific processes a bit later. That's easy on Linux, but something else I don't know how to do on Windows.

Quote
FWIW, there have been a few clashes since I started DSLStats sampling again at 09:20 this morning & only a couple of them resulted in Second stage login failed.

Second stage login failure isn't necessarily a clash. I see it very occasionally on my system when DSLstats is the only monitoring program running. I've tended to assume that it's a glitch of come sort, a critical timing issue perhaps, or just a non-response from the router.

Quote
Let me know if you would like me to pass on any pointers that I have stumbled onto along the way.

Thanks for the offer, but I think it's actually very simple in Lazarus. There's a function "Now" which returns the current date and time. I haven't checked how it's actually implemented in the underlying code, but it seems fairly certain that it actually returns the Windows system time when running on Windows. I can check that quite easily with a little test program.
Logged
  Eric

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: HG612 modem stats - multiple instances.
« Reply #37 on: August 26, 2013, 07:14:25 PM »

Report from me whilst I was out.  Nothing intensive was running on the PC - only mail, DSLstats & HG612stats.
DSLstats is set to report every 60 seconds (not its default 30secs)



HG612 stats has been logging fine and I cant see any missed events.
Its been through 2 complete cycles of retry attmpts with the longest being 40 retries so far.

14:10 - 14:33 & 17:24 - 17:52

retry attempts vary, but very roughly about 20-30ms on average?  I'll attach a log sample so you can see

Code: [Select]
26/08/2013 17:24:00.37 - In HG612_stats.exe - At the start of the main() function,
26/08/2013 17:24:00.43 - There are 1 instances of HG612_stats.exe running. Status = 0,
26/08/2013 17:24:01.65 - 01 trying reply(sh) again.
26/08/2013 17:24:01.88 - 02 trying reply(sh) again.
26/08/2013 17:24:02.10 - 03 trying reply(sh) again.
26/08/2013 17:24:02.33 - 04 trying reply(sh) again.
26/08/2013 17:24:02.56 - 05 trying reply(sh) again.
26/08/2013 17:24:02.79 - 06 trying reply(sh) again.
26/08/2013 17:24:03.01 - 07 trying reply(sh) again.
26/08/2013 17:24:03.24 - 08 trying reply(sh) again.
26/08/2013 17:24:03.46 - 09 trying reply(sh) again.
26/08/2013 17:24:03.69 - 10 trying reply(sh) again.
26/08/2013 17:24:03.92 - 11 trying reply(sh) again.
26/08/2013 17:24:04.15 - 12 trying reply(sh) again.
26/08/2013 17:24:04.38 - 13 trying reply(sh) again.
26/08/2013 17:24:04.61 - 14 trying reply(sh) again.
26/08/2013 17:24:04.83 - 15 trying reply(sh) again.
26/08/2013 17:24:05.06 - 16 trying reply(sh) again.
26/08/2013 17:24:05.28 - 17 trying reply(sh) again.
26/08/2013 17:24:05.50 - 18 trying reply(sh) again.
26/08/2013 17:24:05.72 - 19 trying reply(sh) again.
26/08/2013 17:24:05.94 - 20 trying reply(sh) again.
26/08/2013 17:24:06.17 - 21 trying reply(sh) again.
26/08/2013 17:24:06.39 - 22 trying reply(sh) again.
26/08/2013 17:24:06.62 - 23 trying reply(sh) again.
26/08/2013 17:24:06.85 - 24 trying reply(sh) again.
26/08/2013 17:24:07.08 - 25 trying reply(sh) again.
26/08/2013 17:24:07.31 - 26 trying reply(sh) again.
26/08/2013 17:24:07.52 - 27 trying reply(sh) again.
26/08/2013 17:24:07.73 - 28 trying reply(sh) again.
26/08/2013 17:24:07.95 - 29 trying reply(sh) again.
26/08/2013 17:24:08.18 - 30 trying reply(sh) again.
26/08/2013 17:24:08.40 - 31 trying reply(sh) again.
26/08/2013 17:24:08.63 - 32 trying reply(sh) again.
26/08/2013 17:24:08.85 - 33 trying reply(sh) again.
26/08/2013 17:24:09.07 - 34 trying reply(sh) again.
26/08/2013 17:24:09.29 - 35 trying reply(sh) again.
26/08/2013 17:24:09.50 - 36 trying reply(sh) again.
26/08/2013 17:24:09.73 - 37 trying reply(sh) again.
26/08/2013 17:24:09.96 - 38 trying reply(sh) again.
26/08/2013 17:24:10.19 - 39 trying reply(sh) again.
26/08/2013 17:24:10.41 - 40 trying reply(sh) again.
26/08/2013 17:24:11.40 - Normal End of In HG612_stats.exe






DSLstats appears to also run mostly fine, and although Ive seen a few login failures its not that bad

Code: [Select]
26 Aug 2013 12:12:30 Recording paused
26 Aug 2013 12:13:11 Recording resumed

26 Aug 2013 12:15:01 Recording paused
26 Aug 2013 12:15:30 Recording resumed

26 Aug 2013 13:31:21 Auto snapshots taken
26 Aug 2013 14:04:02 Second stage login failed
26 Aug 2013 14:05:02 Second stage login failed
26 Aug 2013 15:31:45 Auto snapshots taken
26 Aug 2013 17:21:03 Second stage login failed
26 Aug 2013 17:22:03 Second stage login failed
26 Aug 2013 17:23:02 Second stage login failed
26 Aug 2013 17:32:08 Auto snapshots taken

I'll now try setting DSLstats back to its default 30 seconds to see how that goes.



Quote
With assistance from others, I did work out how to use the PC's system time.

Its more than 10yrs since I did anything pascal/dephi, but (Now) uses system time which should be more than accurate enough to work cross platform? 
Ive never used lazurus, but if it has a TTimer function like Delphi could you adapt something like this.  Ignore me if its not helpful, because I know zilch about lazurus, I was just trying to avoid you having to use a windows specific scheduler... and apologies if its totally on the wrong track.


--

Sorry eric..  just seen that you already mentioned (now) in your post.    :-[
« Last Edit: August 26, 2013, 07:17:13 PM by kitz »
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: HG612 modem stats - multiple instances.
« Reply #38 on: August 26, 2013, 07:37:52 PM »

Report from me whilst I was out.  Nothing intensive was running on the PC - only mail, DSLstats & HG612stats.
DSLstats is set to report every 60 seconds (not its default 30secs)



HG612 stats has been logging fine and I cant see any missed events.
Its been through 2 complete cycles of retry attmpts with the longest being 40 retries so far.

14:10 - 14:33 & 17:24 - 17:52

retry attempts vary, but very roughly about 20-30ms on average?  I'll attach a log sample so you can see



That's good news.

Some of that time is no doubt spent adding the details to the log file, so when I remove that, I might have to increase the number of tries or add a slight pause between each try.

Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: HG612 modem stats - multiple instances.
« Reply #39 on: August 26, 2013, 08:23:30 PM »

Quote
Ive never used lazurus, but if it has a TTimer function like Delphi could you adapt something like this.  Ignore me if its not helpful, because I know zilch about lazurus, I was just trying to avoid you having to use a windows specific scheduler... and apologies if its totally on the wrong track.

Yes, I use TTimer, but it's hopelessly inaccurate for any critical timing requirement (that applies to Delphi too) so I use a combination of TTimer and a highly accurate tick count timer for the sample timing in DSLstats.

I've now added two extra checks to DSLstats to try to avoid conflicts:

1. If a possible conflict is detected during sampling, the next sample is delayed by ten seconds.
2. An extra login option "Synchronise with HG612-Modem-Stats" which ensures that recording starts between 15 seconds and 45 seconds after the minute.

I've made the second item a user option because it can lead to a delay of up to 30 seconds before recording starts, which could be a nuisance for users who don't need it (and it applies to the Windows version only, of course).
Logged
  Eric

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: HG612 modem stats - multiple instances.
« Reply #40 on: August 26, 2013, 08:53:42 PM »

Quote
I might have to increase the number of tries or add a slight pause between each try.

I must admit I did wonder about a possible pause if ">AP"  rather than the expected "#" was returned...  Im assuming it previously exited at this point which is why I wasnt seeing any data was logged.

A pause would certainly be a more elegant solution than firing off 50 attempts :)  I suppose a fast & multi threading/core processor which isnt busy could soon go through those before DSLstats has finished.   Although saying that I would imagine it wouldnt happen too often to be such a big deal. 

Quote
I've now added two extra checks to DSLstats to try to avoid conflicts:

Excellent  :thumbs:


Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: HG612 modem stats - multiple instances.
« Reply #41 on: August 26, 2013, 09:03:36 PM »

PS BE...  Ive just noticed that current_stats doesnt appear to have run....  but unfortunately I have to run now... so I'll look properly later :)


argggh...   I was thinking it should have run at 8pm...  but Ive not changed the defaults...  and its not due to 10pm :wall:


* kitz gone
« Last Edit: August 26, 2013, 09:08:07 PM by kitz »
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

c6em

  • Reg Member
  • ***
  • Posts: 504
Re: HG612 modem stats - multiple instances.
« Reply #42 on: August 26, 2013, 09:25:47 PM »

Not sure I'm following all of this - though it is interesting
What if DSLstats never logs out. At the moment it is logging in/out at each sample - I recall that DMT tool and Routerstats keep the telnet connection open all the time from the moment it starts to when it is closed down. (I've no idea how they handle times when the PC is sleeping)
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: HG612 modem stats - multiple instances.
« Reply #43 on: August 26, 2013, 10:44:28 PM »

In order to stay logged in indefinitely, RS/DMT have to disable the normal telnet timeout. If the program is terminated abruptly for any reason, that would leave an orphan login permanently resident in the router. And in addition, it seems that the HG612 will only accept a single full login to the shell, so a permanently logged in program would lock out any other monitoring programs which use telnet.
Logged
  Eric

c6em

  • Reg Member
  • ***
  • Posts: 504
Re: HG612 modem stats - multiple instances.
« Reply #44 on: August 27, 2013, 08:04:14 AM »

Ah  - I see, Thankyou
Logged
Pages: 1 2 [3] 4 5 ... 8