Kitz Forum

Broadband Related => Router Monitoring Software => Topic started by: roseway on September 11, 2012, 04:30:16 PM

Title: rs-ux and rs-w v0.9 released
Post by: roseway on September 11, 2012, 04:30:16 PM
- added option to plot CRC and FEC errors
- added 'Stats' page
- bitloading graph is now rescaled when SNR per tone exceeds 50 dB (really)
- fixed wrong DSL uptime shown with some routers (really)
- fixed wandering/disappearing "Take snapshot" button
- fixed lockup when resizing the window to a very small size
- tidied up some aspects of the graph presentation

https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 11, 2012, 05:39:55 PM
Thank you Eric. Version 0.9 is currently being downloaded to The Cattery.  :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 11, 2012, 06:32:40 PM
I hope you enjoy it. :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: broadstairs on September 12, 2012, 08:38:18 AM
Well I have it running on my backup Linux PC and it seems very sluggish although the system monitor does not show it is eating cpu. However I will test again when my main PC is running again in a few days time as this PC is a tad flaky.

The system up time does not show what I expected though. Currently it says 2568 hours 42 minutes 16 seconds which is probably the time since last boot or power cycle, what I was hoping it would show is what shows on the routers status page of 'Internet Connection Up Time : 28 days, 11 hours, 26 minutes, 44 seconds' which equates to 683 hours, this value is of much more value than the total up time. Is it possible to do this for the D-Link 2740B?

So far re-sizing has not lost the snapshot button, but at least on this PC the font size in the configuration panel is too large especially in the dropdown boxes.

Stuart
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 12, 2012, 10:06:00 AM
Thanks for the report Stuart.

Well I have it running on my backup Linux PC and it seems very sluggish although the system monitor does not show it is eating cpu. However I will test again when my main PC is running again in a few days time as this PC is a tad flaky.

It's certainly sluggish during the 'Sampling' period, which is a significant proportion of the total time. A task I've been rather putting off (because it's difficult) is making it into a multi-threaded application, which should deal with the issue. That's my next job, apart from dealing with any significant bugs in the present version.

Quote
The system up time does not show what I expected though. Currently it says 2568 hours 42 minutes 16 seconds which is probably the time since last boot or power cycle, what I was hoping it would show is what shows on the routers status page of 'Internet Connection Up Time : 28 days, 11 hours, 26 minutes, 44 seconds' which equates to 683 hours, this value is of much more value than the total up time. Is it possible to do this for the D-Link 2740B?

I'm sure it's possible, I'll just have to delve further.

Quote
So far re-sizing has not lost the snapshot button, but at least on this PC the font size in the configuration panel is too large especially in the dropdown boxes.

Fonts can be a bit of a pain in the neck in Linux programming, but I think I know what to do about this.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 12, 2012, 10:35:27 AM
I've uploaded a version 0.91 (for 32-bit Linux only) with smaller fonts than v0.90. The 64-bit Linux and 32-bit Windows versions already have the smaller fonts.
Title: Re: rs-ux and rs-w v0.9 released
Post by: broadstairs on September 12, 2012, 06:38:49 PM
Eric Firefox cannot open the 0,.91 version for downloading, it complains about re-direction being done in a way which will never complete!

Stuart

PS I have my main PC back today so will be able to test better now.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 12, 2012, 06:51:33 PM
I think that must have been a temporary hiccup in Google Docs, Stuart. I just tried it (as a normal user, without signing in), and it downloaded normally.
Title: Re: rs-ux and rs-w v0.9 released
Post by: broadstairs on September 12, 2012, 08:01:45 PM
Well FF still refuses to do it but Crome did work so I have it now.

Stuart
Title: Re: rs-ux and rs-w v0.9 released
Post by: broadstairs on September 12, 2012, 08:05:58 PM
Sluggishness noticed on the other PC is not there now. I did wonder if it was the other PC, yes it is a tad slow if sampling but not unmanageable. Fonts so far look fine now. Be nice if you could sort the uptime thing at some stage. I'll leave it running and see how it goes.

Stuart
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 12, 2012, 08:43:26 PM
Well FF still refuses to do it but Crome did work so I have it now.

Stuart

That's a bit strange. The sharing configuration for that file is exactly the same as all the others, and Firefox has no trouble with it here. I'm afraid I'm baffled by this.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 12, 2012, 08:49:13 PM
Sluggishness noticed on the other PC is not there now. I did wonder if it was the other PC, yes it is a tad slow if sampling but not unmanageable. Fonts so far look fine now. Be nice if you could sort the uptime thing at some stage. I'll leave it running and see how it goes.

Stuart

I'm pondering the uptime problem at the moment. One other suggestion that was made is to use the superframes (SF) figure, and I can get an uptime of the right order from this, but it's inconsistent between different routers, and I don't think it's provided at all on VDSL2 systems. There was a discussion about this some time ago on the Routerstats forum, and I think John ended up tearing his hair out over it. :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 14, 2012, 04:48:49 PM
Sluggishness noticed on the other PC is not there now. I did wonder if it was the other PC, yes it is a tad slow if sampling but not unmanageable. Fonts so far look fine now. Be nice if you could sort the uptime thing at some stage. I'll leave it running and see how it goes.

Stuart

I'm pondering the uptime problem at the moment. One other suggestion that was made is to use the superframes (SF) figure, and I can get an uptime of the right order from this, but it's inconsistent between different routers, and I don't think it's provided at all on VDSL2 systems. There was a discussion about this some time ago on the Routerstats forum, and I think John ended up tearing his hair out over it. :)


I've looked much more closely at this, and I've concluded that, with some routers such as the 2740B, the current DSL uptime simply isn't available in any of the telnet stats. The AS and SF figures both give some approximation to total uptime (although they resolve to somewhat different figures). So, for the moment at least, I can't meet your request. I've added an option in the configuration section to choose AF or SF as the source of the uptime information, and I've added 'days' to the displayed uptime where it's more than 24 hours.
Title: Re: rs-ux and rs-w v0.9 released
Post by: broadstairs on September 14, 2012, 05:43:11 PM
Eric I appreciate that, I am puzzled that the web interface manages to display a figure but they telnet data seems not to contain it, I might have a play and see if there is any way to get this value.

Stuart
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 14, 2012, 06:38:28 PM
I look at the Status --> Device Information page of the GUI of the HG612 and see "System up time  24 days 22 hours 46 minutes 58 seconds ". That is when the device was last power cycled, thus (re-)booted.

I look at the Status --> WAN --> xDSL page of the GUI of the HG612 and see "DSL up time  1405793". Assuming that to be in seconds, it is equivalent to 16 days, 6 hours, 29 minutes and 53 seconds.

Looking in the modem log, I see the line "2012-8-29 10:51:56 Notice 104505 PPP dial-up succeed".

I look at the bottom of the SNR Margin screen of rs-ux and see "DSL connection uptime: 363 hours 5 min 10 sec".

It seems as if rs-ux is reporting in the correct order of magnitude for the xDSL uptime.  :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 14, 2012, 07:00:46 PM
Thanks b*cat. rs-ux currently gets its information from the AS value in --stats, and this value (in seconds) agrees with the "Since link time =" item at the bottom of --stats, so I'm happy that it's correct in the case of the HG612. Unfortunately, other routers present the information differently.
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 14, 2012, 09:46:44 PM
 :thumbs:
Title: Re: rs-ux and rs-w v0.9 released
Post by: rhohne on September 14, 2012, 10:20:51 PM
Impatiently waiting for the next version to see which additional features will be added.

Have now been running rs-w v0.9 on and off for the last couple of days and have the following observations :-


Have found that I can not run rsw for extended periods due to a "The internet connection may have been dropped" error message box. When this happens it looks like the login sequence is bypassed and "adsl info --stats" is sent as the login name.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 14, 2012, 10:57:18 PM
Thanks for the report. Some initial thoughts:

1. Good.
2. Uptime is problematical, as discussed earlier. I'll look at it further, but some information may simply not be available from the telnet interface of some routers.
3. Agreed. I missed that one.
4. Agreed.
5. Yes, I probably need to increase the minimum window size.
6. Agreed.
7. I think that this only happens when the router reports zeros for all the upstream tones. You can check this in the Telnet data/SNR tab.
8. I'll check this out.
9. Agreed.

Concerning your final point, I think I may have already dealt with this in the next version, but I must check further.
Title: Re: rs-ux and rs-w v0.9 released
Post by: asbokid on September 15, 2012, 01:59:52 AM
Thanks for the report. Some initial thoughts:

1. Good.
2. Uptime is problematical, as discussed earlier. I'll look at it further, but some information may simply not be available from the telnet interface of some routers.

Maybe the info that you need could come from the /proc pseudo-file system (mostly where the c/l tools get it from)

From the Huawei: HG612
Code: [Select]
Welcome Visiting Huawei  Home Gateway
Copyright by Huawei Technologies Co., Ltd.
..
BusyBox v1.9.1 (2010-10-15 17:59:06 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# cat /proc/stat
cpu  4557 0 10124 70898040 0 0 161898 0
cpu0 4557 0 10124 70898040 0 0 161898 0
intr 169056848 0 0 0 0 0 0 0 142149240 0 0 9700 0 20443679 0 0 0 0 0 0 0 0 0 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2640536 0 13693 0 0 0 0 0 3798321 5 1594 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
ctxt 398645089
btime 946684800
processes 18047
procs_running 5
procs_blocked 0
#

In the case of uptime, the btime field in /proc/stat "gives the time at which the system booted, in seconds since the Unix epoch" (midnight on Jan 1st 1970.)

And the current time can get gotten as an epoch using the date command. Which is then subtracted from btime (the boot time epoch) - to give system uptime.  That's probably how the Huawei gets it.

Code: [Select]
# date +%s
947396644

e.g.. 947,396,644 - 946,684,800 = 711,844 seconds uptime = eight and a bit days uptime.

cheers, a
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 15, 2012, 07:27:49 AM
Thanks Asbo, that would seem to be an accurate means of getting the system uptime.
Title: Re: rs-ux and rs-w v0.9 released
Post by: zhadoom on September 15, 2012, 03:37:38 PM
Thanks for this very useful software.

I'm ( another ) VDSL2 user from Brazil using Huawei HG612.
I notice that the bitloading show something wrong when compared with telnet data. The connection doesn't use U0 band but the graphic colors part of D1 as upstream.

Code: [Select]
xdslcmd info --pbParams
xdslcmd: ADSL driver and PHY status
Status: Showtime
Retrain Reason: 0
Max: Upstream rate = 8496 Kbps, Downstream rate = 63900 Kbps
Path: 0, Upstream rate = 3998 Kbps, Downstream rate = 38000 Kbps

Discovery Phase (Initial) Band Plan
US: (872,1203) (1972,2779)
DS: (36,867) (1208,1971) (2788,4051)
Medley Phase (Final) Band Plan
US: (872,1203) (1972,2779)
DS: (36,867) (1208,1971) (2788,4051)
       VDSL Port Details       Upstream        Downstream
Attainable Net Data Rate:       8496 kbps         63900 kbps
Actual Aggregate Tx Power:        6.3 dBm          14.1 dBm
============================================================================
  VDSL Band Status        U0      U1      U2      U3      D1      D2      D3
  Line Attenuation(dB):   N/A 16.1 20.1   N/A 17.7 43.6 65.3
Signal Attenuation(dB):   N/A 30.4 48.0   N/A 17.7 43.6 65.3
        SNR Margin(dB):   N/A 15.7 14.9   N/A 13.3 13.3 0.0
         TX Power(dBm):   N/A 6.2 -8.6   N/A 12.2 7.6 -128.0
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 15, 2012, 03:53:49 PM
Thanks for that. I'll see what I can do.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 16, 2012, 03:47:51 PM
I've got Asbo's suggested method for calculating system uptime working properly now (after tearing my hair out over the fact that the calculated value never seemed to change - * see below).
I've covered all of rhohne's points, and I've successfully made it into a multi-threaded program, so that it now remains responsive during the sampling period.
I've still got to refine the handling of VDSL2 band plans, and when that's done I'll upload a new version, which I hope will be the basis for a v1.0 release.

* Just so that the programmers among you can have a giggle at my expense, I've spent ages wondering why the calculated system uptime didn't change, and I did finally work it out. At several points in the program I have to use floating point numbers, and I was using single-precision numbers on the basis that their 6-7 digit precision was more than enough for this program's needs. What I eventually realised was that the "seconds since the Unix epoch" numbers are 10-digit numbers, and what I have to do is subtract one of these very large numbers from another to get a fairly small result. So single precision just wasn't adequate - the changes between successive samples are all in the last few digits. I should have realised this earlier. :doh:
Title: Re: rs-ux and rs-w v0.9 released
Post by: asbokid on September 18, 2012, 03:45:06 AM
Yeh, glad it worked Eric!   Just had a quick look under /proc and didn't immediately find the answer, but perhaps even "date" could be omitted in calculating system up time?  The system info under /proc is so poorly/obscurely documented. E.g. ECI has disabled netstat in the busybox build for the f/w of its B-Focus VDSL2 CPE.  So it's really hard to see what network daemons are listening.  But the info on server sockets is still there under /proc albeit in obscure form.

Keep up the good work. You've got a really good project going there!

cheers, a

Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 18, 2012, 07:51:16 AM
Thanks asbo. System uptime and DSL link uptime are obviously going to be ongoing issues for various routers. For the moment, your suggested method for system uptime works with the HG612, and the AS figure gives a good DSL uptime with the HG612. However, I see that the HG612 and my three Linux PCs all have /proc/uptime; this returns two figures, the first of which (in seconds) corresponds with the value calculated by the 'asbo method'. I'm not sure what the second figure represents.

To add to the above, I've now checked out several other routers, and apart from the HG612, none of them support the 'time' command, so the time calculated from Unix epoch time unfortunately fails. However, all the routers I've tried do support 'cat /proc/uptime' which gives the system uptime directly in seconds, so this is what I'm going to use.
Title: Re: rs-ux and rs-w v0.9 released
Post by: rhohne on September 18, 2012, 03:16:15 PM
As you have pointed out the first number is the total number of seconds the system has been up. The second number is how much of that time the machine has spent idle, in seconds.

Out of interest does the HG612 have a /tmp/wan_uptime or similar. If so then the PPP uptime can be calculated by subtracting /proc/uptime from it, as it a copy of /proc/uptime when the PPP connection is established

The date/time commands depend on the version of BusyBox and whether or not the option was selected to include them in the build.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 18, 2012, 03:40:38 PM
Thanks for that information. I've checked out /tmp in the HG612, but it's empty. There's a wan directory under /var, but that doesn't contain uptimes.
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 18, 2012, 07:36:28 PM
As you have pointed out the first number is the total number of seconds the system has been up. The second number is how much of that time the machine has spent idle, in seconds.

From my laptop, which is running RHEL 6u3, I see the following --

Quote
[bcat@Duo2 ~]$ cat /proc/uptime; uptime
9567.48 14757.53
 19:34:30 up  2:39,  2 users,  load average: 0.56, 0.23, 0.25
[bcat@Duo2 ~]$

How would you interpret that output, please?  ???
Title: Re: rs-ux and rs-w v0.9 released
Post by: rhohne on September 18, 2012, 08:38:43 PM
Besides ...

uptime:            2:39:27
total idle time:  14757s
current time:    19:34:30
uptime:            2:39
No users:         2
load averages for the past 1, 5 and 15 minute intervals: 0.56, 0.23, 0.25

I take it that you are refering to the idle time exceeding the uptime. What I didn't mention was that on multi core systems (and some linux versions) the second number is the sum of the idle time accumulated by each CPU
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 18, 2012, 09:43:24 PM
Quote
. . . the second number is the sum of the idle time accumulated by each CPU

It's so simple once it has been explained.  ;)  Thank you.  :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 19, 2012, 08:09:08 AM
Quote
It's so simple once it has been explained.

It is indeed. We're learning all the time.

In the version of /proc/uptime which you posted, the format was different to the single line with two figures which I found on the routers I checked, but it does look as though the first number returned is probably always the system uptime, and that's the only number I need.
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 19, 2012, 10:24:43 PM
Quote
the first number returned is probably always the system uptime

Very precisely/pedantically it is the "uptime" of the Linux kernel. When one is dealing with a SOC device, such as a modem/router, it will show the elapsed time since the device was powered-on or was re-booted -- whichever event was the most recent.
Title: Re: rs-ux and rs-w v0.9 released
Post by: roseway on September 19, 2012, 10:33:07 PM
Quote
Very precisely/pedantically it is the "uptime" of the Linux kernel.

You're right of course, but from the point of view of a user of this program, a bit over-pedantic perhaps? :)
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 19, 2012, 10:36:46 PM
Just ensuring we appreciate that it is the physical device uptime and not the xDSL link uptime.  :graduate:
Title: Re: rs-ux and rs-w v0.9 released
Post by: rhohne on September 19, 2012, 10:43:49 PM
In the version of /proc/uptime which you posted, the format was different to the single line with two figures which I found on the routers I checked

The output looks correct for the two commands issued.
Title: Re: rs-ux and rs-w v0.9 released
Post by: asbokid on September 19, 2012, 10:50:24 PM
Mysteriously, when I checked this (albeit in the wee hours, so maybe didn't look very carefully),  i was sure that /proc/uptime wasn't present in the HG612 filesystem.  Which is why that alternative (btime in /proc/stat) was sought.   Why could that be?  Bald_Eagle reported other problems  of not being able to get a time sync with ntp, due to the lack of a WAN-side IP4/6 route, when the HG612 is configured for a default ethernet bridge.  Could that be related?

cheers, a
Title: Re: rs-ux and rs-w v0.9 released
Post by: burakkucat on September 19, 2012, 11:08:38 PM
This is what I see, using a HG612 on an ADSL2+ enabled line --

Quote
[bcat@Duo2 ~]$ telnet 192.168.1.254
Trying 192.168.1.254...
Connected to 192.168.1.254.
Escape character is '^]'.

Welcome Visiting Huawei  Home Gateway
Copyright by Huawei Technologies Co., Ltd.
Login:admin
Password:
ATP>shell


BusyBox v1.9.1 (2010-10-15 17:59:06 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# ls /proc
24247        203          135          19           devices      tty
24246        202          134          5            cpuinfo      bus
24245        195          133          4            partitions   brcm
660          193          132          3            stat         sys
639          148          131          2            interrupts   irq
631          147          130          1            slabinfo     misc
619          146          129          self         klobinfo     ioports
549          145          128          loadavg      buddyinfo    iomem
535          144          127          uptime       vmstat       timer_list
533          143          116          meminfo      zoneinfo     kallsyms
531          142          101          version      diskstats    crypto
525          141          59           filesystems  modules      mtd
312          140          49           cmdline      kcore        fcache
310          139          35           locks        net          pktcmf
278          138          34           execdomains  sysvipc      switch
277          137          33           mounts       fs           mii
204          136          32           kmsg         driver
# cat /proc/uptime
2604989.60 2604090.02
#
Title: Re: rs-ux and rs-w v0.9 released
Post by: asbokid on September 19, 2012, 11:19:29 PM
yeah, thanks b*cat. It's definitely there now. Perhaps it always was! Hmm..

cheers, a