Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: roseway on July 30, 2012, 03:41:48 PM
-
I'm developing a DSL router monitoring program inspired by John Owen's Routerstats program, to work in Linux. It was initially intended for use with the Huawei HG612 (with Asbokid's unlocked firmware) but in principle it should also work with other Broadcom-based DSL routers which have a fully functioning telnet interface. Its features are incomplete, but it's reached the point where it can do useful work and I would appreciate some feedback about it. It's free and open source, and will be released under the GPL when I sort out the details.
You should hopefully be able to download it from the link below. Please read the included README.
[Removed]
Original version removed, 32-bit and 64-bit versions from:
https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
-
Looks excellent eric, unfortunately though since I dont have a linux system, Im unable to give it a go.
-
I'll give it a go later and see if it works with my router.
Kitz a perfect opportunity to try Linux ::) ;)
Stuart
-
Well its now running on my D-Link DSL-2740B and graphing nicely.
Not sure how much testing I'll be able to do as my only Linux PC is going to have to go back for repair AGAIN, had a new m/b under warranty plus waiting for my new memory under warranty and now this m/b is playing silly b*****s crashing not posting you name it it does it intermittently.
Stuart
-
I'm developing a DSL router monitoring program inspired by John Owen's Routerstats program, to work in Linux. It was initially intended for use with the Huawei HG612 (with Asbokid's unlocked firmware) but in principle it should also work with other Broadcom-based DSL routers which have a fully functioning telnet interface. Its features are incomplete, but it's reached the point where it can do useful work and I would appreciate some feedback about it. It's free and open source, and will be released under the GPL when I sort out the details.
You should hopefully be able to download it from the link below. Please read the included README.
https://docs.google.com/open?id=0BxbUtOYVZ_SCSmVMaThGM3daYzg
Way hay! Good stuff Eric! :) I will give it a spin straight away :)
cheers, a
-
Very well done Eric.
I will see if I can use when I have got used to driving this HH3.
-
I'm developing a DSL router monitoring program inspired by John Owen's Routerstats program, to work in Linux.
Excellent news, Eric.
You should hopefully be able to download it from the link below. Please read the included README.
https://docs.google.com/open?id=0BxbUtOYVZ_SCSmVMaThGM3daYzg
Even though his evening meal will be delayed, b*cat makes arrangement to download the appropriate software that is offered by a fellow "penguin-hugger". ;D
Edited to add --
[ajb@Duo2 rs-ux-0.1]$ ll
total 5140
-rw-r--r--. 1 ajb ajb 1633 Jul 30 09:52 README
-rwxr-xr-x. 1 ajb ajb 5257000 Jul 30 14:57 rsux
[ajb@Duo2 rs-ux-0.1]$ file rsux
rsux: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
[ajb@Duo2 rs-ux-0.1]$
I run a pure 64-bit system on this laptop, so testing will be delayed until I crank-start the 32-bit workstation (the hardware of which dates from Q4 of 1999).
A very hungry b*cat goes to look at his food-bowl. ::)
-
I run a pure 64-bit system on this laptop, so testing will be delayed until I crank-start the 32-bit workstation (the hardware of which dates from Q4 of 1999).
asbokid is a 64-bitter, too. This worked though. Hold on to your hats, it's a monster package (81MB)
sudo apt-get install ia32-libs-gtk
Excellento, Eric!
cheers, a
-
Actually I may be able to cross-compile for 64-bit. I'm using Lazarus/Freepascal, which has the capability if I can make it work.
Scrap that. I don't think it can be done. I think this may be the time to double my bits, but ia32-libs should get you going if you're willing. :)
-
As we speak, I'm installing Debian Testing 64-bit on my other PC. They can share the source code, so I can build 32-bit executables on this one, and 64-bit executables on the other.
-
I think I'll wait until the source-code is laid in the public's gaze and then compile it myself. :)
-
I think I'll wait until the source-code is laid in the public's gaze and then compile it myself. :)
I can upload it shortly, but are you familiar with Lazarus/Freepascal (because that's what I'm using)?
Source now available at
https://docs.google.com/open?id=0BxbUtOYVZ_SCZ3ZZYzBEQTc5SUE
-
I'm not sure if this is a good sign or a bad one, but I actually had to look that up ::)
I guess I'm also going to have to wait for the 64bit build...
-
Both 32-bit and 64-bit versions and the source can be downloaded from
https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
These are both minor version upgrades on the original (just a small cosmetic change), and I've removed the original to avoid confusion.
-
Thank you, Eric.
I am currently testing the original version on my 32-bit workstation. My comments:
(1) The configuration menu does not allow me to set 24 hours per graph, it appears to have a maximum of 10.
(2) Any possibility of swapping the colour around, please? Red for downstream and green for upstream?
(3) The numbers displayed on the x- and y-axis are rather small for less-than-perfect eyesight. Could they please be enlarged?
(4) . . .
:)
-
Thanks for the feedback b*cat
(1) Yes
(2) I'm planning to make the colours user-configurable
(3) Yes. The font size for the axis marks is 6pt, I can make it 8pt without using up too much space. Possibly another user configuration item...
(4) Possibly :)
-
Excellent Eric!
Storage of data in DSL stats programs is something that has previously been lacking, at least in terms of software for subscribers. Maybe an SQL-based database storage, and separation of the logic for the graphing client and the data scraping client, etc. Then other tools could be used to query or mine the database for complex analyses.
It's really nice the way your graphs can be sized. That's where dmt fell down badly. It was okay for ADSL1 and perhaps 2+, but with all the tones in VDSL2 it just didn't provide enough screen real estate. Perhaps a 'ratio lock' checkbox to keep the window in fixed proportion.
Does that version of Pascal support threading so that the GUI remains responsive while the scraper continues to gather the data from the modem?
The discovery by arobertson545 of the Broadcom Diagnostic tool could be put to great use by tools like yours. Many more metrics could be recorded - e.g. through the diagnostic interface, noise can be measured down to the millisecond or two. Whereas the dslcmd telnet client has a granularity of one second.
Keep up the good work!
cheers, a
-
Storage of data in DSL stats programs is something that has previously been lacking, at least in terms of software for subscribers. Maybe an SQL-based database storage, and separation of the logic for the graphing client and the data scraping client, etc. Then other tools could be used to query or mine the database for complex analyses.
I'd need advice on what form the data should take, but yes, definitely on the todo list.
It's really nice the way your graphs can be sized. That's where dmt fell down badly. It was okay for ADSL1 and perhaps 2+, but with all the tones in VDSL2 it just didn't provide enough screen real estate. Perhaps a 'ratio lock' checkbox to keep the window in fixed proportion.
A ratio lock would be simple as an option.
Does that version of Pascal support threading so that the GUI remains responsive while the scraper continues to gather the data from the modem?
As you no doubt noticed, the program seems to freeze for a few seconds while each sample is being read and processed. Yes, Freepascal supports multithreading (although I've no personal experience). http://wiki.freepascal.org/Multithreaded_Application_Tutorial
-
For those persons who still practice the Black Art (and use BillyGatesWare on their systems) I attach four screenshots from Eric's utility, which is currently executing on my 32-bit workstation.
I approve of the icon. What could be more appropriate than the paw mark of a felis domesticus? ;D
-
I approve of the icon. What could be more appropriate than the paw mark of a felis domesticus?
I'm pleased you approve, but it's a leopard pawprint actually - definitely not domesticus. ;D
-
I'm pleased you approve, but it's a leopard pawprint actually
b*cat suppresses the urge to give Eric his best black look and decides to attach three screenshots from the utility (original version) which had been left running overnight.
Notice the step in the graphs, just before noon (BST), resulting from when I initiated a new PPPoA connection and a re-sync with the MSAN by temporarily disconnecting the DSL cable from the SSFP.
-
Thanks for your continuing testing b*cat. I'm pleased to see that rs-ux dealt calmly with your yanking out of the DSL cable. :)
I expect to be uploading a new version tomorrow, covering the points you raised previously, and also adding the capability to scroll the graphs horizontally.
-
Thanks for posting the screenies b*cat, Its looking very good eric :thumbs:
-
Its looking very good eric :thumbs:
I'll second that! :)
-
Thanks people.
Version 0.2 now available in 32-bit and 64-bit varieties at https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
- added user configuration of graph colours
- increased font size of graph axis figures
- increased maximum time per page to 24 hours
- changed info displayed in status bar
- added horizontal scrolling of the graphs by dragging with the mouse
- some optimisations to improve the GUI responsiveness
- added 'Sampling' indicator (to indicate unresponsive period)
-
Its looking very good eric :thumbs:
I'll second that! :)
It definitely gets my vote.
On the storage issue. One possibility would be to convert the output from the xdsl/adsl stats cmd to a common plain text format and store it using SQLite.
That would put all the device specific logic in the front end away from the data analysis and eliminate any potential problems with endian-ness.
As an aside. While I was looking at the telnet stats tab, it occurred to me that I could do the data collection on the modem itself (hg612) or the router, both of which are MIPS32. Hence the concern with endian-ness :)
-
Version 0.2 now available in 32-bit and 64-bit varieties at https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
Downloaded. I'll now give it a test on my 64-bit laptop. :)
-
I have a bit of a query here (no not me darling), seeing as how I now have two pieces of kit I.E. a modem & a router which should I be monitoring?
I haven't unlocked the Hub yet. I'm thinking would it be worth my while doing it. I haven't used routerstats in quite a while as I knew I would be disheartened by what I saw.
So would it be worth my while, or should I leave things as they are and wait until I have a reason to use Eric's excellent program? Hmm! :hmm:
-
I have a bit of a query here (no not me darling), seeing as how I now have two pieces of kit I.E. a modem & a router which should I be monitoring?
In a two word answer: The modem. ;)
-
I might be being a bit thick here, but by your answer I feel I should be able to plug my Ethernet cable into the modem.
If this is so, what am I doing with a Home Hub? apart from adding the wifi into the mix.
Sorry Eric I don't mean to pinch your thread. If I need to go further I will start one of my own.
-
[OT hijack of Eric's thread]
If you only have one computer to connect to the broadband service and you do not need to use wireless, then yes, you can connect that system direct to the LAN1 port of the HG612.
Before dosing that, you will need to ensure that you have an adequate firewall present & operating on the computer and you will also need to configure the computer to establish a PPPoE connection.
If that reads as something a bit too difficult, I would advise that you leave the Home Hub between your computer(s) and the HG612. That way the Home Hub will provide the firewall service and the PPPoE authentication on your behalf.
Eric's monitoring program, rs-ux, is only usable by those who persons who "hug a penguin". If you use BillyGatesWare on your 'puter, you will need to use the scripts developed by "Feathers McGraw" (a.k.a. Baldy_Bird or Bald_Eagle1). If the latter is true, then he is the person to aid you getting things set up correctly to monitor your line. :)
[/OT hijack of Eric's thread]
-
Thank you B*Cat I get the full picture. I won't have anything other than a penguin in my machine.
I've got my thinking head back on now.
Thanks again.
-
Belatedly, I'll add my congratulations, looks like excellent work! :drink:
My initial reaction was that it wasn't of immediate use to me since my linux boxes are either minimal (diskless myth frontends), or the big server, which is headless. But actually, I have VNC up and running on the server and accessible from the windows PC, the MAC and even the iPad. So running it in a VNC session on the server, which is available 24/7, could be very useful indeed and I now feel compelled to give that a try at some point.
All I need is the time. I still haven't figured out why retirement from full-time work has left me with less time for things like this. ???
Well done, anyway.
-
Thanks 7LM :)
-
I started off by configuring the 64-bit version to my own preferences (image rsux-08.png, below) and set it executing just before 2000 hours BST, yesterday.
Leaving it alone in a work-space of its own, I continued to use my laptop normally -- other than logging out. When it became time to retire to that warm and sleepy spot, I ensured that the laptop was running from the mains electricity supply and closed the lid.
Reviewing the Connection speed graph (image rsux-09.png, below), the first thing that is apparent is the (boring) plotted straight lines (very good)! Note that I managed to obtain the screen-scrape whilst the utility was displaying the confidence-boosting "Sampling" message in the top right-hand corner. Looking at the bottom of that image, we see --
Attn. Down: 50.5dB Attn. Up: 28.5dB Downstream power: 0.0dBm Interleave Depth: 32
My personal preference is to have a space between the numerical value and its corresponding unit. For example, "Attn. Up: 28.5 dB". Is there a convention that covers such detail, please? :-\
Moving on to the SNR margin graph (image rsux-10.png, below), yuck! :tongue: (Not at Eric's excellent work but my line's behaviour, last night.) Again, as above, I would prefer a space between the numerical value and its corresponding unit.
[OT]
With regards to my line, it is a TalkTalk fully LLU'd service. The MSAN has an Infineon chipset and is configured with the DLM disabled, interleaving on and a target SNRM of 6 dB.
As the Huawei HG612 uses a Broadcom chipset (BCM6368), I have tweaked the DS target SNRM to 4 dB by means of the technique documented in the Kitz wiki (http://wiki.kitz.co.uk/index.php?title=Broadcom_CLI#adsl_command).
By consideration of my yucky SNR margin graph, there is clearly some (new) form of RFI being generated at certain times of the day. It's time to blow the surface dust off my Icom IC-R5 and start "sniffing" the radio spectrum at 306 kHz and 612 kHz.
As for the HG612's ability to "hang on" in the presence of noise, I have to say that it is very good. I have been happily using my broadband connection when the SNRM has dropped at low as -1.9 dB! And that was not momentary, it was for about 20 minutes. Finally when the SNRM hit -2.0 dB the HG612 finally "let go" and re-negotiated sync with the MSAN.
[/OT]
-
My personal preference is to have a space between the numerical value and its corresponding unit. For example, "Attn. Up: 28.5 dB". Is there a convention that covers such detail, please? :-\
I agree with you. Originally I was trying to save as much space as possible in order to fit as much in as I could, but it's probably unnecessary. The same status bar is displayed on both the SNR and Speed tabs, so the same comment applies to both.
Like you, I'm connected to an Infineon MSAN and the HG612 is exceptionally stable (although the SNR margin has never dipped much below zero).
-
Even though I cant use it, I must say this is looking very good. Well done eric :)
-
You used to be a Linux chick, no?!
-
Loooooooong time ago, RedHat. Back in the days when I had spare PCs to play on.
-
RedHat?! Eugh! That's for beardies! You seem much more of a Debian (http://www.debian.org/distrib/index.en.html) girl!
cheers, a
-
I did say it was a loooooong time ago... ;D back in the days of RH9 when you could get it on Free cds with PC mags etc. - I didnt have DSL.
It had some sort of gui and better support for hardware than debian did in those days, iirc back then favoured choices seemed to be Mandrake or RH9?
Going all CLI would have been too much for me to take in at once... and even with RH I still ran into a couple of problems with driver support. I may be wrong but Debian seemed to become more popular after it was announced that RH9 was going over to Fedora for future releases? :-\
-
I did say it was a loooooong time ago... ;D back in the days of RH9 when you could get it on Free cds with PC mags etc. - I didnt have DSL.
I remember that time (only just!) :-( People would get their distro on CDs from the LinuxEmporium. Debian came on 8 or 10 CDs, and at £2 a CD that was a no-no except for the ultra wealthy!
It had some sort of gui and better support for hardware than debian did in those days, iirc back then favoured choices seemed to be Mandrake or RH9?
Going all CLI would have been too much for me to take in at once... and even with RH I still ran into a couple of problems with driver support. I may be wrong but Debian seemed to become more popular after it was announced that RH9 was going over to Fedora for future releases? :-\
I don't know what the Linux politics are about. Best to ignore them (especially that RedHat lot) They squabble over whose turn it is to squabble! In practice all Linux distros are much of a muchness now. The Debian packaging (also used by Ubuntu) does seem a bit easier to handle. i.e. idiot-proof is perfect for me!
Do you want a hand obliterating Windows to make space for Debian?! Betcha that GbE interface works in Linux!
cheers, a
-
lol
There's no need to 'retire' Windows if you've got 10GB to spare. Either Debian or Ubuntu will live quite happily in that with a swapfile instead of a partition.
My setup is three primary partitions, 20GB WXP Pro (O/S & programs), 425GB NTFS Data (Games and everything else), 20GB Ubuntu 12.04 (AMD64) with a 2GB swapfile in the root directory - about 7~8GB in total.
There are other ways of doing it of course. In this case the Linux install went in some time after everything else.
-
Version 0.3 now available in 32-bit and 64-bit versions:
- added bitloading graph (with drag scrolling) - see below
- right-clicking on a tone displays the details of that tone
- added separate 'DSL type' entries for VDSL2 profile 8a and VDSL2 profile 17a
- some small cosmetic improvements
https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
[A reminder: these are Linux programs only]
-
Version 0.3 now available in 32-bit and 64-bit versions:
- added bitloading graph (with drag scrolling) - see below
- right-clicking on a tone displays the details of that tone
- added separate 'DSL type' entries for VDSL2 profile 8a and VDSL2 profile 17a
- some small cosmetic improvements
https://docs.google.com/folder/d/0BxbUtOYVZ_SCZ1BnY3RKYnN2b1U/edit
[A reminder: these are Linux programs only]
This is looking very nice indeed :-) I will enjoy playing with it after tea (sausage roll :( )
cheers, a
-
Thank you, Eric. The latest release, for both architectures, has been downloaded.
I shall now "have a play". :angel:
-
I can happily say that Version 0.3 is looking very good.
I have been running it with a setting to harvest the data once per minute and plot sixty minutes to a graph. Horizontal scrolling works quite nicely, I wonder what the limit to the scroll would be?
Four screen-shots are attached below (for the benefit of those whose orientation is directed towards a different OS).
One current request. Would it be possible to have an interstitial mark at the midpoint between each "named" point on the y-axis of each graph, please? I.e. for the SNR margin graph, at 1, 3, 5, 7, etc, dB, for the Connection speed graph, at 1000, 3000, 5000, etc, kbps and for the Bitloading graph, at 1, 3, 5, 7, etc, bits?
-
Many thanks for the feedback.
I don't know what the limit on scrolling is (if there is one), but I will try to find out. For the SNR margin and connection speed graphs I've used a ready-made graphing component which is provided with Lazarus, and the historical data is stored automatically by this component, so the limit will be the amount of storage space allocated. For these graphs, the amount of data stored isn't very much, so I don't envisage a problem.
The bitloading graph is different. Quite a lot of data is produced with each sample, and the off-the-shelf graphing component tends to choke up on the amount of historical data which it stores, unnecessarily in this case. So I've written my own graphing component for this graph, and no historical data is stored. I'll be providing a snapshot facility to allow users to store historical data as external files.
I should be able to add the interstitial marks - this looks like a good idea.
-
I have it running again now on my backup PC. It does seem to give an error if the wrong command is selected to get the data, something about a list index out of range. I'll have a go at capturing the error again later.
Stuart
-
Thanks Stuart. I've got to add some error trapping to deal with that sort of situation.
-
Here is a snapshot...
Stuart
-
Thanks, that enables me to pinpoint where the issue is.
Now corrected for next version (soon).
-
Just found this Eric, I will test at a later date as I have some big issues to sort out with my remote backup system and time is running out fast.