Kitz Forum
Broadband Related => Router Monitoring Software => Topic started by: Axel Foley on December 19, 2014, 03:39:27 PM
-
I'm using DTM and RouterStats to monitor my DGN2200 (v4) router with amod firmware. They're both working fine, but I don't quite like them, for different reasons.
I now found DSLstats, and would like to use it because I really like the GUI and all the features.
I tried setting it up like I did for DTM and RouterStats but it's not working.
I think Telnet authentication is ok, because I don't receive errors in that phase, but when sampling starts, after a while I receive the message: "No data returned in latest sample".
I'm using a custom firmware, called amod (http://alfie.altervista.org/amod2200V4/), which enables telnet by default, allows to save SNR modifications permanently, implements pppoe-relay and a bunch of other nice features.
The telnet prompt string (when I telnet via terminal) is: DGN2200V4 / #
The adsl control commands available from shell are: adsl, adslctl, xdslctl. adsl and adslctl are symlinks to xdslctl.
In DSLstat's "telnet->connection stats" I expected to log telnet data, but it's empty. I don't know if there's another way to debug what DSLstats is sending/receving.
Here's an example of stats I can read from telnet:
DGN2200V4 / # adslctl info
xdslctl.orig: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 1113 Kbps, Downstream rate = 14816 Kbps
Bearer: 0, Upstream rate = 1106 Kbps, Downstream rate = 13230 Kbps
DGN2200V4 / # adslctl info --show
xdslctl.orig: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 1113 Kbps, Downstream rate = 14748 Kbps
Bearer: 0, Upstream rate = 1106 Kbps, Downstream rate = 13230 Kbps
Link Power State: L0
Mode: ADSL2+
TPS-TC: ATM Mode
Trellis: U:ON /D:OFF
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 7.1 6.9
Attn(dB): 30.0 18.8
Pwr(dBm): 19.1 12.1
ADSL2 framing
Bearer 0
MSGc: 58 14
B: 59 13
M: 4 16
T: 7 8
R: 14 12
S: 0.5791 6.4218
L: 3509 294
D: 32 8
Counters
Bearer 0
SF: 454155 458735
SFErr: 427 0
RS: 50865195 284853
RSCorr: 9524836 0
RSUnCorr: 436 0
Bearer 0
HEC: 327 0
OCD: 0 0
LCD: 0 0
Total Cells: 229785163 19207523
Data Cells: 31505886 14495885
Drop Cells: 0
Bit Errors: 0 0
ES: 4041 9
SES: 591 2
UAS: 903 903
AS: 7364
Bearer 0
INP: 0.50 1.00
INPRein: 0.00 0.00
delay: 5 13
PER: 16.21 16.05
OR: 31.57 9.96
AgR: 13210.75 1111.86
Bitswap: 655/655 1/1
DGN2200V4 / #
Thanks for any help you might give me to make it work with my setup.
-
If you get the message "No data returned in latest sample" with the first sample, then Telnet Data --> Connection Stats will be empty because no data has been received. This means that login has not completed although the username and password have been accepted.
There are two possibilities which occur to me:
1. The modem might need an extra 'kick' to get to the BusyBox prompt. Try enabling the option "Modem/router is HG622 type". This option sends an extra Enter after login, and might be what's needed.
2. Some small time delays might be needed. If you use the "Special login" page you can add small delays during login, which might resolve the problem. Fill in the other details to correspond with what you see during a manual login.
-
If you get the message "No data returned in latest sample" with the first sample, then Telnet Data --> Connection Stats will be empty because no data has been received. This means that login has not completed although the username and password have been accepted.
There are two possibilities which occur to me:
1. The modem might need an extra 'kick' to get to the BusyBox prompt. Try enabling the option "Modem/router is HG622 type". This option sends an extra Enter after login, and might be what's needed.
2. Some small time delays might be needed. If you use the "Special login" page you can add small delays during login, which might resolve the problem. Fill in the other details to correspond with what you see during a manual login.
Tried HG622 option: "Error running HG622 workaround".
The router doesn't require extra enters, I thought about that and debugged it manually, it's very clean, here's a simple telnet (windows telnet) session captured: as you can see, as soon as it connects, it asks for login, then enter, then pwd, then I'm at the prompt.
Questions:
1. Do you check the prompt string when you login/connect? This firmware uses a custom prompt as you can see (DGN2200V4 / #).
2. What is the first command executed after the login?
3. Is there a way to debug all telnet communication?
Thanks for the help.
DGN2200V4 login: admin
Password:
DGN2200V4 / # adslctl info --show
xdslctl.orig: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 1125 Kbps, Downstream rate = 15144 Kbps
Bearer: 0, Upstream rate = 1110 Kbps, Downstream rate = 14003 Kbps
Link Power State: L0
Mode: ADSL2+
TPS-TC: ATM Mode
Trellis: U:ON /D:OFF
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 6.5 7.3
Attn(dB): 30.0 18.7
Pwr(dBm): 19.1 12.1
ADSL2 framing
Bearer 0
MSGc: 53 15
B: 240 13
M: 1 16
T: 2 8
R: 14 12
S: 0.5496 6.4000
L: 3712 295
D: 64 8
Counters
Bearer 0
SF: 695917 671733
SFErr: 2872 1
RS: 82118045 2756927
RSCorr: 28881373 332
RSUnCorr: 3316 0
Bearer 0
HEC: 2393 1
OCD: 0 0
LCD: 0 0
Total Cells: 372628502 29530284
Data Cells: 55041343 22426076
Drop Cells: 0
Bit Errors: 0 0
ES: 847 1
SES: 9 0
UAS: 58 58
AS: 11283
Bearer 0
INP: 0.50 1.00
INPRein: 0.00 0.00
delay: 9 13
PER: 16.21 16.80
OR: 29.11 10.00
AgR: 13978.21 1115.64
Bitswap: 2312/2312 1/1
DGN2200V4 /bin #
-
3. Is there a way to debug all telnet communication?
You may want to use wireshark to see if the commands were executed and what data is being sent to DSLstats.
-
The router doesn't require extra enters, I thought about that and debugged it manually, it's very clean, here's a simple telnet (windows telnet) session captured: as you can see, as soon as it connects, it asks for login, then enter, then pwd, then I'm at the prompt.
Yes I know that, but the HG622 still needs that extra kick when logging in using the particular telnet library used by DSLstats. It's probably actually a timing issue, which is why I suggested that you try the Special Login option as well as the HG622 option.
To answer your questions:
1. There's some limited checking for the prompt, but only to check when a line appears containing one of the characters #, $, > and : so it will accept the # from your model.
2. The first thing it does after a successful login is process any custom commands which have been set up (if any) and then it collects the main stats with adsl info --stats ('adsl' might be one of several different command prefixes).
3. As morphium says, wireshark might provide useful diagnostic information (but I won't be much use at interpreting it unfortunately).
-
With Special Login I cannot even pass telnet authentication. It fails with "unable to login to modem/router".
I know I can sniff the network, but I thought there was a way to debug what dslstats was sending/receiving on the application side. I thought there was a debug log somewhere, it could be useful for telnet.
I'm sorry that all the other softwares (DMT and RouterStats) work and DLSstats doesn't, I would have preferred to use this to monitor the line and router.
Thanks anyway.
-
I've just sniffed the telnet session with Wireshark, and it looks good, the adsl command is executed and the last line is the prompt.
The only thing that I can think of is that the ansi codes to color the prompt are being misinterpreted by DSLstats so the prompt is not recognised and it keeps waiting for the end of the output, then goes in timeout waiting for data.
Please check the attached log, it's raw data (tx/rx) of the captured session.
-
Here are two images of the captured sessions:
1. The first is a normal login, authentication goes fine, the adsl command is executed.
2. The second is with special login enabled (without delays or other things set, just set user and pw).
In case 1, right after the prompt is received, the adsl command is executed and when prompt is returned, the software authenticates and then tells me "no data returned".
In case 2, right after the promot, the software gives me the error: "unable to login to modem/router".
So in case 1, the adsl command is sent. In case 2, it is not sent, and even if authentication went ok the software says login error.
From this I can only think that the prompt string is not detected correctly, probably because of the ansi codes used to color it.
Could you confirm if this could be the reason?
Thanks.
-
Thanks for that extra information. It does seem possible that there's a confusion over the prompt. I'll look at it as quickly as I can, but I fear I may not be able to get to it until after Christmas. I'll do my best.
[Later] I think I may have found the problem (as you suggested, it seems that the extra formatting characters in the prompt string are the cause of the problem). After a bit more testing I'll try to upload some replacement executables later tonight.
-
Thanks for answering.
Bad news: I managed to change the prompt, modified in a plain simple one, with no strange color codes or characters (DGN2200V4 / #) and it still doesn't work. :(
I'm out of options now...hope you find the problem, if you want me to test something let me know.
Merry christmas. :)
-
Thanks for that extra information. It does seem possible that there's a confusion over the prompt. I'll look at it as quickly as I can, but I fear I may not be able to get to it until after Christmas. I'll do my best.
[Later] I think I may have found the problem (as you suggested, it seems that the extra formatting characters in the prompt string are the cause of the problem). After a bit more testing I'll try to upload some replacement executables later tonight.
I just read this after posting the previous message. Check my previous post, unfortunately even with a simple prompt it doesn't work, but who knows, let's give it a try. :)
If you need to test it before releasing it I'm more than happy to try it.
-
Yes, let's give it a try anyway. I'm assuming that you use Windows. Here's a replacement executable (just extract the .exe file and copy it over the v5.3 version of dslstats.exe). If it doesn't work, then I'm afraid it will have to wait until after Christmas. :)
http://www.s446074245.websitehome.co.uk/files/dslstats-5.3.1.exe.zip (http://www.s446074245.websitehome.co.uk/files/dslstats-5.3.1.exe.zip)
-
Wooooooooooooooooorks!!! :D
I had an error at first start (check the image below), after 10 seconds from connection. I found that Traffic Monitoring was enabled, but it was configured with a wrong command. I configured the right command (ifconfig and interface atm0). Restarted and now it's running fine. :)
Out of curiosity: what was the issue?
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FMeiQ04g.png&hash=8273039245778c7e78f97f80a6f5d154d9da17db)
-
I changed the router to its default prompt, with ansi color codes, and retried v5.3.1, and it works fine. :)
Thanks for this Christmas gift, I'll call you Santa Claus from now on.
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FSjAyz4z.png&hash=ecc4e37dfd5b7213c6d2e984abdf63487c7037e2)
-
That's a relief, thanks for confirming it.
The change I made was in the detection of the prompt after issuing the various stats retrieval commands - previously I was checking for the prompt character as either the last character of the output or the next to last. Now I look for it in the last 20 characters (or anywhere in the string for commands returning short outputs).
-
That's a relief, thanks for confirming it.
The change I made was in the detection of the prompt after issuing the various stats retrieval commands - previously I was checking for the prompt character as either the last character of the output or the next to last. Now I look for it in the last 20 characters (or anywhere in the string for commands returning short outputs).
Hi
You may have noticed that MyDSLWebStats is not displaying many of your graphs. Reason is that, and you are in Italy I believe, the equipment you are using to upload has European formatting for decimal points and hence the uploaded data has commas wherever they occur. This stops graphs displaying that use float values and totally screws up the Hlog and QLN graphs as well. The facility was really only designed for UK users as it warns on the Registration screen although you are welcome to use it of course.
If you can change the formatting to use periods instead of commas, it will work okay. You are also an hour ahead of the UK which may make the timeline a little odd. I imagine there are also bandplan differences that may affect some of the tone-related displays.
I'll fix the comma problem longer term so the formatting doesn't need changing.
-
Tony,
thanks a lot for the prompt advice. When I tested this version, I took a quick look at mydslwebstats and noticed that some graphs were not displaying, but it was too late to debug. :)
Thanks for letting me know about the issue. You say I can fix the formatting, but I can't find the option in DSLstats, or is it on the server side?
-
That's a relief, thanks for confirming it.
The change I made was in the detection of the prompt after issuing the various stats retrieval commands - previously I was checking for the prompt character as either the last character of the output or the next to last. Now I look for it in the last 20 characters (or anywhere in the string for commands returning short outputs).
My pleasure, and thank you very much for the prompt support. I think your solution should be good for any possible prompt style now. Eventually, if I'm allowed to make a suggestion, consider a debug tab in the telnet section where you output what you send/receive. Not all users can sniff the network and give you details about what's happening on the communication side. :)
Now I have to think about running the software on my NAS (Synology DS1812+), so I don't need to keep the notebook turned on h24. I saw the raspberry version, but it's not quite what I need. Anyway, you made an amazing work. :)
Happy holidays...:)
-
Thanks for your kind remarks. :)
Eventually, if I'm allowed to make a suggestion, consider a debug tab in the telnet section where you output what you send/receive. Not all users can sniff the network and give you details about what's happening on the communication side.
That sounds like a good idea. It was certainly your network sniffing which led to the solution here.
I think the decimal character issue is probably down to me. As far as I know, DSLstats displays its own graphs properly, taking into account the different forms of the decimal divider, but the MDWS uploads make use of a float-to-string conversion which observes the standards of the locale. I should be able to use a different type of float-to-string conversion which always uses a full stop for the decimal divider.
-
That sounds like a good idea. It was certainly your network sniffing which led to the solution here.
I suspected the prompt since the beginning, also because I saw an inconsistent behaviour with special login, but I thought it was the ansi codes in the prompt. Anyway, I'm glad to have been helpful for this improvement. :)
I think the decimal character issue is probably down to me. As far as I know, DSLstats displays its own graphs properly, taking into account the different forms of the decimal divider, but the MDWS uploads make use of a float-to-string conversion which observes the standards of the locale. I should be able to use a different type of float-to-string conversion which always uses a full stop for the decimal divider.
One of the things I learned when I was a young developer is that one important rule is to never trust the user input. So while I agree you could address the issue, I think that the best solution would be on the server side. Which, when receiving data, converts it appropriately. From what I saw, the server side can work with other software too, so the rule applies there, it shouldn't "trust" what the applications send. :)
Is there an API to communicate with mydslwebstats or you just agreed on a data format and you send files?
-
Tony,
thanks a lot for the prompt advice. When I tested this version, I took a quick look at mydslwebstats and noticed that some graphs were not displaying, but it was too late to debug. :)
Thanks for letting me know about the issue. You say I can fix the formatting, but I can't find the option in DSLstats, or is it on the server side?
Local OS ...
-
That would have an impact on too many things Tony. :)
But thanks a lot, you've been very kind to alert me on this issue.
If my data creates problem on the server side please let me know and I'll disable the upload until this issue will be resolved.
-
That would have an impact on too many things Tony. :)
But thanks a lot, you've been very kind to alert me on this issue.
If my data creates problem on the server side please let me know and I'll disable the upload until this issue will be resolved.
It's not a problem on the server as such other than the data stored now for you has commas in where they exist which won't currently display graphs.
I'm unfortunately short of time to today to do much and I'm going to be away for a while but I've adjusted things so that for the moment, at least the SNRM graph will now display properly using existing uploaded data along with any others that don't have floating point data in! I'll see if I can fix some more tomorrow.
I said the Local OS as user morphium who is in Switzerland had the same problem but has resolved it as he's uploading with periods rather than commas, by changing the local OS settings I assumed.
Longer term it can get fixed by modifying the uploaded data but needs some experimentation/testing first. If roseway can do something with DSLStats that would be a lot easier but it's also more work for him that I suspect he doesn't need at the moment :'(
-
Yes he said that the function used uses OS localisation, so the problem lies there, but I also wrote that imho, the best solution, since the server side could also receive data from other apps, is that a check on data be done at that level.
Is there an API to communicate with the server or it's just a datafile transmission and it's then parsed on the server?
-
Yes he said that the function used uses OS localisation, so the problem lies there, but I also wrote that imho, the best solution, since the server side could also receive data from other apps, is that a check on data be done at that level.
Is there an API to communicate with the server or it's just a datafile transmission and it's then parsed on the server?
All the data is uploaded preformatted to the server by both of the apps that can upload.
I have also fixed a few other graphs plus HLog, SNR/Tone and QLN.
-
That's very kind of you Tony...but enjoy your holidays, this is stuff that can wait...no priority. :)
This forum is very nice, I found very kind and competent people.
So data should come from the clients with a predefined format, including decimal and thousands separators. I understand, thanks for the info. :)
-
Yes he said that the function used uses OS localisation, so the problem lies there, but I also wrote that imho, the best solution, since the server side could also receive data from other apps, is that a check on data be done at that level.
Is there an API to communicate with the server or it's just a datafile transmission and it's then parsed on the server?
All the data is uploaded preformatted to the server by both of the apps that can upload.
I have also fixed a few other graphs plus HLog, SNR/Tone and QLN.
Tony, I noticed most of the graphs are working fine now...there are a total of 6-7 out of all that don't work but I don't even know what they are. :)
One question: could you also address the timing issue by asking clients to upload data universally in GMT time then allowing users to configure mysdslwebstats to configure the proper timezone? Or ask clients to put the timezone into the uploaded data?
Thanks a lot for your work. :)
Thank you very much for this.
-
Tony, in 5.3.1 Eric also fixed the locale issue. Can you tell me if everything's fine now? I started using last 5.3.1 version some hours ago.
Thanks.
-
Tony, in 5.3.1 Eric also fixed the locale issue. Can you tell me if everything's fine now? I started using last 5.3.1 version some hours ago.
Thanks.
Shouldn't really see any difference and I can't here, all looks fine. Maybe there are other graphs working now that were not before though?
-
One question: could you also address the timing issue by asking clients to upload data universally in GMT time then allowing users to configure mysdslwebstats to configure the proper timezone? Or ask clients to put the timezone into the uploaded data?
Thanks a lot for your work. :)
Thank you very much for this.
Looking at the graphs with a timezone an hour ahead on a laptop, there doesn't seem to be much of a problem - the main criteria is that your own graphs show the time correctly along with data up to that time and other users have theirs correctly shown also. There are a couple of places where the time is GMT and should be an hour ahead or vice-versa but I guess these are not a deal breaking problem.
The work involved in trying to fix this both for the local client and the server is not something I can attempt at the moment or in the foreseeable future though I'm afraid due to other 'real' work commitments.
-
One question: could you also address the timing issue by asking clients to upload data universally in GMT time then allowing users to configure mysdslwebstats to configure the proper timezone? Or ask clients to put the timezone into the uploaded data?
Thanks a lot for your work. :)
Thank you very much for this.
Looking at the graphs with a timezone an hour ahead on a laptop, there doesn't seem to be much of a problem - the main criteria is that your own graphs show the time correctly along with data up to that time and other users have theirs correctly shown also. There are a couple of places where the time is GMT and should be an hour ahead or vice-versa but I guess these are not a deal breaking problem.
The work involved in trying to fix this both for the local client and the server is not something I can attempt at the moment or in the foreseeable future though I'm afraid due to other 'real' work commitments.
It's a good result for now. With time you'll work out a definitive solution. :)
Thanks a lot.
-
It's a good result for now. With time you'll work out a definitive solution. :)
Thanks a lot.
Done :)
There are now changes to support all international users in Time Zones other than the UK providing they have a compatible modem/router that works with DSLStats primarily.
Panes now display the correct local times for Non-UK users. Non-UK users have an asterisk against their User Names in the Whois online/uploading displays and the offset from UK time in Hours is shown at the top of a Control Box against their User Name. There are now users in France, Switzerland, Italy, USA and Canada although not all are currently uploading.
-
Great job Tony, at a first glance everything's working as expected. :)
-
If there only was a DLStats client, web based, so we could run it, headless, on a small NAS, a raspberry etc. it would be perfect in combination with MDWS:
http://forum.kitz.co.uk/index.php?topic=14595.msg278083#msg278083