Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: Help Requested - Attempting to monitor Noise from HLog and SNR  (Read 2042 times)

boozy

  • Reg Member
  • ***
  • Posts: 119

This is something I've been playing with to try and measure noise on my line.  It's grown a few arms and legs as I look at things, so I thought I would put it up in case anyone finds it useful.  Most useful if you have two or more vdsl circuits.

Despite my best intentions I never got to the modem configuration - so anyone using it will have to edit the Modem.config file themselves.

Here's the program (called DslStats.Net as I couldn't think of anything else better than the Form1 it was called very recently, I'm open to suggestions): https://1drv.ms/u/s!AofPcWieuJD2trcWmKAr981H5lU8oA

You can add 5 modems to it (5 beacuse I haven't done the config page and have only 5 colour sets hard-coded).  Each modem has an entry like:

Code: [Select]
  <Base xsi:type="ZyxelVMG8924B10A"> <!--Possible Values ZyxelVMG8924B10A, ZyxelVMG8324B10A, ZyxelVMG1312B10A, ZyxelVMG1312B10B, ZyxelVMG1312B10D, HG635, HG612  -->
    <IP>192.168.1.1</IP>
    <Password>*******</Password>
    <FriendlyName>VMG8924-B10A</FriendlyName>
    <Username>admin</Username>
    <Port>23</Port>
  </Base>

The type is important as it is how the modem is created in memory together with any thing special that modem needs to emit stats (e.g. a shell command).

Things I know that are broken:
The telnet client isn't 100% and can decide it's finished getting output incorrectly and it can miss a bit in the middle of the output.
There is little error checking - If you fill the Config incorrectly the program won't start.  Most of the time the program will fail gracefully and the chart will just not show whatever has an error (the underlying data is refreshed every 60 seconds, but not what you are looking at - so to see an update you'd need to pick it again from the menu).
It tries to work with DslStats, but not always sucessfully :).
The click and drag zoom draws incorrectly, but works as expected (right click graphs for more options).
I have yet put in on a linux box for Mono testing, so only Windows is guaranteed to be working.
One of the projects is copying some extra assemblies, so the zip is much bigger than it should be.
I don't have an ADSL modem here, so it might default to 4096 tones or fail.


Help I would like:

I'm trying to figure out how to determine the noise on the circuit, which I'm doing by predicting SNR from the QLN and HLog and it seems pretty accurate (would likely be better if it had the SNR at the time the other two were captured), but I want to know if it's generic or per line.  I'd appreciate some grabs of that screen and/or the data from the Debug directory.


How to use it:

There's only two buttons - start and stop.  These start/stop capturing.  Only the first grab of data will hold the program up and a background thread is used after that. 

The very first press of start captures all the data in the Debug Directory whether it fails or not - which will be useful to fix anything.

There are 6 graphs at present the first 3 are comparisons across the modems - with the other 3 specific to a particluar modem:

SNR - SNR per tone with each modem in a different colour set. https://1drv.ms/u/s!AofPcWieuJD2trcSvvILFJY8kxdwJA
HLog - ditto for HLog  https://1drv.ms/u/s!AofPcWieuJD2trcUIf7wbDSf2cYF7w
QLN - Same again https://1drv.ms/u/s!AofPcWieuJD2trcVApmGWjqoZkdrBw

Health Graph - A combined graph showing QLN and HLog, together with the bands that were tried and which were actually used https://1drv.ms/u/s!AofPcWieuJD2trcTF3t8STjqiNRTXA
Bits - Bit Loading against the SNR https://1drv.ms/u/s!AofPcWieuJD2trcRf2o0f8QFEWeF5g
Predicted SNR Per Tone - Derives a formula for SNR (Upstream and DownStream separately).  I've ignored ADSL tones due to the extra noise there https://1drv.ms/u/s!AofPcWieuJD2trcQJ_lr_w9-2WAPww

And a sample of what a zoomed graph looks like https://1drv.ms/u/s!AofPcWieuJD2trcPq5e9i5bvD3XXNg



If anyone needs a particular BroadCom Modem added I just need the name, stats command and, if necessary, the shell command.
Logged

IanG

  • Member
  • **
  • Posts: 40
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #1 on: June 04, 2019, 12:49:34 AM »

Hello,

I did run your program, but without success, as it didn't get as far as producing a window, or sending out any packets. Nor did it dump anything in the debug folder. After poking about a bit, I suspect that a file called Analyzers.dll is missing, but may be wrong.

I also have been working on logging software to suit my needs, as programs I had previously tested were not suitable. You are right about automated telnet comms being somewhat tricky. I spent quite some time writing telnet negotiation commands to turn off the echo, and wondering why it didn't work, before discovering that the telnet client in most modems is brain dead, and will echo commands whatever you tell it. So the telnet response to a command is sometimes, but not always, an immediate echo of the command followed by a short delay before the main response. For what it's worth, I find the response to an SNR spectrum request takes about 100 ms over a wireless link, and involves about 30 IP packets. But as there is nothing to prevent the modem from deciding to take time off to do something else, it may be necessary to look at what has arrived, and figure out if there is more to come. I send more than one command at a time, and would prefer to wait for one command to be answered before sending the next one.

Good luck,  anyway.
Logged

IanG

  • Member
  • **
  • Posts: 40
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #2 on: June 05, 2019, 04:12:44 PM »

Good news. I did a lot more poking around and found the problem, which is a new one on me, and involved the longest .NET exception message I have seen.

An attempt was made to load an assembly from a network location which would have caused the assembly to be sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly, please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more information.

That link leads to the following page about unblocking downloaded .NET assemblies.

https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2010/ee890038(v=vs.100)

In the attached image I have outlined the unblocking option. Only the one file needs to be unblocked. I was then able to use your program to capture data from the Netgear DGND3700v2 modem. The zipped data is also attached.

Two of the modem commands ("info --pbParams" and "info --vectoring") didn't get a sensible response, because they aren't supported.

xdslctl info --pbParams
xdslctl: too many parameters
#


Command syntax is shown below.

# xdslctl
Usage: xdslctl start [--up] [--mod <a|d|l|t|2|p|e|m|M3|M5>] [--lpair <(i)nner|(o)uter>]
           [--trellis <on|off>] [--snr <snrQ4>] [--bitswap <on|off>] [--sesdrop <on|off>]
           [--sra <on|off>] [--CoMinMgn <on|off>] [--i24k <on|off>] [--phyReXmt <0xBitMap-UsDs>]
           [--Ginp <0xBitMap-UsDs>]  [--TpsTc <0xBitMap-AvPvAaPa>] [--monitorTone <on|off>]
           [--forceJ43 <on|off>] [--toggleJ43B43 <on|off>]
       xdslctl stop
       xdslctl connection [--up] [--down] [--loopback] [--reverb]
           [--medley] [--noretrain] [--L3] [--diagmode] [--L0]
           [--tones] [--normal] [--freezeReverb] [--freezeMedley]
       xdslctl configure [--mod <a|d|l|t|2|p|e|m|M3|M5>] [--lpair <(i)nner|(o)uter>]
           [--trellis <on|off>] [--snr <snrQ4>] [--bitswap <on|off>] [--sesdrop <on|off>]
           [--sra <on|off>] [--CoMinMgn <on|off>] [--i24k <on|off>] [--phyReXmt <0xBitMap-UsDs>]
           [--Ginp <0xBitMap-UsDs>] [--TpsTc <0xBitMap-AvPvAaPa>] [--monitorTone <on|off>]
           [--forceJ43 <on|off>] [--toggleJ43B43 <on|off>]
       xdslctl bert [--start <#seconds>] [--stop] [--show]
       xdslctl afelb [--time <sec>] [--tones] [--signal <1/2/8>]
       xdslctl qlnmntr [--time <sec>] [--freq <msec>]
       xdslctl inm [--start <BB_THRESH 10*dB> <INMIATO> <INMIATS>] [--stop] [--show]
       xdslctl snrclamp [--shape <shapeId>] [--bpshape [bpIndex-bpLevel,]]
       xdslctl nlnm [--show ] [--setThld <Thld_Num_Tones>]
       xdslctl diag [--logstart <nBytes>] [--logpause] [--logstop] [--loguntilbufferfull <nBytes>] [--loguntilretrain <nBytes>]
       xdslctl info [--state] [--show] [--stats] [--SNR] [--QLN] [--Hlog] [--Hlin] [--HlinS] [--Bits]
           [--linediag] [--reset] [--vendor] [--cfg]
       xdslctl profile [--show] [--save] [--restore]
       xdslctl --version
       xdslctl --help
#
Logged

boozy

  • Reg Member
  • ***
  • Posts: 119
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #3 on: June 05, 2019, 08:50:02 PM »

Thanks Ian,

I'd forgotten about the blocked assemblies problem - I'll recompile against v3.5, it should work better and still be Mono compatible.

The pbParams not being available is annoying as (at least) Hlog output can be wrong if you can't see the final band values from pbParams, data outside that final band range is not updated on a resync. 

Your other outputs look OK and it's ADSL which I don't have access too.  I think I've figured out how not to need to know the modem type by adding an extra two configuration items so I've a few modifications to make.  It'll be a few days as I've just been on a few trips to London and am very behind on sleep.

I'll post the source when I get it stable.  I intend to write all the data to a DB, so it'll be before I do that as that'll be the point it's a pain for others to use; B*cat provided some useful docs which will mean the amount of data needing to be stored is much smaller than it seems.

Cheers
Logged

burakkucat

  • Global Moderator
  • Senior Kitizen
  • *
  • Posts: 27839
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #4 on: June 05, 2019, 09:10:25 PM »

By concatenating the *.out files (to create a Snapshot_Log.txt file) I can then create the four usual snapshot plots . . . a montage of which is attached, below.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

IanG

  • Member
  • **
  • Posts: 40
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #5 on: June 07, 2019, 04:44:00 PM »

If anyone is interested, one way to mitigate the uncertainly of the telnet timing response is to concatenate multiple commands. This requires the modem to have a type-ahead buffer, and the PC receiver not to overload its receive buffer. If command sets are to be sent at regular intervals, one can wait until the end of an interval before analysing the responses to the previous set.

Purely as an experiment, I sent a Netgear modem a 2 kB string consisting of 100 Hlin requests. Over the following few seconds I received a 1.5 MB reply, consisting of 100 correct responses. A TP-Link modem also gave 100 correct responses.

To separate out the responses, the telnet command prompt is a useful delimiter.
Logged

boozy

  • Reg Member
  • ***
  • Posts: 119
Re: Help Requested - Attempting to monitor Noise from HLog and SNR
« Reply #6 on: August 19, 2019, 11:32:56 PM »

Some things at home interrupted but I've done a bit of tinkering (not much as home life kept me busy).

1. I altered the telnet code so that it now reliably gets the results of a command.
2. Configure Modems now allows Broadcom based modems to be added/removed and updated.
3. I've changed the internal data structures to allow me to add my fritzbox too (not added, but the structures are now compatible).
4. I've disabled being able to load someone else's debug files - related to making it more generic for non Broadcom modems.  This will be temporary.
5. Graphs can be saved by right clicking on them and choosing "Save Image as"

Windows users will need to unblock downloaded assemblies to allow it to run, but it should be fine for Linux users under mono (tested using the Linux subsystem in Windows...  so about 95% confident).

As before I'm most interested in the Predicted SNR per tone and the associated data.
Logged
 

anything