Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: My iPad tool is finished  (Read 2238 times)

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
My iPad tool is finished
« on: August 20, 2021, 10:51:10 PM »


I may have mentioned that I’ve been working on an iPad utility that summarises the state of my three DSL lines and ZyXEL modems; it reports errors, ES, sync rate and SNRM out of range, that kind of stuff. It distils the whole lot down taken from the modems’ detailed stats into a three-way good / slightly bad / seriously bad tri-state value and otherwise only reports the minimum amount of information. The first version of the tool is now finished and here is a screenshot:



I have some DSL problems shown by the display, with line #1 upstream running too fast, and the other lines running a bit too fast at 3dB downstream SNRM too. The tool queries the modems using our own member Johnson’s stats access over http protocol which is a feature provided by his custom firmware for my ZyXEL VMG 1312-B10A modem (and the ZyXEL 8x24 modems too).

The app moans "serious" if you have more than a certain threshold of ES per hour; I have it set to 60 ES/hr - what do you think?

If the down- or upstream sync rate is too low the app reports a warning, and so too if the SNRM is way out of range, in case that has been caused by some serious trouble.

Just for fun, I might adapt it to handle 8x24 modems too. Could someone send me the detailed varStats page from a ZyXEL 8x24 modem, the one that gives detailed stats and ‘latest’/‘previous’ 15-mins
Logged

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: My iPad tool is finished
« Reply #1 on: August 21, 2021, 12:57:06 AM »

Congratulations. I'm pleased to know that you have a version working to your satisfaction.  :)

The app moans "serious" if you have more than a certain threshold of ES per hour; I have it set to 60 ES/hr - what do you think?

I'm not sure what to suggest. Perhaps "tune" it over a period of time, as you become accustomed to its output?
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: My iPad tool is finished
« Reply #2 on: August 21, 2021, 09:47:22 AM »

Well done!


Perhaps if the Err Secs were over a shorter time frame and thus grouped more together it would be more likely to highlight an error spike.... but this is the reason why I stopped using alerts for Err Secs.   
I personally find that an alert at >200 CRCs per min or 500 FECs per min gives me a better indication if the line is incurring problems.
 
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: My iPad tool is finished
« Reply #3 on: August 21, 2021, 02:38:35 PM »

Hi Kitz, wow good to know. Looking at my lines over a year or two it seems that for me ES are very roughly equivalent to CRC (very very roughly), and I could have used either. I was talking about this to one of the (clued up) OR engineers who recommended using ES.

I’m using the buckets that the modem’s detailed stats (varstats) uses, scaled up from eg 15 min to be per hr; I’m not counting them in an hour, in fact I’m not counting ES at all, the modem is doing that, just to make that clear. I’m merely collecting the reports from the modem and summarising them, I’m not polling the modems. The limitations of the iPad and especially those of the iOS Shortcuts programming framework would make that impossible.

If one of the modems is out of action, I mark this in the config in order to prevent a hang when trying to query that modem, and this appears, noted, in the first section of the report. If one of the modems is in the middle of a resync or has not come up yet, ie it is not in ‘ShowTime’ yet, then this is detected and a sensible status error message is reported just below the first section.


@Kitz - would you post up the varstats ie detailed stats page with 15-min, 24 hr etc event counts for your modem? I’d be interested to see how much it differs from a VMG 1312-B10A.



One thing that I haven’t fully managed to decide on yet is the choice of "out of range" settings for SNRM values. My line #2 goes up and down in SNRM in an irregular square wave fashion every day and this irregularity and high variation makes it difficult to decide on what is "usual" and "ok" and what is not. On idea I have had is having per-line variable min low and max high acceptability settings for SNRM.
« Last Edit: August 21, 2021, 03:47:08 PM by Weaver »
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: My iPad tool is finished
« Reply #4 on: August 21, 2021, 07:07:26 PM »

It not something you can get from the standard connection stats without collecting each minute. ;/
I use DSLstats to collect mine.  iirc you don't have a PC, but you can successfully run it on a Raspberry Pi


>>  would you post up the varstats ie detailed stats page with 15-min, 24 hr etc event counts for your modem?

Here you go.  I think its similar

Code: [Select]
adsl info --stats
adsl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:    0
Last initialization procedure status:   0
Max:    Upstream rate = 25029 Kbps, Downstream rate = 66294 Kbps
Bearer: 0, Upstream rate = 20000 Kbps, Downstream rate = 60000 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Link Power State:       L0
Mode:                   VDSL2 Annex B
VDSL2 Profile:          Profile 17a
TPS-TC:                 PTM Mode
Trellis:                U:ON /D:ON
Line Status:            No Defect
Training Status:        Showtime
                Down            Up
SNR (dB):        7.4             7.3
Attn(dB):        0.0             0.0
Pwr(dBm):        4.5             4.4

                        VDSL2 framing
                        Bearer 0
MSGc:           -6              150
B:              178             236
M:              1               1
T:              0               5
R:              10              16
S:              0.0950          0.3771
L:              15922           5410
D:              1               1
I:              189             255
N:              189             255
Q:              16              0
V:              2               0
RxQueue:                33              0
TxQueue:                11              0
G.INP Framing:          18              0
G.INP lookback:         11              0
RRC bits:               0               24
                        Bearer 1
MSGc:           122             -6
B:              0               0
M:              2               0
T:              2               0
R:              16              0
S:              8.0000          0.0000
L:              32              0
D:              1               0
I:              32              0
N:              32              0
Q:              0               0
V:              0               0
RxQueue:                0               0
TxQueue:                0               0
G.INP Framing:          0               0
G.INP lookback:         0               0
RRC bits:               0               0

                        Counters
                        Bearer 0
OHF:            0               2347950
OHFErr:         3               299
RS:             4101169520              573323
RSCorr:         336796          3005
RSUnCorr:       0               0
                        Bearer 1
OHF:            69813914                0
OHFErr:         0               0
RS:             558510817               0
RSCorr:         27              0
RSUnCorr:       0               0

                        Retransmit Counters
rtx_tx:         495616          0
rtx_c:          415             0
rtx_uc:         62              0

                        G.INP Counters
LEFTRS:         0               0
minEFTR:        59987           0
errFreeBits:    1026068011              0

                        Bearer 0
HEC:            0               0
OCD:            0               0
LCD:            0               0
Total Cells:    540830305               0
Data Cells:     1787004815              0
Drop Cells:     0
Bit Errors:     0               0

                        Bearer 1
HEC:            0               0
OCD:            0               0
LCD:            0               0
Total Cells:    0               0
Data Cells:     0               0
Drop Cells:     0
Bit Errors:     0               0

ES:             2               262
SES:            0               0
UAS:            28              28
AS:             1121677

                        Bearer 0
INP:            48.00           0.00
INPRein:        0.00            0.00
delay:          0               0
PER:            0.00            6.15
OR:             0.01            202.87
AgR:            60083.56        20203.27

                        Bearer 1
INP:            2.00            0.00
INPRein:        2.00            0.00
delay:          0               0
PER:            16.06           0.01
OR:             63.75           0.01
AgR:            63.75   0.01

Bitswap:        375/376         396/400

Total time = 1 days 23 hours 35 min 7 sec
FEC:            336796          3005
CRC:            3               299
ES:             2               262
SES:            0               0
UAS:            28              28
LOS:            0               0
LOF:            0               0
LOM:            0               0
Latest 15 minutes time = 5 min 7 sec
FEC:            0               0
CRC:            0               0
ES:             0               0
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
LOM:            0               0
Previous 15 minutes time = 15 min 0 sec
FEC:            0               0
CRC:            0               0
ES:             0               0
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
LOM:            0               0
Latest 1 day time = 23 hours 35 min 7 sec
FEC:            131             192
CRC:            0               23
ES:             0               20
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
LOM:            0               0
Previous 1 day time = 24 hours 0 sec
FEC:            76702           240
CRC:            0               23
ES:             0               19
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
LOM:            0               0
Since Link time = 12 days 23 hours 34 min 37 sec
FEC:            336796          3005
CRC:            3               299
ES:             2               262
SES:            0               0
UAS:            0               0
LOS:            0               0
LOF:            0               0
LOM:            0               0
 >
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: My iPad tool is finished
« Reply #5 on: August 27, 2021, 08:46:07 AM »

I said "it is finished", and straight away, there’s a new version, v3.01. The config module that supplies data to the program about what is expected behaviour for the various lines/modems has been upgraded; it now offers a pair of values, min and max SNRM, per link per up/down direction, whereas before there was just one configured SNRM ‘target’ value. Now if a modem goes outside the min-max range the tool displays warnings, whereas before it just used calculated min and max values, derived from one single target SNRM value by multiplying by fixed multipliers, and the old way you could not have lines with very different behaviour in respect of their daily SNRM variation (or none). My Line #2 upstream goes up and down like a yo-yo every day but that’s just natural for that line and now that link’s min and max SNRM values are set wide apart to handle this. The other links have min and max values set fairly close together for the normal case.

It may never be finished. I might carry on tweaking it and improving it from time to time. I’m already thinking about ‘individual specific disease detector’ modules, the first of which might be a ‘hollow curve’ disease detector module. I haven’t decided that I’m definitely going to do this, not yet. I would like to get hold of some bit-loading vs tones graphs for other lines, especially a very fast ADSL2+ one and some VDSL2 ones.

Thanks to Kitz for posting her adsl info —stats. I ran my program on Kitz’s data file as input, found one small bug and made one very small adjustment and now it works perfectly with a simulated ZyXel 8x24 modem doing VDSL2. Fortunately the program has very wide tolerances for acceptance of syntactically irrelevant or extraneous content in the input from the modem. The differences in syntactic format that mattered to me between Kitz’s file and the output from my own VMG1312-B10A were extremely minor. Some debugging of regexes was required where one model used tabs and the other used spaces in one place, and I had stupidly made assumptions about which was used, very naughty and not my usual way at all. For testing and debugging regexes I do love the https://regex101.com/ website, and I give them donations when I can. The program ticked Kitz off for excessively high SNRM, ie above configured max. That was one example that showed it was parsing and picking up the detailed data items in the file properly.
Logged

tubaman

  • Senior Kitizen
  • ******
  • Posts: 12472
Re: My iPad tool is finished
« Reply #6 on: August 27, 2021, 09:34:48 AM »

...
I personally find that an alert at >200 CRCs per min or 500 FECs per min gives me a better indication if the line is incurring problems.
 

Your line is really clean, as mine runs at around 4000 FECs/min on the DS in normal working - normally zero CRCs though.
 :)
Logged
BT FTTC 55/10 Huawei Cab - Zyxel VMG8924-B10A

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: My iPad tool is finished
« Reply #7 on: August 27, 2021, 05:04:48 PM »

I would like to get hold of some bit-loading vs tones graphs for other lines, especially a very fast ADSL2+ one and some VDSL2 ones.

Data for a (real) VDSL2 circuit, at "The Cattery", is available. Just let me know when you would like a copy.
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 33879
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: My iPad tool is finished
« Reply #8 on: August 28, 2021, 08:33:33 PM »

Quote
I would like to get hold of some bit-loading vs tones graphs for other lines, especially a very fast ADSL2+ one and some VDSL2 ones.

My stats are here if you wish to see what figures I log.   The real time remote upload is paused atm but the figures dont change much, so it shouldn't make any difference for want you are wanting to do.  When I get chance I'll clear out the old stored history files and re-commence uploading every min.

https://kitz.co.uk/linestats/

Click the various pages to see output of relevant commands from my Zyxel.
 

I think chrys also is real time logging too, so theres some more stats you can grab.   The link to his are in his forum sig.
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker
 

anything