Kitz Forum

Broadband Related => Broadband Hardware => Topic started by: Weaver on May 17, 2018, 02:40:21 AM

Title: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 17, 2018, 02:40:21 AM
Referring to the v14-based custom firmware for the ZyXEL VMG 11312-B10A on github (https://forum.kitz.co.uk/index.php/topic,21273.msg370865.html#msg370865) that was referred to in a neighbouring thread earlier (https://forum.kitz.co.uk/index.php/topic,21273.msg370534.html#msg370534), I would very much like someone here to build it from the sources, if they have not already done so, or help me do it myself. (To do it myself though I would need to get a VM (i386 / P6) going for a build machine and a lot of help because I although a very experienced C programmer I am very new to Linux still.)

This is partly because I would really like to be able to keep this branch up to date with other fixes and updates that come from the main branch of development and from other Broadcom and ZyXEL insights.

Would anyone be kind enough to join in / offer to help us out? There are a few of us using this custom v14-based firmware now.

[Admins: would it be an idea to make this a new thread, what do you think ?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on May 17, 2018, 03:06:31 AM
I have built it from a git clone of Olipro's repo and it successfully boots, I have yet to test it on my line though.

I found a few hiccups to get around in the build process and was brewing up to making a thread like this myself detailing them.

I have also acquired the VMG1312-B10A and VMG8324-B10A sources from zyxel and can share the dropbox links they gave me.

Next on the list is trying to build straight from zyxels sources, and trying the changes for the 8324.

 
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 17, 2018, 03:15:15 AM
Your efforts are very very much appreciated sir. Thank you! Indeed, please keep us informed, am very interested.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 17, 2018, 03:34:01 AM
Btw I presume it uses a C or C++ cross-compiler, for what? MIPSx, ARMx target ?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on May 17, 2018, 03:40:37 AM
Yeah, Zyxel supplied a cross compiler for the MIPS architecture the BCM63168 is based on.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: j0hn on May 17, 2018, 09:50:43 PM
I got copies of both the 1312-B10A and 8924-B10A open source packages direct from Zyxel.
I built my own firmware image rather than use the 1 on git-hub.

As long as Zyxel keep sharing the complete build packages then I'd be happy to keep modifying them for 1508 MTU.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 17, 2018, 10:33:50 PM
John that would be very good of you, most appreciated. And I'm sure others will agree too.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on May 18, 2018, 02:22:32 AM
So I got around to trying to build from zyxels sources and it just worked, the issues I ran into compiling from the github repo disappeared.

Just follow the instructions that come in the source package and out pops the image.

Troubleshooting the previous build did reveal something interesting though, one of the things that failed to build initially was conntrack-tools, and watching the boot log with a serial terminal it seems even in the official firmware its not built correctly:

Code: [Select]
Netfilter messages via NETLINK v0.30.
nf_conntrack_netlink: Unknown symbol __nf_ct_l4proto_find
nf_conntrack_netlink: Unknown symbol nf_conntrack_find_get
nf_conntrack_netlink: Unknown symbol nf_conntrack_register_notifier
...
nf_conntrack_netlink: Unknown symbol nf_conntrack_l3proto_generic
nf_conntrack_netlink: Unknown symbol nf_ct_unexpect_related
nf_conntrack_netlink: Unknown symbol nf_ct_remove_expectations
insmod: can't insert '/lib/modules/2.6.30/kernel/net/netfilter/nf_conntrack_netlink.ko': unknown symbol in module, or unknown parameter

Not sure what effect not having this module working has if any, conntrack-tools appear to be for monitoring stateful packet inspection.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 18, 2018, 08:57:19 AM
That's pretty horrendous, doesn't say much about their attitude to quality if they can't even get their build system bombproof. They should be greping the output and / or checking the return status and making sure that the build then fails completely and loudly.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: tubaman on May 18, 2018, 09:13:45 AM
That's pretty horrendous, doesn't say much about their attitude to quality if they can't even get their build system bombproof. They should be greping the output and / or checking the return status and making sure that the build then fails completely and loudly.

If you think that's bad it's probably best that you don't look at the logs produced by an average Windows PC as it boots!
The attitude nowadays seem to be 'if it works then that's good enough'. :o
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 23, 2018, 09:33:20 PM
Yet another Thank You to Johnson - https://forum.kitz.co.uk/index.php/topic,21273.msg373268.html#msg373268
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 17, 2018, 01:50:20 PM
How does the build set that funny long-winded version number string? Could we somehow append something to it to indicate which customer version we have got but also including the base ZyXEL version string that it is based on?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 17, 2018, 01:55:26 PM
Thats a good question, would be nice to append more descriptive version info that could be seen from telnet and the firmware update page for the myriad different ones, I'l have a dig around in the build package.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 17, 2018, 02:01:28 PM
Have also put in a request for the latest source package since the AAJZ.16 release happened if people want it, even though the changes are minor.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 17, 2018, 02:27:56 PM
It shows that version string in the web GUI somewhere, there is a menu option to get to it anyway, It is iirc the first menu item in the rightmost menu.

I use it to remind me which modem I am looking at since I have so many on the go concurrently. I have made the contents of the three modems’ config files different in that respect as well as a couple of other things.

I am writing a simple ipad app to take one config file and generate derived versions of it for modem n.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 17, 2018, 02:38:26 PM
Sorry if I misunderstand you, but are you finding a more detailed version string than:

 1.00(AAKL.21)C0

?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 17, 2018, 05:09:30 PM
Looking at it again I realise now that I was wrong, my apologies. In the web GUI it shows two fields which are just the contents of two particular user-defined XML elements from the config file and so can be any strings you want. I had chosen to put in some text that looked a bit like a version number so that was what lead me astray, thinking it was the version number from the image file. My apologies.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 17, 2018, 05:13:33 PM
No problem, was just making sure I wasn't missing something.

More detailed version strings that show up in the update page and else where would definitely be useful though.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on June 18, 2018, 12:56:35 PM
does the vmg1312-b10a hardware support 30a? also some "official rumor" about my ISP:

http://en.globes.co.il/en/article-bezeq-upgrading-to-200-mbit-surfing-speeds-1001235706

so i'm getting vectoring if I'm on 17a too?

PS: 250/7.5 is what I expect from Bezeq :-X so i'm staying on vmg1312 as 40/3 is enough [next step is 100/3  ???]
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: j0hn on June 18, 2018, 02:00:53 PM
does the vmg1312-b10a hardware support 30a?

No, it only supports up to 17a

also some "official rumor" about my ISP:

http://en.globes.co.il/en/article-bezeq-upgrading-to-200-mbit-surfing-speeds-1001235706

so i'm getting vectoring if I'm on 17a too?

No, not necessarily.

Your ISP appears to be rolling out profile 35b, not 30a.

*I very much doubt your ECI M41 DSLAM can be upgraded to accommodate profile 35b.

They will probably be fitting different DSLAMs. Profile 17a and profile 35b share tones so vectoring can be done over both at the same time so you may indeed get vectoring while on 17a.

*The ECI M41 DSLAM doesn't support system level vectoring, only line card vectoring.
This is incredibly inefficient and requires individual phone bundles to be routed to the same line cards.
I'd be very surprised if it could be upgraded to run system level vectoring with 17a/35b combined if they couldn't even do 17a right.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on June 18, 2018, 02:08:57 PM
I know its 35b xd just thought maybe 30a would be relevant somehow? oh well I don't need vectoring or more than 17a [we all prefer it though], was just hoping, hopes shattered then  :D

Quote
*I very much doubt your ECI M41 DSLAM can be upgraded to accommodate profile 35b.
I thought they'd mention fitting another cab, the rumor says it's "already installed"  :-\

Edit: right, rumors...  :P
Quote
"Globes" has learned that the new equipment is deployed in all parts of Israel, but it has not yet been activated, which is expected to be expected in a few months. "

Bezeq stated that "Bezeq is constantly examining various possibilities for improving and upgrading its communications infrastructure, as is the case in this case. These are preliminary stages of examining the technology and its capabilities, and there is no detailed plan for its use. "

This sentence says that everything is currently talk and only God knows when it will actually take effect or where. In addition, as noted above, Bezeq and its members collect thousands of shekels from business customers on symmetric lines, so that in the future (even far), we will not see a symmetric line for private customers from Bezeq.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 19, 2018, 02:03:06 AM
Found where the version string is stored, but it seems the webinterface truncates it to cut off anything after 1.00(AAJZ.14)C0.. so probably have to shorten the whole thing to get any useful info represented by it.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 27, 2018, 09:59:25 AM
Here are 2 builds from the latest AAJZ.16 version for the VMG1312-B10A, the differences to last version are minor so no real need to upgrade if you dont feel like it. Just a version with the jumbo frames patches and another with the old telnetd binary if you enjoy having more than one telnet session open at a time, no updated dsl_phy as it seemed it only benefited me and caused more confusion than good.

Both tested to boot, but not on a line yet.

https://www.dropbox.com/sh/c5no9h8ckyp2i4i/AADwAR6oVbzwFjtE-bLdy81Za?dl=0
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on June 27, 2018, 05:05:10 PM
Thank you for building and make those images available to all.  :)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 27, 2018, 06:36:02 PM
Many thanks, much appreciated!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 27, 2018, 07:32:18 PM
I did not know how to give johnson a donation to thank him, so I gave one to Kitz instead. Thanks guys.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 27, 2018, 10:47:33 PM
What is your experience if the sync rate following this upgrade? My sync rates are really not good compared with what they were, but I broke all the rules of proper testing and it could just be because I have resynched and nothing more. Conditions may have simply changed. I should have done a comparison test with the same firmware first.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 27, 2018, 11:26:06 PM
Could someone remind me where we are with low-level DSL driver versions included? Is d26a the version number for the low-level DSL driver that is included in this latest release?

Another question: could someone help me with a recap on performance of the recent DSL driver releases - opinions on merits of the different releases?

I have a bit careful with trying to comparison test the different releases regarding performance because my three lines have rather different characteristics, for reasons unknown, and also simply resyncing a lot without even changing software can possibly cause odd annoying effects on sync rate, and that can muck up the results. It is very easy to get completely bogus results.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on June 28, 2018, 01:30:28 AM
i have no DLM, but my line is VDSL, yours is ADSL, so I can't exactly test for you

about different releases the adsl_phy needs to be from the same SDK version [open source firmware?] of several other accompanying components, I PM'd johnson a page full of that kind of stuff that I stumbled upon on google, don't know what was done on the x1 so let's wait for his answer

i don't think x1 performs how it was seen as it's used on the 3925, leaving that aside the next adsl version that im curious about is w1

from aajz.16 pdf
Quote
ZyXEL VMG1312-B10A & VMG1312-B10B

Bootbase version: V1.63 | 02/01/2016 15:12:03
Internal Firmware version: V1.00(AAJZ.16)C0
External Firmware version : V1.00(AAJZ.16)C0
Kernel version: 2.6.30
DSL mode code version: A2pv6F039v
DSL driver version: d26a
WLAN code version: 6.30.102.7.cpe4.12L06B.0
3G dongle WWAN package version: V1.16
APP agent version : 1.0.1

aajz.14
Quote
Bootbase version: V1.63 | 02/01/2016 15:12:03
Internal Firmware version: V1.00(AAJZ.14)C0
External Firmware version : V1.00(AAJZ.14)C0
Kernel version: 2.6.30
DSL mode code version: A2pv6F039v
DSL driver version: d26a
WLAN code version: 6.30.102.7.cpe4.12L06B.0
3G dongle WWAN package version: V1.16
APP agent version : 1.0.1
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on June 30, 2018, 04:44:02 PM
VMG8924/8324 firmware from latest version up:

https://forum.kitz.co.uk/index.php/topic,21273.msg377112.html#msg377112
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on June 30, 2018, 05:08:33 PM
Many thanks !  ;D  Much appreciated
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on July 06, 2018, 02:38:18 PM
A couple of feasibility questions:

1. there is some magic incantation that will make the modem reconnect at a reduced downstream target SNRM, an override. I wonder if there might be some way of storing a setting persistently, in non-volatile storage, so that whenever the modem connects it could use this override parameter to set the initial downstream SNRM according to that value?

2. I wonder if the same tweak could be applied to the upstream direction? Perhaps this is impossible for some reason because one end (relatively) always controls this. I am not aware of the existence of any CLI command for an upstream SNRM override.

An upstream tweak down from 6dB to 3dB would help me enormously as I do struggle with upstream throughput of only 1.1 Mbps. This has got worse since I changed to the ZyXEL modems too, by about 12%. On my iPad uploading backups to the iCloud, sending files via iMessage and uploading photos is a real pain and it is the bottleneck for Facetime conversations.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on July 30, 2018, 02:03:17 AM
1. there is some magic incantation that will make the modem reconnect at a reduced downstream target SNRM, an override. I wonder if there might be some way of storing a setting persistently, in non-volatile storage, so that whenever the modem connects it could use this override parameter to set the initial downstream SNRM according to that value?

Yet another thing that I think about and would love to be able to do, however from my pokings around I have yet to find anywhere one could easily insert the magic "xdslctl configure --snr" to be executed every time the modem tries to sync. I have thought that maybe decoding and modifying the stored configuration on the persistent data partition in the file "psi" as per these posts: https://forum.kitz.co.uk/index.php/topic,21936.msg378071.html#msg378071 could lead to something... but I guess the same could be achieved by uploading a modified xml config. Maybe there is some kind of bodge to be done by inserting the extra "--snr" switch at the end of another field thats read from the config and passed to xdslctl when bringing the connection up. But one would think there should be some validation logic ensuring sane values are read from the config that needs to be bypassed, but who knows.


Quote
2. I wonder if the same tweak could be applied to the upstream direction? Perhaps this is impossible for some reason because one end (relatively) always controls this. I am not aware of the existence of any CLI command for an upstream SNRM override.

An upstream tweak down from 6dB to 3dB would help me enormously as I do struggle with upstream throughput of only 1.1 Mbps. This has got worse since I changed to the ZyXEL modems too, by about 12%. On my iPad uploading backups to the iCloud, sending files via iMessage and uploading photos is a real pain and it is the bottleneck for Facetime conversations.

As the proud owner of a VDSL connection sporting 26mbps up and 1.17mbps down, I would love this to be possible and feel your pain... this kind of asymmetry means any use of the upstream has a direct impact on the downstream as a full speed set of TCP streams is close to saturating the uplink on its own. I often also have 7db of margin on the US, could easily be dropped a few db for an invaluable increase in throughput. Alas I have no recollection of being able to mess with US SNR even with ADSL1, and assume its totally out of the control of end user equipment.

Are you using "xdslctl configure --snr" on any of your lines currently Weaver? I recall reading in another post that your DS is running at 1.5db or so, is this by design or maybe due to the recent storms?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: banger on July 30, 2018, 03:06:19 AM
Weavers case is ADSL which the SNR tweak works but I dont know if weaver is using it. As of VDSL I think it is set on the DSLAM and cannot be overridden by end user.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on July 30, 2018, 04:12:09 AM
I have tried the tweak with impressive results.

I am wondering though if it is pushing things too far, so I ought to find out what the ratio of corrupt to total packets received is. Not sure how to do that. Could either look at modem stats, but not exactly sure, or look for L4 retx but then the packet loss or timeout that causes them could very well be for other reasons than corruption, and interpreting long periods of packet cap action with the tools that I have is a pain, hurts the brain.

With the DLink modems the downstream SNRM was usually that anyway, so perhaps they had a built-in naughty tweak factor. It would sometimes drop to 0.6-0.8dB downstream and stay there. With the old modems, each one would resync after a week or so. With the ZyXELs you never ever see resyncs.

I am thinking that the lightning might have pushed the noise floor up too much so that more FEC bits were needed. Or noise spikes could have overwhelmed the system, and it would have had to be set up with more overhead to have the required level of robustness to cope with sporadic errors of such a magnitude. I am assuming that it does not normally have to cope with spikes because there is no electrical activity of any kind near the line, no humans, no houses, no neighbours, apart from long-distance RF from: known broadcast audio RF transmitters, fish-finders, airport equipment at Tiree airport possibly, weather and marine safety info data broadcasting. If Aiseag airport by the beach at Aiseag Maol Ruibhe, between Broadford and the Skye Bridge gets going [if only!] then the level of noise might shoot up knocking out certain frequencies completely and might stuff my 5GHz WLAN channels ≥ 100.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on July 30, 2018, 07:27:32 AM
Regarding the thing about adding the tweak command at a suitable point. I assume that that command brings the link down first, then brings it up again with the new values. And bringing it down is not what we want, so unfortunately it means a nightmare of digging, trying to find where the system initially gets certain crucial values from when it synchs for the very first time and every subsequent time. I suppose digging to find out where the magic invocation command stores the given tweak value would be a start. That location would be read on every subsequent synch.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on August 18, 2018, 05:41:29 AM
So it turned out to be easy to get persistent storage of any tweak commands you like. Once I found out that commands issued with "xdslctl configure" stick between resyncs from the thread about line capping, its just a matter of having them run once on boot, no need to try and find a way to have them run every time the modem syncs.

So one line in /etc/profile to run a script (and detach it), and a file in the /data partition and away you go. Unfortunately the way the modem flashes itself there doesnt seem to be way to build files into the firmware that get extracted to the data partition, so the script thats run at boot needs to be written manually, but its only a few lines, for example:

Code: [Select]
sleep 30

xdslctl configure --maxDataRate 25000 1500 27000

Had to add the sleep as it seems /etc/profile gets run before everything is in place for xdslctl to work, don't worry its not blocking so shouldn't increase the boot time at all.

Have made a firmware for a 1312 & 8324 and I'l test it with my line in a bit, let me know if anyone is interested in having the functionality.

Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on August 18, 2018, 08:38:15 AM
Certainly am! Brilliant stuff.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on August 21, 2018, 01:18:31 AM
Got around to trying it on my line with my 8324, works great although I needed the sleep duration increased a little, guess the 8324 takes longer to boot.

Also refined it a little to avoid any confusion over file names, this is the only change to the firmware - added to /etc/profile:

Code: [Select]
if [ ! -f /data/boot-cmds.sh ];
then
 echo "Creating initial boot-cmds.sh"
 echo "#!/bin/bash" > /data/boot-cmds.sh
 echo "" >> /data/boot-cmds.sh
 echo "sleep 30" >> /data/boot-cmds.sh
 echo "" >> /data/boot-cmds.sh
 echo "# Place any commands here to run at boot" >> /data/boot-cmds.sh
 echo "# eg:" >> /data/boot-cmds.sh
 echo "" >> /data/boot-cmds.sh
 echo "#xdslctl configure --maxDataRate 25000 2000 27000" >> /data/boot-cmds.sh
 chmod +x /data/boot-cmds.sh
else
 echo "Running custom boot commands"
 /data/./boot-cmds.sh &
fi

So it tests if the file exists and creates a placeholder if not for you to edit. Editing does require using vi, which may be slightly uncomfortable if you have not used it before:

Login as supervisor or if just normal admin user then issue this to gain proper shell:
Code: [Select]
xtm && sh

Change to the data directory and edit using vi:
Code: [Select]
cd /data
vi boot-cmds.sh

Key "i" to enter insert mode and either uncomment the example line or write your own, eg:

Code: [Select]
xdslctl configure --snr 75
or
xdslctl configure --maxDataRate 25000 2000 27000

Escape key to exit insert mode and ":x" to save and quit. You can test to see if the command works as expected by:

Code: [Select]
./boot-cmds.sh

Wait 30 seconds then check for your changed settings with:

Code: [Select]
xdslctl info --cfg

For example my line now capped shows:
Code: [Select]
adslAnnexAParam:        00000980
adslTrainingMarginQ4:   -1
adslShowtimeMarginQ4:   -1
adslLOMTimeThldSec:     -1
adslDemodCapMask:       00000000
adslDemodCapValue:      00000000
adsl2Param:             00000000
adslPwmSyncClockFreq:   0
adslHsModeSwitchTime:   0
adslDemodCap2Mask:      00000000
adslDemodCap2Value:     00000000
vdslParam:              007f00ff
vdslParam1:             00000000
xdslAuxFeaturesMask:    00000000
xdslAuxFeaturesValue:   00000000
vdslCfgFlagsMask:       00000000
vdslCfgFlagsValue:      00000000
xdslCfg1Mask:   00000000
xdslCfg1Value:  00000000
xdslCfg2Mask:   00000000
xdslCfg2Value:  00000000
xdslCfg3Mask:   00000000
xdslCfg3Value:  00000000
xdslCfg4Mask:   00000000
xdslCfg4Value:  00000000
maxDsDataRateKbps:      20000
maxUsDataRateKbps:      2000
maxAggrDataRateKbps:    25000
xdslMiscCfgParam:       00000000
AFE_ID:                 10608700 00000000

Should do all this with the modem disconnected from the line to avoid causing several resyncs. Now reboot the modem and check again with "xdslctl info --cfg" to make sure its applied at boot.

I have tried putting faulty commands in the boot-cmds.sh file and no harm done to the boot, so unless you put something seriously bad like "reboot" in there I cant see any issues. If you wish to disable it just comment everything in the file out or just delete it, another placeholder will be made after a reboot with nothing but the sleep command in.

As always disclaimer that even though I have tested on several modems you are flashing at your own risk etc etc.

With just the jumboframes patches and this addition for the VMG1312-B10A:
https://www.dropbox.com/sh/vs35ss2pptlfm36/AABZkf219xF7M73dOTevx8ZBa?dl=0

If anyone is interested in this for an 8324/8924 just ask and I will make one (the one I made for my 8324 has the x1 adsl_phy etc).
 
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: banger on August 21, 2018, 01:46:36 AM
Brilliant.  :)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: stranky on August 23, 2018, 12:32:40 AM
wow yes please for the 8324!

great work though, thanks for your effort doing this :)



Quote

If anyone is interested in this for an 8324/8924 just ask and I will make one (the one I made for my 8324 has the x1 adsl_phy etc).
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: LordSven on August 23, 2018, 04:47:53 PM
Thank you so much for your hard work! And likewise for the 8924 please if differing in any way from the 8324. Do you see any benefit from the x1 adsl_phy for VDSL FTTC, approx 450m from Huawei cab? Solid 80/17.5 speeds on stock v18 firmware. No ipv6 though (as per normal with BT Infinity and the 8324/8924 I believe). Cheers!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: j0hn on August 23, 2018, 05:10:37 PM
Thank you so much for your hard work! And likewise for the 8924 please if differing in any way from the 8324. Do you see any benefit from the x1 adsl_phy for VDSL FTTC, approx 450m from Huawei cab? Solid 80/17.5 speeds on stock v18 firmware. No ipv6 though (as per normal with BT Infinity and the 8324/8924 I believe). Cheers!

This is for bridge mode. 1508 MTU can be done without this when using as a router.
I don't believe this firmware makes the changes persistent when used as a router and a command needs issued unless that has been resolved.

The newer adsl_phy seems to corect some Vectoring bugs but no other obvious performance improvements.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on August 23, 2018, 11:36:52 PM
wow yes please for the 8324!

great work though, thanks for your effort doing this :)

Sure thing, I'l make one for the 8924/8324.

Just for clarity, this latest feature for running commands on boot is only really useful if you wish to apply a cap to your line in a persistent way or as in Weavers case, for applying target SNR changes to an ADSL line.


Thank you so much for your hard work! And likewise for the 8924 please if differing in any way from the 8324. Do you see any benefit from the x1 adsl_phy for VDSL FTTC, approx 450m from Huawei cab? Solid 80/17.5 speeds on stock v18 firmware. No ipv6 though (as per normal with BT Infinity and the 8324/8924 I believe). Cheers!

Several people tried the x1 version, it did not appear to offer any increase in speed or stability on either VDSL or ADSL, its only benefit was fixing an odd bug that I have experience on a vectored line. Of course you are welcome to test on your line but from others experience its unlikely to improve things and adds another variable to any troubleshooting you may be doing.

This is for bridge mode. 1508 MTU can be done without this when using as a router.
I don't believe this firmware makes the changes persistent when used as a router and a command needs issued unless that has been resolved.

Indeed, this change only allows the execution of commands on boot, the 1508 commands that need to be run when using these devices in router mode have to be applied to the interfaces that are brought up once a ppp session has been made.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on August 24, 2018, 02:47:57 AM
It is I hope clear just how much we all appreciate your hard work.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: banger on August 24, 2018, 02:59:55 AM
+1
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on August 24, 2018, 03:10:45 AM
So it seemed like a good time for a consolidation post.

These are firmwares based of the latest release from Zyxel, AAJK.16 for the VMG1312-B10A & AAKL.24 for the VMG8924/8324-B10A.

Each folder has several firmware with naming as follows:


-jumboframes
Patches applied to allow 1508 byte packets to pass from a router to the modem in bridge mode. See this thread (https://forum.kitz.co.uk/index.php/topic,21273.0.html) for further info.

-oldtelnet
Including a slightly older version of the telnetd binary that allows multiple concurrent telnet sessions. This is useful if you want to be able to log into your modem while running monitoring software such as DSLStats. Basic comparisons of it with newer versions indicate that the only change made by zyxel was the disallowing of multiple sessions.

-customcmds
A small change to add the execution of a user modifiable script located in the persistent /data partition during boot. This is useful if you wish to add a cap to your line to possibly remove banding/reduce retransmission, in a persistent way that survives reboots. See this post (https://forum.kitz.co.uk/index.php/topic,22086.msg379683.html#msg379683) and thread for more info on reasons to cap. It is also useful if you have an ADSL line and wish to have persistent storage of changes to your target SNR. Please see this post (https://forum.kitz.co.uk/index.php/topic,21545.msg381240.html#msg381240) for how to add the commands you want.

-x1
These firmwares contain a newer version of part of the broadcom DSL driver - adsl_phy.bin. It has been tested by several users and does not seem to offer any speed or stability improvements, however it does seem to fix issues I have personally encountered on a line with vectoring. If you have vectoring (unlikely unless you live in a BDUK (https://www.gov.uk/guidance/broadband-delivery-uk) area) then these versions are worth trying.

-x6
As above but including the newer A2pv6F039x6 adsl_phy.

VMG1312-B10A
https://www.dropbox.com/sh/c5no9h8ckyp2i4i/AADwAR6oVbzwFjtE-bLdy81Za?dl=0&lst=

VMG8924/8324-B10A
https://www.dropbox.com/sh/4xfif7ssq8pdq6u/AAChWy_N8UIVmpqQizmI2CUra?dl=0&lst=


They have all been at a minimum tested to boot have the correct changes on my devices, but as always you are flashing at your own risk and having a backup modem is highly recommended.

Edit: Change dropbox view to list

[Moderator edited to add new information from the OP.]
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on August 24, 2018, 03:32:24 AM
It is I hope clear just how much we all appreciate your hard work.

Its just nice to be able to provide something that people find useful.

If there are any other combinations of the options from the above post required that are not in the folders, just post here and I can make, test and upload them.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on January 18, 2019, 03:42:12 AM
regarding the graphical stats web server build, a feature request: How about a couple of links, maybe inside a <ul><li> etc which just point to /data/whatever, the textual stats report pages, for convenience. It would be a finger saver.

Also musing about another ui convenience request, could there be any way of arranging for a preceding call to/getdata to happen automatically before these individual per-command requests? If asking for such a thing even makes sense.

--

I was thinking about the on-boot custom script execution feature. There's nowhere appropriate to store the custom boot commands in writeable flash. Could we pull in a boot script by fetching it from the internet somewhere somehow ? Pull its content from a download from some fixed url and then run it, or perhaps even better fetch a well-known dns TXT record and execute its contents ? But what would the 'standard' well-known external reference be ? Could such an external reference be made variable by defining it in a config xml entry somehow, perhaps re-purposing an entry.

Or go further and simply put the whole custom script content into the config xml somewhere?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: 7cfm on February 27, 2019, 09:15:02 PM
I have just got a VMG1312-B10A from Ebay and tried to upgrade from 1.00(AAJZ.17)C0 to any of the johnson firmwares, I also tried to revert to stock 1.00(AAJZ.16)C0 but the router won't play ball, any ideas?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: vultura on February 27, 2019, 09:28:42 PM
I have just got a VMG1312-B10A from Ebay and tried to upgrade from 1.00(AAJZ.17)C0 to any of the johnson firmwares, I also tried to revert to stock 1.00(AAJZ.16)C0 but the router won't play ball, any ideas?

Welcome to the forum.

Have you factory reset the unit prior to attempting firmware upgrade?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: 7cfm on February 27, 2019, 09:33:19 PM
Yep, no difference, I suspect this might have something to do with the router not letting me downgrade to an older firmware?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: vultura on February 27, 2019, 09:45:29 PM
Yep, no difference, I suspect this might have something to do with the router not letting me downgrade to an older firmware?

I think it is possible to downgrade, but it may well require the .rom file from the firmware archive file you downloaded to be applied first using the restore configuration option.  It is simply a standard configuration file.

Others may be able to advise better than I.



Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 03, 2019, 04:25:25 AM
There’s an emergency boot mode that you can force it into with the appropriate hole-poking procedure and magic incantation. I don’t know if that would be relevant with this.

Burakkucat or Johnson may be able to help with that.

There is a procedure for killing the .rom file and indeed as vultura said that might be the way to go. High priests can advise with .rom-killing.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 03, 2019, 04:27:17 AM
Welcome, 7cfm!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 15, 2019, 01:24:18 AM
I must have been talking utter rubbish in this earlier post of mine (https://forum.kitz.co.uk/index.php/topic,21545.msg390524.html#msg390524) when I thought there was no writeable flash area, no? I was thinking that /data/ was ramdisk? Is that wrong?

The stats code writes to /tmp and that obviously is writeable, and it survives a reboot? I haven’t tried pulling the power - ?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 15, 2019, 01:59:50 AM
The convenience links, urls I mentioned earlier, would be a nice tiny finger-saver. Could I even do it myself by modifying the html file, by code in the custom boot cmds thing? It would probably be too late and the web server would have loaded that html into ram and be running that copy permanently so it wouldn’t make sense, I presume. I would have added some
Code: [Select]
<a /> elements into the html with
Code: [Select]
sed perhaps.


More wish list:

2. I wonder if the graph could have a range limit. Sometimes, when things go wrong, huge SNRM values get introduced as giant spikes, and dominate the scale making it impossible to read the normal items as they are so tiny by comparison. I don’t know what to suggest about the details of sorting that out and I don’t know where to set range limit parameters, or just to hard-code some seemingly sensible range limits since we know what SNRM values make sense, from experience.

3. What about logging attenuation ? What do you think?


Note: I am finding the stats logging and graphing thing so incredibly useful. I’m using it all the time and it has already helped me with a recent intermittent high-resistance fault on my line #3 which is now sorted.

— The easy access to the raw ZyXEL current stats at http /data/varStats is fantastic. I’ve used it in a program in my iPad which queries the upstream sync speeds from all the modems, calculates downrated IP data rates derived from these numbers them and puts these rates into a dynamically modified Firebrick router config file which is uploaded into my router.  This ensures that the router drives each modem at the correct max speed, and the total upstream traffic gets split between the links in the right ratios according to the links’ differing speeds.

So thank you so very much indeed for your work!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on March 15, 2019, 03:43:04 AM
I must have been talking utter rubbish in this earlier post of mine (https://forum.kitz.co.uk/index.php/topic,21545.msg390524.html#msg390524) when I thought there was no writeable flash area, no? I was thinking that /data/ was ramdisk? Is that wrong?

The stats code writes to /tmp and that obviously is writeable, and it survives a reboot? I haven’t tried pulling the power - ?

Indeed /data is writeable and persistent, not sure of its size though. There is about 155K of certificates and things there, but I wouldnt want the the log file written there every 30 seconds. The html/js/logfile etc is all copied to /var/tmp (in ram) at boot, so a reboot will unfortunately lose the days log.

Quote
The convenience links, urls I mentioned earlier, would be a nice tiny finger-saver. Could I even do it myself by modifying the html file, by code in the custom boot cmds thing? It would probably be too late and the web server would have loaded that html into ram and be running that copy permanently so it wouldn’t make sense, I presume. I would have added some

Have a look at the current state of the interface I have running on my VMG1312:
https://imgur.com/a/DzvYY9E

Buttons for each graph, options to scale linear or log and for tone range data either full or scaled to the top tone with bits used.

Also a button that goes to the data folder where you can select each raw stats file. They all update on access now so no /getdata call required.

Playing with having a side bar with useful info next to the graphs, will look something like this:
https://imgur.com/a/Iz3UsGY

Does this sort of interface appeal?

Quote
2. I wonder if the graph could have a range limit. Sometimes, when things go wrong, huge SNRM values get introduced as giant spikes, and dominate the scale making it impossible to read the normal items as they are so tiny by comparison. I don’t know what to suggest about the details of sorting that out and I don’t know where to set range limit parameters, or just to hard-code some seemingly sensible range limits since we know what SNRM values make sense, from experience.

3. What about logging attenuation ? What do you think?

Are there other giant spikes than the ones caused by the bugged upstream SNRM (going to 100 for negative values)? The log/lin button helps with visualising graphs with spikes in, but certainly clipping would be good.

Can certainly add attenuation, I imagine the graphs will be pretty boring on most lines but maybe they will offer some insight on lines with problems like yours.


Quote
Note: I am finding the stats logging and graphing thing so incredibly useful. I’m using it all the time and it has already helped me with a recent intermittent high-resistance fault on my line #3 which is now sorted.

Really glad its useful!  :) Apologies for the glacial speed with which I'm updating it.

One more thing I have got working on the testing version is in browser zipping of files, so one click to get an archive of the log file and all stats, should be useful for capturing events easily.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 15, 2019, 07:09:23 AM
> Does this sort of interface appeal?

Wow, just a bit. That looks fantastic. You have been working hard. Very appealing.

I had a problem where the attenuation jumped up by 8-10dB and so I’m thinking that that could possibly be a common pattern where there is some fault condition? Of course as you say, deathly boring for the most part.

> Are there any other giant spikes

No, I am certain that’s the only source of them. I’m still running that version, can’t remember where we are now.

Regarding zipping, does the small webserver used for stats support gzip encoding for http?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on March 15, 2019, 08:00:28 AM
I had a problem where the attenuation jumped up by 8-10dB and so I’m thinking that that could possibly be a common pattern where there is some fault condition? Of course as you say, deathly boring for the most part.

> Are there any other giant spikes

No, I am certain that’s the only source of them. I’m still running that version, can’t remember where we are now.

Regarding zipping, does the small webserver used for stats support gzip encoding for http?

Ok will add attenuation. The 100+db spikes should be sorted by the fudge of subtracting upstream SNRM values from 100 when > 100, sure there are no cases when SNRM would be that high else where so no problems.

Good idea. I dont think the server supports dynamic compressing of files, but maybe with the right headers set it could serve pre-gziped ones - should save some ram! But the zipping I was talking about is in browser using a javascript library (96K - 28K gzipped  ;D) mostly just to take a snapshot of the various stats files and logs and present a single archive to download. Good for debugging as with the 100db spikes you had and with the possibility of being able to "upload" (not really because its all in browser) archived stats to view at a later date.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 15, 2019, 08:51:21 AM
Good stuff about the javascript library. This is fantastic stuff, much appreciated.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Alucidnation on March 24, 2019, 12:19:31 PM
Hi guys,

Is there a command to manually adjust the SNR on these?

Had a look at the support notes and couldn't find anything obvious found in there.

:)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Hitman on March 24, 2019, 01:31:35 PM
Hi guys,

Is there a command to manually adjust the SNR on these?

Had a look at the support notes and couldn't find anything obvious found in there.

:)

Only for adsl, dslstats has an advanced tweaks page for this or manual telnet - adsl configure --snr X (X=value of snr reduction - 50(3db),75(1.5db) etc...).

Does not work with VDSL- ignored.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Alucidnation on March 24, 2019, 01:38:00 PM
Only for adsl, dslstats has an advanced tweaks page for this or manual telnet - adsl configure --snr X (X=value of snr reduction - 50(3db),75(1.5db) etc...).

Does not work with VDSL- ignored.

 I’ll take that as a no as I’m in vdsl.

Is there any reason why it’s not available?

I don’t think it’s a Broadcom thing as there is a section on the billion 8800.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Hitman on March 24, 2019, 02:07:08 PM
I’ll take that as a no as I’m in vdsl.

Is there any reason why it’s not available?

I don’t think it’s a Broadcom thing as there is a section on the billion 8800.

The command get rejected/ignored by the DLM side for VDSL.

Router/modem's with Infineon chipset can adjust the VDSL snr eg.Draytek, so would assume it is Broadcom related.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on March 24, 2019, 02:09:06 PM
Without using questionable chipsets there is no way to manually change SNRM target on VDSL lines. Those that can do not adhere to standards and will probably give you worse sync speeds than you had without messing with it.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: j0hn on March 24, 2019, 03:19:47 PM
I don’t think it’s a Broadcom thing as there is a section on the billion 8800.

It's a Broadcom thing.
It only works on ADSL on the Billion also.

Lantiq chipsets can do what you want.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: ejs on March 24, 2019, 03:22:11 PM
I doubt you'd be saying the Broadcom chipsets were questionable if their command to adjust the SNRM worked on VDSL2. Lantiq chipsets are used in various Openreach MCT approved devices.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: ejs on March 24, 2019, 03:46:57 PM
If you wanted to increase your SNRM with a Broadcom modem, you could set a rate cap to reduce your speed, and then you'd end up with a higher SNRM.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on March 24, 2019, 03:58:42 PM
I doubt you'd be saying the Broadcom chipsets were questionable if their command to adjust the SNRM worked on VDSL2. Lantiq chipsets are used in various Openreach MCT approved devices.

Yeah you are right, I was conflating the mediatek chipsets that let you do all sorts of strange stuff with lantiq. Apologies!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: hacktrix2006 on March 27, 2019, 03:37:41 PM
Loving the interface? I take it this has everything that DSL stats has? I take it everything can be saved to USB?

Sent from my LLD-L31 using Tapatalk

Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 06, 2019, 09:00:15 AM
The easy stats interface is just so excellent. Am using it as an API all the time. Apart from the sheer usefulness of it, as an api I can now generate automatically customised sections of config for my Firebrick router which contain upstream speed limiter values which the router requires. These speed values are derived by querying the modems using the Johnson easy-stats api. The router needs these to control the fractions of the split of traffic going to each modem according to its own speed capability. Before I had to set these numbers by hand and query the required numbers by hand. Now everything is automated including configuring the router according to the possibly changed speeds as the program can upload changed config into the router. Unfortunately I don’t have a service process / daemon running continuously to oversee this and keep things up to date as I don’t have a suitable box to run it in. The program is running in an iPad and I can’t use such a machine to do that anyway. But aside from that, it’s now just one click to update the router’s configuration to match changed modem upstream sync rates.

Having the easy-stats interface avoids the need to log in, which is irrelevant anyway and would be annoying. It would also be a minor potential security threat. For an application program’s use, if it logs in to the web admin UI, you would have to store some account username + pw somewhere, required for it to get into the modem’s admin interface, and the more places where you store passwords the more possibilities for trouble.

I haven’t yet tried writing an iPad Shortcuts program to log in using the web UI, and I don’t know how much pain that would be.

What I would like to do is test whether or not I have a johnson-easy-stats-api-speaking modem present or not. I want to handle failure conditions properly which include: (i) no modem, (ii) wrong kind of modem (one which doesn’t speak easy-stats), (iii) no speed value available because the link is not up. And I hope that that’s the complete list.

I have a problem with the program hanging in some of the error cases. It maybe that I can’t fix some of the hangs if they are due to a lack of error handling in iOS Shortcuts library routines used to talk to the modem.

I am however wondering if my application program could be made more reliable and more intelligent if it explicitly detected lack of an easy-stats speaker rather than relying on error mechanisms reporting errors or getting a bad value back from http-based queries and checks on its value turn the situation into a ‘fail’ where the fact that the return value is garbage is because we do have some modem, but it is a non easy-stats speaker, could easily be the wrong modem altogether - one of my DLink backup/swapout modems, for example. This doesn’t remove the need to do error handling properly. But if I can’t get error handling the way I want it, then I can perhaps do a workaround for some common cases, and such a workaround might improve things for the most part.

If there is a non easy-stats-speaking modem present, it will be that the wrong model of modem is connected, in my particular situation. So what I’m thinking of doing is poking it with some harmless query first. I could do a getdata, and check that it returns "DONE".

So in future releases, could we keep the getdata / "DONE" protocol? Even if it doesn’t do anything any more, it is a nop that works as a very useful presence-test.

For me though, using the pre-poke-with-getdata strategy might just replace one error-handling problem with another in some cases and it might easily be a stupid idea. There are two questions, one is: can I just fix the error handling anyway? And the second is whether or not this will improve things in practice in a real test with the wrong model of modem present. I suspect I am likely to replace one kind of hang with another. I can hope that with a DLink modem present the http queries will fail straight away, with an error code, and if so I can pick up on that without the pre-poke strategy, and if not then I’m probably stuffed anyway. But I’d like to keep this nop in the api, just in case, if that’s ok?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on April 10, 2019, 11:34:18 PM
Promise I am still working on this! Have been side tracked by a few things though.

I have gziped html/css/js working with the tiny mongoose server, turned out to only be a few magic lines required, setting the media type for js.gz etc and setting the content type to gzip in the headers for responses to the directory containing them.

Also have an example version with SSL working, although it required compiling OpenSSL for MIPS and then statically linking it against the web server making the binary much heavier at ~ 1.9MB. Not sure how useful this is though, how many people actually use https (and click through the cert warnings) to access the default Zyxel GUI? Guess its a good idea for the future if we add a privileged section that can actually change settings on the modem.

The main hold up came after I started trying out the graphs on my (old) android phone, basic 24 hour or 4096 tone graphs were seriously sluggish to load. I'm not a daily chrome user, but their debugging tools are out of this world - a usb cable and "developer mode" selected on an android device and you can see detailed performance graphs for how a site loads natively on your phone on your desktop. This revealed that Chart.js is not the fastest library, so I'm looking for another. So far dygraphs is the winner, its slick and has lots of options to customise without being a nightmare to draw a basic graph and best of all its fast, a 2.9 second graph draw (on the ancient phone) down to 0.8. So need to get around to changing everything to use this.

Also, even though its probably a fools errand I have been trying to shoe horn enough of the VMG3925s build into the 1312s to get the later kernel (3.4.11 vs 2.6.30) and so broadcom dsl driver running. So far I have it booting the kernel and most of the modules loading, but nothing actually working.

@Weaver
Of course /getdata can be left in place if its useful. Was also thinking about something you mentioned in another thread about automating resyncs. This functionality could be exposed via a rudimentary API with basic authentication tokens or some such. Does 'shortcuts' let you manipulate the http headers?

@hacktrix2006
Glad you like it! Currently only has a small subset of DSLstats features. SNRM, FEC & CRC for 24 hours and current graphs for SNR, bitloading, QLN etc. USB storage for longer time periods is a definite target, but for now keeping everything as simple as possible, just viewing in the browser and saving from there is where I am at.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: hacktrix2006 on April 10, 2019, 11:57:23 PM
how is the custom kernel doing? I am thinking of setting up something like DSLStats on my WRT32x but not sure if A it will work or B the CPU (Dualcore) would handle it as well as the internet traffic at once. I am only using the VMG1312-B10A as a modem, will continue to do so once i get Fibre (First order just got rejected).
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 11, 2019, 12:43:57 AM
. . . how many people actually use https (and click through the cert warnings) to access the default Zyxel GUI?

I only have https, ssh and ICMP enabled LAN side on my VMG1312-B10A. I just had to create and store an exception in the browser.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 11, 2019, 04:17:54 AM
@johnson about the remote reboot thing. Yes, I have done ‘basic authentication’ before with the iOS Shortcuts framework. I already have written library routines to do everything, so it would be a good choice for me. Could we use the existing password out of the current ZyXEL config - would that be more work or less work?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 24, 2019, 05:00:40 AM
[A recap post. Duplicate of a post in another long thread.]

This post at an earlier point in the thread (https://forum.kitz.co.uk/index.php/topic,21685.msg384925.html#msg384925) contains the download link (https://forum.kitz.co.uk/index.php/topic,21685.msg384925.html#msg384925). That is the version I am using on all my (many) modems. Rock solid, ignore the reports in posts immediately following as I just made stupid mistakes and failed to install it. It is just superb.

That version has A new version is under development.

I use all of the features of it all the time as I don’t currently have a suitable machine that can display stats via a telnet-based protocol because of limitations in the tools I use. I am iPad-only these days and I have not learned Python yet, which is probably the way to go for easy casual app development on iPad.

Another opportunity to thank our kitizen and friend Johnson for his outstanding (and continuing) work on this.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 25, 2019, 02:17:13 AM
A few questions about the toolchain supplied by ZyXEL: the cross-compiler for MIPS target was mentioned.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: RTouris on May 28, 2019, 07:47:33 PM
I'd be inclined to test this on my 8924 in bridge mode, however seeing that I have the patched v.24 one with the babyjumbo frames, the multiple telnet sessions and the new DSL PHY driver can I ask if we can possibly have the newer version with the webserver stats incorporated? ...and while we are at it can I ask how any updates off the current 1312 fork are shaping up? ;)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on July 09, 2019, 06:14:36 AM
Experiments with docker have given pleasing results, with this minimal Dockerfile:

Code: [Select]
FROM yamamuteki/ubuntu-lucid-i386

WORKDIR /tmp

RUN sed -i 's/archive/old-releases/' /etc/apt/sources.list

RUN echo "dash dash/sh boolean false" | debconf-set-selections

RUN DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash

RUN apt-get update && apt-get install -y unzip bzip2 patch \
    g++ flex bison gawk make autoconf zlib1g-dev libcurses-ocaml-dev libncurses-dev

I have a fully working build environment for the 1312/8x24. So with a working docker install its 2 commands to build the images on linux/mac/windows. (  :fingers: )

Have also succeeded in getting a 3925 build running in docker, but its a little more complicated.

Anyway, have also got 99% of the the way there to a nice helper script that will use this docker container to build any version of the image required, jumbo - telnet - x1 - x6 - stats etc. Works great in linux, but need to try on other platforms.

Has anyone got the new source package we all requested in january(?), I believe the date they said it would be available was some time in june, but have not been checking my spam email box and they always seem to go there. Would be nice to include the latest source.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on July 09, 2019, 06:34:20 AM
A few questions about the toolchain supplied by ZyXEL: the cross-compiler for MIPS target was mentioned.
  • where does the compiler come from - is it included or a reference and a separate download ?
  • is the target MIPS32 ?
  • is the compiler’s code generation any good?

Sorry for not looking at this thread in an age Weaver!

1. The toolchain is supplied with the source code package from Zyxel, others have rollled their own and produced working binaries (that I have tested). The cross compile tools are mips uClibc 4.4.2-1.

2. Yes the target is MIPS32, the BCM63168 has 2 400mhz MIPS cores.

3. I have no idea, all I know is that it works. The build is Makefile soup, so I am not sure if the best compiler flags are used, but I should look at that.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on August 24, 2019, 04:59:18 AM
Since the previous post, I have been running this same VMG1312-B10A 2018-10 version (https://forum.kitz.co.uk/index.php/topic,21685.msg384925.html#msg384925) on all my modems. I have since tested a new very snazzy JavaScript UI for the SNRM stats graphing function by remote access to a UI-test-box server, but as for modems, I have not acquired a newer released modem build. That October VMG1312-B10A release has worked superbly well.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 28, 2020, 10:23:15 PM
I’m now running the 2020-02-xx (mid February) release of the VMG 1312-B10A build. As I’ve said earlier elsewhere, it’s superb. Many thanks yet again.  ;D  ;D

FYI I have my own "my-ref" version numbers for new Johnson releases; this number space of mine doesn’t tell you which build variant it is in the sense of which features are in/out amongst the many build options. The weaver-ver numbers are


A couple of feature requests arising from something that was discussed briefly some while back:

1. Would it be possible to take your Johnson build version number and append it to your build version identifier string as I see that the build version id string / bin filename is now visible in the web UI?

This is info something I would like to use in order to :
2. Could you add entries to the main stats numbers page so that all the component build version strings and Johnson build version number are showing? That would be really handy as I then don’t need to log in. Note: I’d like the Johnson build version number to be shown separately please.  :)

3. And the same version info components as above, making it structured in machine-readable format such as eg "<field-name> ":" <field-value> <LF>" or whatever, with easy-to-parse syntax, and returned by an http request such as "http://192.168.1.1:8000/Version/". The field names can be whatever you like, something suitable to explain the various component items of version identification info exactly as in #2 above.

That way I could query the build version number most easily.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 29, 2020, 08:21:59 AM
Very vague possibility of a bug report. After I upgraded to a new 2020-02-xx version of Johnson’s custom firmware now with a vastly enhanced multi-graph web UI as opposed to the 2018-10-xx single graph version I had been using before I got very confused because it looked like the upgrade had failed. I couldn’t see the new sophisticated web UI, I was still seeing an old-style web page but digging around in the ZyXEL admin web-UI’s menus I could see the new expected version string. I tried all sorts of things floundering. Then I discovered that it was simply a matter of stale cached content in the browser. Somehow getting the browser to really refetch the latest page at http://192.168.3.1:8000 was all that was needed to reveal the new style stats server’s numbers page finally.

Apologies if all the following is old news to you, which I’m sure is probably the case, but just in case it’s useful, here goes:

So it looks to me as if the cache control info is wrong. See https://www.mnot.net/cache_docs/#SCRIPT In fact the whole article if you’re not au fait with web cacheability and caching-friendliness. It looks as if there is caching of stale data going on within the browser here, and I’m guessing it’s because because cacheability headers are missing.

It would be a really bad thing to try and disable all caching by fighting the browser, I recommend the best thing to do is to add Etag http headers to the generated web pages. (See above url.) A good web server will do this for you when it can and I’m assuming that Mongoose (is that right) will do so but only when it can, and here you’re taking generated content produced by an application and turning it into web pages, not serving up files from the file system and so in this case webserver engines need to be told what to do as they can’t generate cacheability headers automatically from metadata about a served-up content file’s size, last-modified date and even a hash of the file content. I have written this kind of code in a server-side module before. Again, the best thing would be to add ETag headers whose values are cheapo cheapo hashes (ie fast and not cryptography-grade) of the entire generated data itself which is the content of your webpage. That way the browser will know when the content of the web page has changed because the etag hash will change and if the value is not changed then the browser can skip a repeat download. The system is then bomb-proof and there’s no fiddling about with a guess about freshness times.

In this case the problem was I assume the reverse, if my guess/diagnosis is right, in that old information which was stale was regarded as good by the browser either because there was no cacheability header information published by the web server or else the cacheability headers were wrong, which is unlikely unless there was a definite bug. When the content of the page changes then the ETag value computed from the page content, which is the cheap and fast hash, becomes different from the ETag value last seen by the browser and so the browser knows to do a download. Since the browser, so it seems, was unwilling to download the page in question (in this case ":8000/") even though the relevant page content had in fact changed inside the server my guess is that there were no ETags. Maybe there is an overall default in force for a long cache time which is totally inappropriate here.

I think back then I was still seeing the browser’s cache which contained a webpage produced by the old firmware and the browser was ignoring the fact that the content of "/" was by then different on the server. The server presumably got a get http head request or got a conditional get which returned not_modified. There are various if-modified queries and conditional requests, which check dates if applicable, or else ETag values, if not using dates in cacheability headers, as here, with non-(static)-file content being served up.

Hopefully that’s all you have to do. One albeit long line of code. I had to do a lot more than just adding calculated ETag headers in the server application code but my circumstances were different.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 29, 2020, 05:06:50 PM
Another bug report: on one of my modems, the axes in SNRM are showing extreme values such that the current dataset will not be visible. I can scroll or pinch to head to the area where the data is but it’s either too far or it simply is not to be found. A reboot did not fix this.

Do we need a ranges state reset control in order to be able to get back to sanity if the user has scrolled right of the visible dataset? I may have dropped the iPad face down so it got bizarre multitouch events. I would suggest not, just don’t allow scrolling/zooming right off/away from the range of visible data.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Dad-Of-5 on March 31, 2020, 08:33:36 AM
sorry to change the subject but l'm on a Huawei cabinet that support g.inp and vectoring.
1. what firmware should I be on please for mtu 1508? looking for the one that would give me stats read out and do I just login and upgrade as normal via the router gui?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 31, 2020, 11:41:41 AM
Depends which modem you have. Do you have a separate modem and router? The ones with easy stats web server capabilities have ‘stats’ in the filename. And yes, you just upgrade firmware using the normal router web UI as usual.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on March 31, 2020, 09:35:38 PM
Further to that previous bug report: the axes that are scrolled off to extreme values are those that show date-time. The horizontal axis on SNRM is showing 1 Jan 1970. A reboot doesn’t get me out of this.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on April 01, 2020, 07:04:35 AM
Weaver thank you for the excellent and detailed feedback, the changes to version numbers and visibility all seem reasonable, I will have a good look at the best way to do it. Thank you for bringing up the problems with caching, I have experienced similar even with real projects like openwrt, but it seems to have slipped through the cracks with my tests of the firmware.

The time problem I thought was under control by having the stats logging script detect changes of over and hour and updating all the stamps in the log file, while also using the modems current time and calculating the difference to browser time to display. I had issues with the stats logging beginning before boot had completed so including stamps at 0 seconds since the epoch (1970) but I thought I'd stopped that happening.


To start with I really want to fix the graphs broken by incorrect timestamps, it would be great if you could upload the result of hitting "Save" on the stats page of a modem with the problem present (unfortunately I believe its over the limit for attachments with 48hrs of data so will require dropbox or similar).
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 01, 2020, 04:28:53 PM
Here’s a saved dataset with the problem showing. The server has not been running for any length of time at all; hope that doesn’t matter.

BTW this is downloaded into iOS Safari on my iPad which is set to a UTC/GMT not BST. I can’t imagine that that makes any difference, but there is one ‘just in case’ odd thing about my system. I have installed the new version in more than one modem and iirc some modems don’t show the problem and only one does. Guessing completely; one iPad fell onto its front on the bed so was getting pinch/scroll multitouch input events all the time while I dozed off. That could well have sent lots of events to the web server because a graph was showing in Safari at the time.

[Later edited version: should now have attachment correctly uploaded, was ‘zero bytes’ long before; see zip file from webserver (https://forum.kitz.co.uk/index.php/topic,21545.msg413486.html#msg413486) ]

[Moderator edited to detach the duplicate attachment.]
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 01, 2020, 08:49:12 PM
The attached ZIP archive is of size zero bytes.   :(
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 01, 2020, 11:33:45 PM
Damn that keeps happening to me when I attach something. I’ll try again and actually check the size.

See attached, attempt II
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 01, 2020, 11:34:29 PM
That looks better, yes? How does one manage to get zero bytes?

Maybe the attachment gets deleted in my system (somehow, god knows how that might be?) before I hit [post], and it is only then that the file content gets uploaded to the kitz web server? I was assuming the file content gets uploaded early, uploaded when I choose the attachment.



I had to go back to the older version (autumn 2018) because otherwise I couldn’t see the SNRM vs time graph. I call the new version, uploaded mid feb 2020-20-16, weaver-v06.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 01, 2020, 11:57:18 PM
That looks better, yes? How does one manage to get zero bytes?

Yes, much better. To the second query: I'm not sure.

Quote
Maybe the attachment gets deleted in my system (somehow, god knows how that might be?) before I hit [post], and it is only then that the file content gets uploaded to the kitz web server? I was assuming the file content gets uploaded early, uploaded when I choose the attachment.

Ah, that may be the reason. If one specifies a file to attach to a reply, it is not committed until the reply is posted. So if the file is moved, zapped or lost in a passing black-hole before the reply is posted then the attachment will be of size zero bytes. The name is known but the content is null.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 02, 2020, 12:10:10 AM
Agreed. I really don’t know how the file might get zapped at my end before it has a chance to be uploaded. That’s a question about how Safari etc works in iOS. Either that or a question about what on earth Weaver is doing. This zero bytes thing has happened half a dozen times with me. Of course it’s my fault because I should check the size by reviewing the post after it has been posted.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 02, 2020, 12:16:40 AM
Of course it’s my fault because I should check the size by reviewing the post after it has been posted.

No worries. One of us will eventually notice. But now you have two separate attachments of the same file . . . Perhaps I should delete one of them?

<Later> The attachment to the earlier post has now been removed.

And having that data, I could not resist the urge to generate the usual plots.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 02, 2020, 12:22:40 AM
Good point. Perhaps delete my later post? I suddenly realised that since I could go back and edit the original post, I could perhaps change the attachment, so I did so, went back and attached it again and checked it afterwards.



I rebooted the server on the modem but I was still stuck in 1970 on the SNRM vs time graph and a reboot didn’t fix the problem. I don’t know where the state info is stored otherwise I could delete the state info and restart the server.

So it seems once the state is bad then you’re stuck with it, given my level of ignorance; I can’t see that even reflashing it will reset the state to restore sanity?

Bug aside, it’s a superb piece of software. Greatly impressed- thank you for all your work.



Off topic: half iOS question ; what’s the easiest way of me uploading one of my graphs as spring did there? I can of course do lots of screenshots and crop them, as I usually do. I would have to give all the bitmaps to my wife to make use of her expertise if I wanted a collage of them, tiled, as spring did back there.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 02, 2020, 12:31:44 AM
Good point. Perhaps delete my later post? I suddenly realised that since I could go back and edit the original post, I could perhaps change the attachment, so I did so, went back and attached it again and checked it afterwards.

I've done the simplest and detached the attachment to the initial post, on your behalf.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 02, 2020, 01:28:42 AM
@spring - what’s the cause of the tone 154 [?] (150 < x < 160 somewhere anyway) QLN spike ? And the other two minor spikes?

I have identified candidate radio stations and have come up with dubious shaky candidate ideas for certain other types of activity for other spikes in my graphs.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 02, 2020, 09:11:20 PM
QLN idk, but for SNR it's both interference and modem code tendency to do that. idk what else to say because i don't know how to solve such a thing. what version do you run? x6/V ?

lately i had interference too. no idea why but it's almost gone now.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 02, 2020, 10:04:07 PM
I have several modems so I’m running both versions at the moment. X6 seems to be good.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 02, 2020, 11:37:18 PM
yes x6 must be good. the exception would be if V turned to somehow be better which idk yet. or if x6 has some exceptions like x1 had. x1 ran better on some days but had days that were sometimes two or three-fold Forward Error corrections. i associate that with timeouts and stuff like that. it implies less stability in general.

only testing can know and it's still a personal result. i didn't really have a problem with x1 it's just that it performed slightly worse overall. [for non-vectoring lines like mine, that is]. i don't like x1 at all...

my dad also uses the internet so ill ask him and also see how games perform. i don't have anything new to add as i already know more or less how the SNR graphs will look and what error counts i would get. it's just that i didn't test to see what error counts V has during this time of external interferences. if only they were completely gone.

edit: x1 also had better bitswap, but sometimes sync'd with some bad bitswap and got stuck like that. that's why i need to re-sync x6 after 2 weeks and run it for another week. i think bitswap doesn't have a deeper meaning like SNR graphs. it's just what it is, unlike SNR graphs that can actually indicate whether or not there's something wrong with the modem code.

edit2: truth is, only long term experience of usage really counts in the end. not the error counts, SNR or bitswap graphs since those are ultimately meaningless. any modem code will always have ups, downs, and variations between different people.

edit3: i won't use dslstats again because it force restarts my 1312-b10a after a while, but johnson's stat graphs look a bit different than dslstats' graphs. that's why i can't just say "V's graphs looked better" even though i have old graphs.

edit4: any idea why dslstats did that to my device? the crash appeared to be random. did it ever crash your device? i prefer how dslstats graphs looked. more detailed and accurate than stats-server.

edit5: i'll use DSLStats before i restart the device since it has better graphs. that way i won't risk a crash. i also speed tested x6 and i can't imagine it working better. well i'm talking/thinking too much too soon

edit6: ISP is capped at 60mbit downstream. [i'll eventually call to update the package to 100mbit for free, maybe even next week. my line has 95mbit almost all the time. generic microfilters, no ferrites, generic 2(?) meter DSL cable. i used to have a better line with 110mbit but it was stolen... ISP gives 3mbit upstream for everyone]. if i connected the modem to the master socket i'd have no errors and 100mbit. but how can i pass cat6 through the walls of the house? stuck with using powerline adapters. i know, they emit unhealthy electromagnetic radiation.
(https://i.imgur.com/YtKnnA2.png)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 03, 2020, 11:13:09 AM
external interferences are back, line lost sync at 4:00 AM and SNRM graph showed a dip in upstream.
guess i won't be able to run it for 1-2 weeks so i'll just switch back to V to compare.

Code: [Select]
x6
----------------
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 4
Last initialization procedure status: 0
Max: Upstream rate = 26533 Kbps, Downstream rate = 81243 Kbps
Bearer: 0, Upstream rate = 5024 Kbps, Downstream rate = 75987 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Link Power State: L0
Mode: VDSL2 Annex B
VDSL2 Profile: Profile 17a
TPS-TC: PTM Mode(0x0)
Trellis: U:ON /D:ON
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 9.1 16.6
Attn(dB): 12.1 0.0
Pwr(dBm): -10.7 -10.8

VDSL2 framing
Bearer 0
MSGc: -6 363
B: 243 29
M: 1 1
T: 0 1
R: 6 16
S: 0.0000 0.1840
L: 19560 2000
D: 1 59
I: 250 46
N: 250 46
Q: 8 0
V: 0 0
RxQueue: 57 0
TxQueue: 19 0
G.INP Framing: 18 0
G.INP lookback: 19 0
RRC bits: 0 24
Bearer 1
MSGc: 186 -6
B: 0 0
M: 2 0
T: 2 0
R: 16 0
S: 5.3333 0.0000
L: 48 0
D: 3 0
I: 32 0
N: 32 0
Q: 0 0
V: 0 0
RxQueue: 0 0
TxQueue: 0 0
G.INP Framing: 0 0
G.INP lookback: 0 0
RRC bits: 0 0

Counters
Bearer 0
OHF: 0 973369
OHFErr: 7 0
RS: 1297315408 2308324
RSCorr: 966 1
RSUnCorr: 0 0
Bearer 1
OHF: 2072715 0
OHFErr: 14 0
RS: 24871844 0
RSCorr: 43 0
RSUnCorr: 86 0

Retransmit Counters
rtx_tx: 212898250 0
rtx_c: 3444 0
rtx_uc: 279385 0

G.INP Counters
LEFTRS: 53 0
minEFTR: 75982 0
errFreeBits: 1268046475 0

Bearer 0
HEC: 0 0
OCD: 0 0
LCD: 0 0
Total Cells: 570088300 0
Data Cells: 21029941 0
Drop Cells: 0
Bit Errors: 0 0

Bearer 1
HEC: 0 0
OCD: 0 0
LCD: 0 0
Total Cells: 0 0
Data Cells: 0 0
Drop Cells: 0
Bit Errors: 0 0

ES: 61 0
SES: 52 0
UAS: 387 101
AS: 33299

Bearer 0
INP: 45.00 1.50
INPRein: 0.00 0.00
delay: 0 3
PER: 0.00 17.04
OR: 0.01 173.23
AgR: 76065.11 5197.09

Bearer 1
INP: 4.00 0.00
INPRein: 4.00 0.00
delay: 3 0
PER: 16.06 0.01
OR: 95.62 0.01
AgR: 95.62 0.01

Bitswap: 11/11 0/0

Total time = 6 days 25 min 3 sec
FEC: 342981 466
CRC: 2935 0
ES: 61 0
SES: 52 0
UAS: 387 101
LOS: 0 0
LOF: 0 0
LOM: 42 0
Latest 15 minutes time = 10 min 3 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 15 minutes time = 15 min 0 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Latest 1 day time = 25 min 3 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 1 day time = 24 hours 0 sec
FEC: 95093 28
CRC: 911 0
ES: 20 0
SES: 16 0
UAS: 48 34
LOS: 0 0
LOF: 0 0
LOM: 13 0
Since Link time = 9 hours 14 min 57 sec
FEC: 966 1
CRC: 7 0
ES: 3 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0

edit: DSLStats data.
Code: [Select]
Stats recorded 03 Apr 2020 13:44:11

DSLAM type / SW version: IFTN:0xb206 (178.6) / v0xb206
Modem/router firmware:  AnnexA version - A2pv6F039x6.d26a
DSL mode:                VDSL2 Profile 17a
Status:                  Showtime
Uptime:                  9 hours 46 min 7 sec
Resyncs:                0 (since 03 Apr 2020 13:38:37)

Downstream Upstream
Line attenuation (dB):  12.1 0.0
Signal attenuation (dB): Not available on VDSL2
Connection speed (kbps): 75987 5024
SNR margin (dB):        9.1 16.6
Power (dBm):            -10.7 -10.8
Interleave depth:        1 59
INP:                    45.00 1.50
G.INP:                  Enabled Not enabled
Vectoring status:        5 (VECT_UNCONFIGURED)

RSCorr/RS (%):          0.0001 0.0000
RSUnCorr/RS (%):        0.0000 0.0000
ES/hour:                0 0


adsl info --pbParams
adsl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:    4
Last initialization procedure status:   0
Max:    Upstream rate = 26533 Kbps, Downstream rate = 81197 Kbps
Bearer: 0, Upstream rate = 5024 Kbps, Downstream rate = 75987 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Discovery Phase (Initial) Band Plan
US: (6,31) (882,1193) (1984,2770)
DS: (33,857) (1218,1959) (2795,4083)
Medley Phase (Final) Band Plan
US: (6,31) (882,1193) (1984,2770)
DS: (41,857) (1218,1959) (2795,4083)
                  VDSL Port Details               Upstream                Downstream
Attainable Net Data Rate:           26533 kbps              81197 kbps
Actual Aggregate Tx Power:         -  10.8 dBm            -  10.7 dBm
====================================================================================
  VDSL Band Status U0 U1 U2 U3 U4 D1 D2 D3
  Line Attenuation(dB): 0.9 11.2 15.6 N/A N/A 7.9 13.8 21.0
Signal Attenuation(dB): 0.8 11.1 15.4 N/A N/A 7.8 13.7 20.9
        SNR Margin(dB): 20.1 16.2 16.7 N/A N/A 8.8 9.3 9.1
         TX Power(dBm): -15.6 -13.2 -21.5 N/A N/A 12.4 7.8 7.0
 >
https://imgur.com/a/W8XybLB


edit2: first V stats:
Code: [Select]
V
-------------------
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 0
Last initialization procedure status: 0
Max: Upstream rate = 26589 Kbps, Downstream rate = 81147 Kbps
Bearer: 0, Upstream rate = 5024 Kbps, Downstream rate = 76519 Kbps
Bearer: 1, Upstream rate = 0 Kbps, Downstream rate = 0 Kbps
Link Power State: L0
Mode: VDSL2 Annex B
VDSL2 Profile: Profile 17a
TPS-TC: PTM Mode(0x0)
Trellis: U:ON /D:ON
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 8.9 16.4
Attn(dB): 11.8 0.0
Pwr(dBm): -10.9 -11.1

VDSL2 framing
Bearer 0
MSGc: -6 363
B: 243 29
M: 1 1
T: 0 1
R: 6 16
S: 0.0000 0.1840
L: 19697 2000
D: 1 59
I: 250 46
N: 250 46
Q: 8 0
V: 0 0
RxQueue: 57 0
TxQueue: 19 0
G.INP Framing: 18 0
G.INP lookback: 19 0
RRC bits: 0 24
Bearer 1
MSGc: 186 -6
B: 0 0
M: 2 0
T: 2 0
R: 16 0
S: 5.3333 0.0000
L: 48 0
D: 3 0
I: 32 0
N: 32 0
Q: 0 0
V: 0 0
RxQueue: 0 0
TxQueue: 0 0
G.INP Framing: 0 0
G.INP lookback: 0 0
RRC bits: 0 0

Counters
Bearer 0
OHF: 0 43569
OHFErr: 0 0
RS: 28959648 3191001
RSCorr: 0 0
RSUnCorr: 0 0
Bearer 1
OHF: 46007 0
OHFErr: 0 0
RS: 551346 0
RSCorr: 0 0
RSUnCorr: 0 0

Retransmit Counters
rtx_tx: 212919842 0
rtx_c: 0 0
rtx_uc: 0 0

G.INP Counters
LEFTRS: 0 0
minEFTR: 76496 0
errFreeBits: 862410 0

Bearer 0
HEC: 0 0
OCD: 0 0
LCD: 0 0
Total Cells: 108745406 0
Data Cells: 79367 0
Drop Cells: 0
Bit Errors: 0 0

Bearer 1
HEC: 0 0
OCD: 0 0
LCD: 0 0
Total Cells: 0 0
Data Cells: 0 0
Drop Cells: 0
Bit Errors: 0 0

ES: 0 0
SES: 0 0
UAS: 27 27
AS: 739

Bearer 0
INP: 45.00 1.50
INPRein: 0.00 0.00
delay: 0 3
PER: 0.00 17.04
OR: 0.01 173.23
AgR: 76597.87 5197.09

Bearer 1
INP: 4.00 0.00
INPRein: 4.00 0.00
delay: 3 0
PER: 16.06 0.01
OR: 95.62 0.01
AgR: 95.62 0.01

Bitswap: 10/10 0/0

Total time = 12 min 46 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 27 27
LOS: 0 0
LOF: 0 0
LOM: 0 0
Latest 15 minutes time = 12 min 46 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 27 27
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 15 minutes time = 0 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Latest 1 day time = 12 min 46 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 27 27
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 1 day time = 0 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Since Link time = 12 min 19 sec
FEC: 0 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
https://imgur.com/a/clusHVp


edit3: first spikes in SNR for V: https://imgur.com/a/24F3NB1
doesn't look good at all. x6 must be better.. i think. maybe V stabilizes to be on par. we'll see.

edit4: no more spikes but the last tones still logically point downward. not linearly though. i think at this point it's safe to say only usage like browsing, gaming etc has any value. total SNR is ~9db for both. error stats still matter though.

edit5: error rates looking pretty good.
Code: [Select]
V
--------------------
Latest 1 day time = 5 hours 45 min 49 sec
FEC: 216 0
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 27 27
LOS: 0 0
LOF: 0 0
LOM: 0 0
i'll post about the final results when i have them. i think enough information was posted by now.

edit6: it seems i do have DLM but it never reacts to any resyncs. only error counts and apparently the last 10 days had enough errors to limit my sync to 80. guess i do have some really smart DLM.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: GigabitEthernet on April 04, 2020, 02:37:20 PM
So I'm not sure exactly why this is, or whether it is my imagination - but when using a VMG1312-B10A with the custom X6 firmware (I have vectoring on my line), the Internet feels less responsive. I've set the MTU at 1500 on my OpenWrt TP-LINK Archer C7 but still things just feel more sluggish.

I've plugged my Huawei HG612 back in and things are feeling normal again, what could be going on here?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 04, 2020, 04:34:39 PM
X6 can glitch out sometimes. like X1 had the same tendency. that's why i decided to test it. i don't have DLM unless errors are so high they repeatedly [10+ times] drop the sync. i can test whatever i want with no consequences.

V is more stable. no doubt about it. i'm even seeing it in error counts. like i said:
Quote
i think resilient is the word to describe x6. i've been using the internet, gaming, and my dad as well. no complaints from him or me. downloads are stable. the best way to describe x6 is smoothness. for V it's more like "old fashioned cautions" that get through the hardest times. that's why x6 might not work for everyone. especially since (theoretically) it's a bit incompatible with dsl driver d26a. in practice i haven't seen that "incompatibility" manifest yet. [except maybe in SNR graph spikes... dunno yet]. it also bitswaps much better than x1 and even V. x6 is less conservatory than V. if that makes sense. so it's both more prone to errors but also better results. whether that potential is worth it i dunno yet.
it's not for everyone. X1 had the same problem. not for everyone.

maybe not even for me. i don't have the results yet.. all i know is i don't have vectoring so... might be useless for me to use X6.


edit: i misread you saying that you have vectoring as "non-vectoring". sorry. it then must be the incompatibility with dsl driver d26a.

i guess that one guy who said: "don't swap new modem code with old dsl driver" was right. actually, he said something similar to that.. not exactly that. but still.


i guess we do have results? don't try this at home.

lmao..


Quote
the Internet feels less responsive.
i've experienced that too. i thought it was my ISP... hah. guess it's not actually x6, it's the incompatibility with d26a.
maybe zyxel do other stuff before they just "change adsl-phy.bin". i don't know. i just know that,.. yes, the testing must be futile because it must be so simple.. so simple that the original modem code must be left in place for stability.




edit: if you compare X6:
(https://i.imgur.com/YtKnnA2.png)
to V
(https://i.imgur.com/I7oDCU6.png)

i think V does better in upload for example.
maybe that initial stutter with X6? idk. could be the ISP as i haven't tested it without the ISP. [i can do that though]
i'll switch to x6 in 2 weeks and see again if i have stutter etc because then i'll be sure it's not the ISP/line etc.

edit: don't mind the V having just 3.0 upload because my line has 5.0 upload. it's just the ISP ending up with 3.0. the graph is the only difference between the two.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: j0hn on April 04, 2020, 05:36:21 PM
You state that as a matter of fact.

The reality is every line is different and while your line will work best with 1 DSL driver that isn't necessarily the case for everyone else's lines.

I found no difference between any of the different DSL drivers but choose and recommend the original that came with the modem firmware.
It has the correct matching kernel.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 04, 2020, 05:42:29 PM
i never said it was the actual modem code.. i said it was incompatibility
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: GigabitEthernet on April 04, 2020, 06:24:05 PM
You state that as a matter of fact.

The reality is every line is different and while your line will work best with 1 DSL driver that isn't necessarily the case for everyone else's lines.

I found no difference between any of the different DSL drivers but choose and recommend the original that came with the modem firmware.
It has the correct matching kernel.

The only reason I changed firmware was because the original had issues with vectoring. Has this now been fixed, if so I will use the original.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: spring on April 04, 2020, 06:44:35 PM
choices choices... maybe just get a 3925 or whatever that was. i think 3925.
x6 was taken from 3925. or something similar to 3925. i don't know zyxel modems except 1312.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: GigabitEthernet on April 05, 2020, 03:59:36 PM
What is the latest official firmware for the VMG1312-B10A?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on April 05, 2020, 05:02:12 PM
What is the latest official firmware for the VMG1312-B10A?

Looking on the ZyXEL ftp server (http://ftp.zyxel.com/VMG1312-B10A/firmware/) I see VMG1312-B10A_1.00(AAJZ.17)C0.zip

[Edited to fix the link. (Thanks to tickmike (https://forum.kitz.co.uk/index.php?action=profile;u=329) for spotting the error.)]
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 05, 2020, 09:26:19 PM
Does anyone know how to parse and analyse the long ZyXEL version strings that we see in their filenames?

And I could do with a recap on the meaning of x1 vs x6 vs etc
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 10, 2020, 01:21:31 AM
Regarding the bug where graphs that involve a time x-axis are broken/absent/stuck on 1970; this only happens on my modem #3, not the others. I’ve tried everything I can think of to fix it. I’ve reinstalled the firmware, deleted the directory tree where the log file and all the application data lives - if I got it right, anyway I think I deleted at least enough stuff as I got a ‘not found’ error message from the app, then reinstalled the software but cannot fix it.

I don’t understand what’s different about that modem. I thought it must be something in the ZyXEL XML config file that is somehow different, so I diffed that xml with the one for modems #4 and there’s no difference so that’s out. (No difference apart from IP addresses being 192.168.n.* as necessary for the different modems’ slots, .3.* instead of .4.* etc)

Suggestions as to what’s going wrong and suggestions for a workaround?

I can always go back to the earlier software version for this modem, then I will have SNRM graphing back.

That suddenly makes me think. Why does the old software (autumn 2018) work on modem #3?

Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 18, 2020, 04:41:52 AM
Feature request or howto/tip request - no idea how to define this

Seeing as I have four modems, at times I keep having to edit the url at the top address line in my web browser, which is Safari iPadOS btw, in order to flick between the modems when I want to compare results. This is really fiddly and annoying.

Can anyone suggest an easy way of doing this? Or else design a way of adding an assistive feature into the UI so that this can be made into a feature request for a new software release? I have no idea how to do this and it’s cheeky asking a lot of our friend Johnson and the rest of you?

The only think I can think of would be a list of urls which you include in a config file somewhere, somehow. The list of urls for me would be something like
    http://m1.csw.me.uk:8000/*
    http://m2.csw.me.uk:8000/*
    http://m3.csw.me.uk:8000/*
    http://m4.csw.me.uk:8000/*

The current page url might be something like
    http://m1.csw.me.uk:8000/#Stats     for modem 1, say,
and the * denotes a part of the current url that is taken from the corresponding part of the current page; ie it’s common to an old page and a new page when changing from one modem to another. That part is initially null; if not then we would need to enhance this syntax.

The list’s entry count tells us how many modems there are.

The domain name I quoted is just a convenient alias for 192.168.x.1 for each modem and a literal IP address could be used instead.

The UI might be something like a load of links in a horizontal menu on the right of the menu strip at the top, the links being named m1 m2 m3 m4 etc, next to where it currently says "full"/"used" and "lin"/"log". An alternative UI might be a floating or drop-down vertical menu.

Possible syntax simplification, no * : It might be that you can just forget the * markers and assume that the current url is always longer than the current list entry and you compare the final part of the two, so for example the current list entry might be
    current list entry = http://m1.csw.me.uk:8000/            and the
             current url = http://m1.csw.me.uk:8000/#Stats
So the difference calculated, final = "#Stats" gives you the extra amount of text that needs to be added to transform a url for one page into that for another modem. You obtain this by reverse-comparing two strings, backwards from the end. Does that work ?

Example: So if you want to go from the current url above, which is for modem 1, to modem 3 say, and the current per-modem list entry is as above, then :
  list entry for new modem  = "http://m3.csw.me.uk:8000/"
     dest url for new modem = "http://m3.csw.me.uk:8000/" + final = "http://m3.csw.me.uk:8000/#Stats"

To switch from one modem to another, the user can either click on one of the m1 m2 m3 links, or use forward/ backwards or edit or type in a url or goodness knows what. It’s only in the case of clicking on the n1 m2 etc links that we need to do the calculation of new dest url based on current url and other arguments.

I have absolutely no idea about how to write the code; my javascript isn’t at all up to it and I haven’t done any in 15 years.

I really don’t know how to include the list in the config. That’s somewhere else that I’m stuck.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Alex Atkin UK on October 18, 2020, 04:30:41 PM
A local web page? eg

Code: [Select]
<!DOCTYPE html>
<html>
 <head>
  <title>Modem Monitor</title>
 </head>
 <body>
 <p>
<a href='http://m1.csw.me.uk:8000/' target='routerui'>Router 1</a>
<a href='http://m2.csw.me.uk:8000/' target='routerui'>Router 2</a>
<a href='http://m3.csw.me.uk:8000/' target='routerui'>Router 3</a>
<a href='http://m4.csw.me.uk:8000/' target='routerui'>Router 4</a>
</p>
 <iframe name='routerui' style='width: 100%; height: 768px;' src='http://m1.csw.me.uk:8000/'></iframe>
 </body>
</html>

I looked into getting the url of the current page in the iframe to extract the anchor (so it would link to the same page when you click on different routers) but it seems to be blocked by modern browser security due to the ports being different between the local web server and the router stats web server.

So it could probably work if that page was embedded in one of the routers, that way the ports would match.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: GigabitEthernet on October 18, 2020, 06:35:41 PM
Seems like the best firmware for vectoring is the one Openreach tested as working with it.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 18, 2020, 09:03:27 PM
@Alex We discussed the problem of getting a local web page to work on an ipad in another thread some time ago for this reason. Thank you for your ingenious solution and the work involved. My html is very rusty, haven’t done any in 15 years.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: GigabitEthernet on November 20, 2020, 01:04:20 PM
Big thanks to siofjofj for this firmware for the VMG1312-B10A: https://www.dropbox.com/s/8hd7soax9m5tyqf/1312-B10A-17-jumbo-v2.bin.

This has the jumbo frames patch applied and also the A2pv6F039v2 DSL firmware. This was tested by Openreach as being compatible with vectoring.

I've been testing for just over 12 hours and no issues so far. Thanks again!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on November 22, 2020, 12:51:48 AM
Got it. Thanks, will report back, will take a little while though.



On an unrelated matter: Firmware enhancement request: could the graphical stats server code reject various eg negative numbers present in the raw data when they are nonsense? I’ve noticed very very rare spikes in the CRCs graph going to approx ~-32030 and in the FECs graph to ~-4 850 000 which I would put down to Broadcom software weirdness. They are just a nuisance as they push the graph scaling into a highly zoomed-out weird state.

I have saved all the data from the johnson easy-stats server - attached.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on November 23, 2020, 12:22:35 PM
Could someone remind me as I’ve forgotten: what are the time intervals used for the counts in eg the CRC events graphs ? That is y-axis = CRC events per what? Did say in an earlier post somewhere.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on November 23, 2020, 06:16:31 PM
Per minute, I do believe.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: tiffy on November 23, 2020, 07:06:20 PM
Quote
Could someone remind me as I’ve forgotten: what are the time intervals used for the counts in eg the CRC events graphs ? That is y-axis = CRC events per what? Did say in an earlier post somewhere.

I assume you are refering to displays within DSLStats, if so, as B*cat has said the "Y" axis is per min..

Note, the DSLStats "Averages" page displays CRC, FEC, HEC, ES, SES errors per hour, daily rates.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on November 23, 2020, 07:56:09 PM
@tiffy No, to the johnson easy-stats graphs that are published by the second http server in the modem with custom firmware. As in the title of this thread. I should have said, sorry.

@Burakkucat - to which software were you referring, my friend ? How did you interpret my confusing post?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: tiffy on November 23, 2020, 08:00:13 PM
Quote
@tiffy No, to the johnson easy-stats graphs that are published by the second http server in the modem with custom firmware. As in the title of this thread. I should have said, sorry.

Ahh, apologies for butting in without reading the full thread.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on November 23, 2020, 08:24:53 PM
My fault though Tiffy, as it was very unclear.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on November 23, 2020, 10:37:38 PM
@Burakkucat - to which software were you referring, my friend ? How did you interpret my confusing post?

Hmm . . . I based my response on the tingles felt in the whiskers whilst thinking about the last time you asked about sampling rates. So I could just be caterwauling nonsense.  :-[
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on November 23, 2020, 11:28:05 PM
It’s in an old thread somewhere, but I don’t know what to search for.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on December 06, 2020, 06:01:46 PM
Did anyone manage to work out what the story is about the per-time units and the event counts on the main stats summary page eg CRCs ES SES are since-when? Or total of everything, cumulative. Or cumulative since what point. Or per most recent time interval, per-what?
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 14, 2021, 12:54:18 PM
Did anyone ever work this out? Of course our friend johnson will know.

On another matter, wish-list item:

In a future version I would like the items in the horizontal menu to be links to new ‘pages’, with real current-state URLs, so that the back button would work and we could bookmark the various individual pages/graphs. Would that be difficult? I don’t know enough about javascript and web pages.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Trunkz on May 27, 2021, 08:40:36 AM
I'm currently running AAXA.7 on a VMG1312-B10D; the current modified firmware is .8 however i believe from posts on here that it should not be used as .8 has issues with ipsec vpns. Does somebody have the .7 or .10 modified firmware with support for MTU1508?

Thanks!
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: meritez on May 27, 2021, 09:27:12 AM
I'm currently running AAXA.7 on a VMG1312-B10D; the current modified firmware is .8 however i believe from posts on here that it should not be used as .8 has issues with ipsec vpns. Does somebody have the .7 or .10 modified firmware with support for MTU1508?

Thanks!

There's a source code copy of AAXA5 for the VMG1312-B10D here: http://datashed.science/misc/bcm/ which has a compiled copy for bridge mode allowing MTU1508, you need to flash the ras.bin from here: http://datashed.science/misc/bcm/firmware/

As the page states, it's untested though there's sourcecode if anyone is willing to setup a docker instance and build it.

Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 27, 2021, 10:55:12 AM
> Did anyone manage to work out what the story is about the per-time units

Our good friend Burakkucat expressed his opinion that the unit time interval is one minute, so eg error count per min etc.  :'( ??? I just missed it earlier in this very thread. Duh. Attempting to blame the drugs is a poor excuse but it is indeed true, amongst other things. And I believe that this is not the first time he has reminded me either. Thanks to Burrakucat once again.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Trunkz on May 30, 2021, 10:22:18 AM
There's a source code copy of AAXA5 for the VMG1312-B10D here: http://datashed.science/misc/bcm/ which has a compiled copy for bridge mode allowing MTU1508, you need to flash the ras.bin from here: http://datashed.science/misc/bcm/firmware/

As the page states, it's untested though there's sourcecode if anyone is willing to setup a docker instance and build it.
Thanks for this. The site states a replacement config needs to be applied after the .bin file is flashed as well as manner of hash decription? Is this necessary or can one simply flash the firmware file noted above.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 26, 2021, 04:39:02 PM
Could someone help me with a small enhancement ? I’d like to force spurious negative values to be zero in certain graphs, as I get gigantic negative spikes occasionally, which cause the rescaling of the graphs to something stupid such that I can’t even see the real data, as on that scale it’s too small.

I don’t have the kit to rebuild the project on, and my brain is fried, so I need a friend to locate the right place to add a line of code here and there.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on April 03, 2022, 01:20:05 AM
The answer was 30 s, not 1 min, as Burakkucat said. Mr Johnson has now done a new release that explicitly states the units: "/ 30s" etc on the y axis.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on May 07, 2022, 12:35:47 PM
See also https://forum.kitz.co.uk/index.php/topic,21273.msg452852.html#msg452852 - build request.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenanc
Post by: Weaver on October 01, 2022, 09:56:32 PM
Talked about this a long time ago. There’s a bug deep down somewhere or other that causes an occasional single large negative error count, in one single bucket only. I have never seen two adjacent buckets with negative counts in them. I have just seen an error count of about -50k / 30s in the CRC dataset for modem 2. The full stats are attached, zipped up.

It might be a nightmare to find this. One thing that would be nice to have though, so a workaround/feature-request would be to:

    * Limit all negative values to zero in the disparaged graphs where negative values are nonsense. Is there an appropriate point to just dispose a limit on the data values in held RAM just before they go off to be graphed ? In certain graphs where appropriate, that is.

* I have just noticed something: the big negative spike in the graph below is at 06:00 pretty much exactly. That has to be suspicious. What was BT (or AA) doing at or just before 06:00? When I zoom right in on it, I can see that there seems to be downstream data ‘missing’ (apparently, I haven’t examined the dataset itself in the relevant file by eye yet) from 05:58:48 - 00:00:48, so 2 mins of weirdness, like a gap. Just after 06:00, the next sane value is zero, so things go straight back to normal; as mentioned before, just the one mad negative sample.

(https://i.postimg.cc/507Y8rrB/9-F463309-2871-403-B-887-F-1-F7-DB231-E79-E.png)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on October 18, 2022, 11:26:33 PM
Thank you for the data!

As far as I can tell the issue lies with xdslctls output. I was sure it must be something like an issue with parsing data under certain conditions resulting in subtracting a running total from zero (maybe when the length of the stats file output changes during resync), but from the data supplied it seem the total counters actually did reset to zero themselves. Summing the columns in the logfile I reach nearly (not sure why not exactly) the totals in the stats file listing here:
Total time = 2 days 1 hours 8 min 59 sec
FEC:      131978      5693
CRC:      57628      27
ES:      4249      10
SES:      1260      0
UAS:      13784      12700
LOS:      128      0
LOF:      287      0
LOM:      7      0

So including the spurious negative values we reach the same total, which means the modems totals must have reset to zero, something that normally never happens without a reboot, or issuing the command 'xdslctl stop'.

If we graph the sum of downstream FEC it makes the reset clear:
(https://i.postimg.cc/Y0w4smhs/FEC-sum.png)

So the easiest fix would just be resetining to zero on a negative value for the FEC and CRC graphs.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 19, 2022, 12:52:16 AM
Thank you for that and for your research work!

On a different matter, a general question: using the http port 8000 API, is there a way to get hold of some values such as the attenuation, the link-uptime and o/s-uptime/time since boot? (One of these latter values might be available at the end of one of the long, long data listings though.)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on October 19, 2022, 01:35:09 AM
Thank you for that and for your research work!

On a different matter, a general question: using the http port 8000 API, is there a way to get hold of some values such as the attenuation, the link-uptime and o/s-uptime/time since boot? (One of these latter values might be available at the end of one of the long, long data listings though.)

Attenuation and link uptime as displayed in the status page would need you to parse them out of :8000/data/stats just like the javascript in the page does, and unfortunately neither are stored in the log file if you mean finding historical values - i.e what was the attenuation before this resync - but could be added.

No way to get uptime with the modem as it is.

I've been playing with the httpd server included with busybox in the stock firmware (rather than the zyxel httpd that runs the normal web UI), its remarkably easy to get old school cgi scripts working for what ever you might want. If you were feeling adventurous you could try something like:

Code: [Select]
~ # cd /data
/data # mkdir -p www/cgi-bin
/data # busybox httpd -h /data/www/ -p 8001
/data # echo "#!/bin/sh" > www/cgi-bin/uptime
/data # echo "echo" >> www/cgi-bin/uptime
/data # echo "uptime" >> www/cgi-bin/uptime
/data # chmod +x www/cgi-bin/uptime

Visiting modemip:8001/cgi-bin/uptime you would be greeted by:
 08:54:26 up 40 min, load average: 1.03, 1.05, 1.00

The other values you mentioned could be awked out in similar scripts residing in /data/www/cgi-bin. Starting of this server could be added without issue to your processing of /data/boot-cmds.sh.

Edit:
I haven't forgotten about the ssh server btw, these additional values could be added to the normal stats server then. 
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on October 19, 2022, 03:39:18 AM
If you are bored and feel like testing:
https://www.dropbox.com/s/tdvgv6a9sb1anzr/1312-ssh-test.bin

No fix for the negative error graphs, but it does have:
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 19, 2022, 10:51:43 PM
Amazing, thanks for all that work and those excellent tips! As we talked about before, by my own version numbering system, shall I then call that release my.v.9.0?

I’ll blow one modem shortly and start playing with it. Having a way to inject files into them will make life so much better. I have a pile of spare modems that I should get round to setting up in readiness as in the event of another lightning strike, I don’t want the added pressure of having to flash modems in addition to the other worries on such an occasion.

By the way, I’m perhaps being daft or misunderstanding but I couldn’t find "attenuation" in port:8000/data/stats, even though I did a search for the text rather than just scanning the contents by eye. What am I doing wrong?

One further thought for you and Burakkucat. My stupid iPad drops any TCP connection to the modem when I task-switch away from the telnet/ssh app. Normal and maddening iOS/iPadOS behaviour. When this happens, does your modem’s shell then terminate? It does for me so I keep losing my session within a timeout of a few tens of seconds. I need the "screen" program to prevent this, I think, which works well on my Raspberry Pi.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: johnson on October 20, 2022, 12:32:33 AM
Amazing, thanks for all that work and those excellent tips! As we talked about before, by my own version numbering system, shall I then call that release my.v.9.0?

Blast! I did at least remember to change the version string, but just to "V17-ssh-test", v9 it should be.

Quote
I’ll blow one modem shortly and start playing with it.

One thing to mention is that (as the ssh login is a a real one reading the /etc/passwd file, rather than escaping the zyxel created one that runs as root) to do anything interesting requires the supervisor account. A handy script is in the $PATH to find the supervisor password if you dont already have it, superpass, its just base64 encoded on my modem at least.

Quote
By the way, I’m perhaps being daft or misunderstanding but I couldn’t find "attenuation" in port:8000/data/stats, even though I did a search for the text rather than just scanning the contents by eye. What am I doing wrong?

The line begining Attn(db):

Line Status:      No Defect
Training Status:   Showtime
Down      Up
SNR (dB):    7.6       6.1
Attn(dB):    68.0       41.6
Pwr(dBm):    16.4       12.4

To awk  downstream out:
Code: [Select]
awk '/^Attn/ {print $2}' stats

Quote
One further thought for you and Burakkucat. My stupid iPad drops any TCP connection to the modem when I task-switch away from the telnet/ssh app. Normal and maddening iOS/iPadOS behaviour. When this happens, does your modem’s shell then terminate? It does for me so I keep losing my session within a timeout of a few tens of seconds. I need the "screen" program to prevent this, I think, which works well on my Raspberry Pi.

Hmm, I'l try and recreate it. See if this version of dropbear produces the same behaviour, I believe its timeout is set to zero - but I'm sure screen or tmux will compile if needed.
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: burakkucat on October 20, 2022, 12:57:44 AM
One further thought for you and Burakkucat. My stupid iPad drops any TCP connection to the modem when I task-switch away from the telnet/ssh app. Normal and maddening iOS/iPadOS behaviour. When this happens, does your modem’s shell then terminate? It does for me so I keep losing my session within a timeout of a few tens of seconds. I need the "screen" program to prevent this, I think, which works well on my Raspberry Pi.

Either "screen" or "script" would be appropriate for your need.  :)
Title: Re: ZyXEL Modems - eg VMG1312-B10A - Custom Firmware MTU 1508 Maintenance
Post by: Weaver on October 20, 2022, 03:40:53 AM
Ah, "Attn" - I forgot about searching for the abbreviated form. :-[

I’m pretty sure that the dropping of the TCP connection is the fault of iOS, but I forget where I got that idea from. I used to have these problems with the Raspberry Pi until Burakkucat set me up with screen. I looked into tmux too but didn’t get around to playing with it.

As for the version number, we’ll have future opportunities. Having agreed version numbers means that we can talk to one another with a common language and know what we’re talking about, and it really helps me in being able to appropriately create version-numbered subdirectories that we both understand.

I do already have the supervisor password for all my modems.