Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Weaver]

My ZyXEL VMG 1312-B10A modems, which are in modem-only mode and have custom firmware built by our own Mr johnson, have a few problems with their networking.

1. They don’t have a default gateway set for IPv4. I have yet to fix this, would appreciate some help. I can’t see an opportunity to do it in the ZyXEL XML config file, but I may have missed it. I need to research how to declare it in Linux, bearing in mind that this is a very cut-down version and a lot of the usual executables and facilities are absent.

Without this, a lot of networking is simply broken. I’m making up for this by intercepting all known IPv4 coming from the modem and getting my Firebrick router to NAT it. But it’s not ideal.

2. IPv6 doesn’t fully work. I would like to clean this up, but it’s hardly a priority. I’m not sure how to get this minimal Linux build to listen for an RADVD prefix announcement so it can generate a global IPv6 address or two for its LAN i/f. It does have link-local IPv6 addresses, but that doesn’t help us much, since this is presumably a different L2 broadcast domain (a different ‘LAN’) from my normal main LAN and has only two machines on it: the modem, and my Firebrick’s per-modem facing i/f.

IPv4 addressing conventions: The Firebrick’s per-modem i/f facing a modem is 192.168.n.254, and a modem’s admin i/f facing the Firebrick is 192.168.n.1, where n = 1..4 the number of the modem, from four modems in total.

3. NTP time; Yay! I now have timekeeping working successfully because the Firebrick is a relay NTP server and with the correct XML config I have the modem looking at 192.168.n.254 set as a good time server in the list and with this I don’t need to go to eg my ISP’s nearby NTP servers or 1.1.1.1 or whatever - which would be difficult given the system’s current inability to talk directly to the internet, which is a showstopper. (That’s because of the lack of an IPv4 gateway definition, plus the use of RFC1918 addresses for the modem, and the fact that the current Firebrick NATing only takes the modem to the main LAN.)

- [ ] 4. I’ve just got DNS working for the first time. I set /etc/resolv.conf by hand to make it use the Firebrick’s accessible IPv4 i/f where the Firebrick provides a local caching relay recursive DNS server service. Another minor triumph. But how to make this persist? Need the config changes initially made by hand to survive a reboot. For this I used Mr johnson’s /data/boot-cmds.sh custom boot commands feature, writing a few commands into the aforementioned custom boot script file to define three name servers, which are 127.0.0.1 and ::1 then 192.168.n.254 for the Firebrick again, which is the nearby DNS
I don’t yet have a clean way of dealing with the variable .n.254. I can’t remember how to expand variable names in bash or whatever busybox shell we have here. Currently I’m ashamed to say that I have written out four custom script files, one per modem, eg modem-set-dns-n.sh and I have to remember which one to run on which modem!

Rather than customising the script files with a variable at the top or passing a parameter in, what I really need to do is work out which modem I am. I could perhaps inspect the XML config but that seems a bit awkward. Assuming the XML config has already taken effect and the modem has booted configured in the correct per-modem state we see the correct IPv4 address on its LAN-facing i/f. We do have say interface br0 showing the expected IPv4 address. Is i/f br0 the right one to choose to inspect ?

Anyway, I could get the modem number out of the IPv4 address - I could attack it with sed? Tips would be appreciated as it’s a long time since I looked at this, but regexes are my daily pleasure on the iPad. Deriving the modem n value and then injecting it into the script to specify the name servers, that would make the script per-modem-independent, so much less faffing about and no longer error-prone. The basic problem is to transform 192\.168\.n\.1 or even 192\.168\.n\..* into 192\.168\.n\.254, as simply as that. I’m thinking grep alone might do it, rather that sed, but I’m not sure which approach would win the beauty contest.

5. A bash tip appreciated again. I am thinking that I should not write over the entirety of the content of the boot-cmds.sh script here. If somewhere down the line, I end up having various unrelated functions invoked in the custom boot script, then these functions’ respective installers or updaters should certainly not stomp over and overwrite each other on installation in the boot-cmds.sh file. What I’m thinking of is to always have calls to a separate script from the main boot-cmds file. The separate script will have a meaningful name such as johnson-custom-boot—set-dns.sh. What I should be doing is to detect that line that contains a call, and if it’s not found, then insert it, being careful not to add or append an initialisation call twice so we’re making multiple calls to install the same thing. Is this another larger job for sed? If so, could do with some help.

6. Would be nice to somehow have a choice of insertion point in the main custom boot script. It’s not necessarily good enough to just always append everything as I can foresee circumstances where there might need to be a certain ordering because of dependencies. For example in this particular case later calls might need the availability of DNS which only comes into being after  this insertion point in the overall main script, so always appending the DNS setup thing would not be at all clever. I don’t know how to specify this. Perhaps define it by dependencies, so ‘before-xx’ and ‘after-xx’ ?

7. SSH. It just fails completely. I just get as far as some incomprehensible message about certs or something when I SSH-connect to a modem from my iPad. The username and password are correct; the password is pasted in from a password-store in the terminal program, so it’s not being mistyped. That username+password works at a normal shell login prompt after a telnet connect. I’m using the app ‘Prompt’ on my iPad to access the modem. Anyway, straight after the incomprehensible message, the CLI process on the iPad just crashes and disappears. BANG! All the modems do it, and it’s 100% reproducible. It would be so incredibly useful if I could get SSH going. This is because I’ve been cobbling together iPad programs using the iOS Shortcuts framework and that has library functions to perform operations over SSH but not Telnet. But at the moment it’s all telnet-only for me.
8. SAMBA? Having said it’s a minimal installation, this Linux build has some rich components still left in it. I see SAMBA. That might be incredibly useful if I can get IPv6 installed properly, as I can easily have real IPv6 addresses both link-local and global/routable but I would be grateful for some IPv6 installation-related Linux tips and then the SMB server config might hopefully not be too bad? That might be great to help in installation work.
9.   Would it be a nightmare getting other basic communications services going? Bonjour/Avahi/mDNS would require a new build perhaps with the help of my good friend Mr Johnson, who has already done so much. I wonder how much of a pain it would be to get SMTP going? It would be a lot easier to get something such as that going over IPv6 as there there is no problem arising from the RFC1918 addressing, and as mentioned before, there’s nothing in the way of getting real IPv6 addresses going as needed. I can also see what looks certain to be TFTP services in this Linux build.
    10. Other kinds of notification services? Having the modems send you a message when something bad happens, or something inserting happens, that could be very useful indeed. What other facilities in this area could there be found in a modest Linux installation such as this?
I believe Alex is fond of SNMP? Not something that I have ever got my head around.

Anyway, tips, guidance, reading matter, all appreciated, Especially a bit of hand-holding, as my brain has turned to mush just now.

[tubaman]

With SSH, assuming public key authentication, you will get a certificate message the first time you connect. As long as you are happy that you are connecting to the right place then enter 'yes' and you shouldn't get asked again.

[Weaver]

Understood. And immediately after that prompt, SSH crashes on the remote end, ie the process running on the modem dies.


Update: I’ve made a lot of progress getting IPv6 set up on the modem. I’ve been googling linux ipv6-related address and route setup commands with success. But then the next thing hit me. This isn’t going to work because my Firebrick (router) doesn’t have the correct routing set up for IPv6 to go to the modems. Also it does not have any ipv6 addresses allocated for each modem yet. So this simply isn’t going to work.

I can fix that, but it’s quite a lot of work. I suppose that I should allocate a /64 to each modem from my existing /48 ? I’m at a loss here and rather than starting adding ip addresses and forcing routing as I did in the Firebrick for the modems with IPv4 RFC1918 addresses, I suspect that I want to define interfaces or subnets or something, and that’s only the start of it. I would definitely have to ask AA for help.

The other thing is that I can’t start experimenting with the XML config on my Firebrick; it’s a critical unit, in use all the time and I cannot mess it’s config up while it’s in use. I do have a spare Firebrick but setting it up with a realistic configuration would be difficult as it would need a normal-looking internet access link. I physically can’t do the work needed to set up a test Firebrick alongside my main one.

So with regret, I’ll just have to say that IPv6 is off the schedule for now at least, as in any case, it was always a ‘nice to have’.

[tubaman]

Just had a play with SSH on my VMG8924-B10A and it refuses to connect via my Linux laptop as it is using an old, and now considered insecure, key exchange method - diffie-hellman-group1-sha1.
Apparently Putty still allows that key exchange but I haven't tried it yet.

[Weaver]

Does anyone have MIB files for ZyXEL VMG1312-B10A modems ?

[johnson]

1. They don’t have a default gateway set for IPv4. I have yet to fix this, would appreciate some help. I can’t see an opportunity to do it in the ZyXEL XML config file, but I may have missed it. I need to research how to declare it in Linux, bearing in mind that this is a very cut-down version and a lot of the usual executables and facilities are absent.

Setting it with ip should work fine:

Code: [Select]

ip route add default via 192.168.1.254


Code: [Select]

~ # ip route add default via 192.168.1.1
~ # ip route
192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.5
default via 192.168.1.1 dev br0
~ #

Trying this on a 1312 I need to also disable the firewall on the modem to reach the internet, just setting it to "Low" in the web interface is not enough, not sure why. Are you trying to get internet access on the modems? I cant test this very easily as I dont have the same firebrick set up as you.

2. IPv6 doesn’t fully work. I would like to clean this up, but it’s hardly a priority. I’m not sure how to get this minimal Linux build to listen for an RADVD prefix announcement so it can generate a global IPv6 address or two for its LAN i/f. It does have link-local IPv6 addresses, but that doesn’t help us much, since this is presumably a different L2 broadcast domain (a different ‘LAN’) from my normal main LAN and has only two machines on it: the modem, and my Firebrick’s per-modem facing i/f.

Afraid I dont have a lot to add, if your firebrick setup escapes me then the finer points of how it works with IPv6 is beyond me. I can say that the radvd binary exists in the build and runs:

Code: [Select]

~ # radvd
[Jun 22 06:42:42] radvd: can't open /etc/radvd.conf: No such file or directory
[Jun 22 06:42:42] radvd: Exiting, failed to read config file.

Looks like it needs a config, other config files in /etc are symlinked to /var ram filesystem with things filled in at boot by the zyxel software, we can add one for it but in the meantime you can play with it by specifying its config with --config=PATH.

- [ ] 4. I’ve just got DNS working for the first time. I set /etc/resolv.conf by hand to make it use the Firebrick’s accessible IPv4 i/f where the Firebrick provides a local caching relay recursive DNS server service. Another minor triumph. But how to make this persist? Need the config changes initially made by hand to survive a reboot. For this I used Mr johnson’s /data/boot-cmds.sh custom boot commands feature, writing a few commands into the aforementioned custom boot script file to define three name servers, which are 127.0.0.1 and ::1 then 192.168.n.254 for the Firebrick again, which is the nearby DNS
....
Anyway, I could get the modem number out of the IPv4 address - I could attack it with sed? Tips would be appreciated as it’s a long time since I looked at this, but regexes are my daily pleasure on the iPad. Deriving the modem n value and then injecting it into the script to specify the name servers, that would make the script per-modem-independent, so much less faffing about and no longer error-prone. The basic problem is to transform 192\.168\.n\.1 or even 192\.168\.n\..* into 192\.168\.n\.254, as simply as that. I’m thinking grep alone might do it, rather that sed, but I’m not sure which approach would win the beauty contest.

Excellent, someone has a use for the boot-cmds script! As far as getting hold of the modem number, something like this?

Code: [Select]

ip route | awk '{print $9;exit}' | sed 's/.$/254/'


Code: [Select]

~ # ip route
192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.5
default via 192.168.1.1 dev br0
~ # ip route | awk '{print $9;exit}' | sed 's/.$/254/'
192.168.1.254
~ #

There is probably a nicer or more suitable way of doing it depending on how you intend to use the variable after, if you show how and what you are setting in resolv.conf from boot-cmds.sh.

7. SSH. It just fails completely. I just get as far as some incomprehensible message about certs or something when I SSH-connect to a modem from my iPad. The username and password are correct; the password is pasted in from a password-store in the terminal program, so it’s not being mistyped. That username+password works at a normal shell login prompt after a telnet connect. I’m using the app ‘Prompt’ on my iPad to access the modem. Anyway, straight after the incomprehensible message, the CLI process on the iPad just crashes and disappears. BANG! All the modems do it, and it’s 100% reproducible. It would be so incredibly useful if I could get SSH going. This is because I’ve been cobbling together iPad programs using the iOS Shortcuts framework and that has library functions to perform operations over SSH but not Telnet. But at the moment it’s all telnet-only for me.

Yes not having SSH is annoying, use to be that you could specify to allow the weak key exchange and it would work, but modern clients now just refuse.

I found the magic incantations to get dropbear compiled using the build environment:

Code: [Select]

./configure --enable-static --host=mips-linux-uclibc LDFLAGS="-L/opt/toolchains/uclibc-crosstools-gcc-4.4.2-1/lib -L/opt/toolchains/uclibc-crosstools-gcc-4.4.2-1/usr/lib" --disable-harden --disable-zlib

After some faff with psudoterminals it works, but only on another port, 22 is occupied by the zyxel ssh server. Not sure how to stop that yet, its not running as sshd or similar.

A modern SSH server was high on the list of reasons I wanted to make a minimal modem only firmware using http://buildroot.org/, but I guess shoe horning it into the normal build isnt that bad an idea.

[Weaver]

Wow, thank-you for your generous and helpful reply! I’m not too well at the moment so will return to looking into this later on now I’m armed with your tips.

I would like to think of a way of automating the business of getting custom boot commands injected into a modem. Currently I have an iOS Shortcuts program that writes out a customised XML config and I just load that into the modem by hand using the usual admin web page. This is because I might have say six modems to configure in one go, four ‘live’ ones and some clones as immediate backup ready-cooked devices. I can’t see how to push say bash script files into the modem from an iPad. I don’t have SAMBA running at my end and it’s understanding not running in the modem, thank god. As I might have mentioned before, finding somewhere to hide something in the XML config so that it won’t upset anything, and then somehow have the modem pull that skipped-over XML element out and put it into the filesystem. I wouldn’t know how some code inside the modem might find the uploaded XML config.

If only SSH worked. I can remotely run shell scripts via SSH on a remote machine using an iOS Shortcuts library function, so everything I would need is already there. It’s just that SSH bombs out part way through login.

To answer your question, no I’m not trying to get internet access. I was, for NTP, but now I don’t need to because the Firebrick itself has become a local NTP server. About the gateway thing, I found that a lot of the networking understandably did not work without a default gateway defined. You’ve now given me the solution. I forgot about AWK, haven’t used that for a while, although I used it for some tools with good results a few years ago.

[johnson]

If only SSH worked. I can remotely run shell scripts via SSH on a remote machine using an iOS Shortcuts library function, so everything I would need is already there. It’s just that SSH bombs out part way through login.

Do the iOS SSH tools allow the use of a non standard port? I have dropbear compiled and running, but it cant use port 22 as the main zyxel config program contains the ssh server and fails to run if it cant bind to 22.

To answer your question, no I’m not trying to get internet access. I was, for NTP, but now I don’t need to because the Firebrick itself has become a local NTP server. About the gateway thing, I found that a lot of the networking understandably did not work without a default gateway defined. You’ve now given me the solution. I forgot about AWK, haven’t used that for a while, although I used it for some tools with good results a few years ago.

Yeah I like awk, {print $2} etc seems fairly readable to me. Maybe some would scoff at using it for simple things like getting the last field of a line, but hey.

As I might have mentioned before, finding somewhere to hide something in the XML config so that it won’t upset anything, and then somehow have the modem pull that skipped-over XML element out and put it into the filesystem. I wouldn’t know how some code inside the modem might find the uploaded XML config.

I wouldnt know where to start with getting the XML config to trojan in scritps to run, the zyxel software doesnt store it anywhere in plain text that I know of. Would be nice to just have the single step in your commisioning process though.

[Weaver]

The Shortcuts’ library fn "run script over SSH" does allow an alternative arbitrary choice of port.

That would indeed mean I could just get the whole thing automated, because I could pass the content of a script that is to be stored to the remote machine’s filesystem as a parameter and use a little sh workhorse routine I’ve written to finish the job off.

I do wonder what is up with the standard SSH app though. I wonder if they’re using a bad elderly version of that code. I’m trying to remember- was there something about you needing to go back a version to get multiple telnet sessions to work? I think, am not sure, that I did try other SSH apps on the iPad too, to see if the problem was confined to just the iOS ‘Prompt’ app, but iirc I had no luck elsewhere.

With your tips, I finally got the IPv4 default gateway set, (yay!), but given the force of the existing bodged up networking I have already set on the Firebrick side, I was able to access the internet outbound anyway. I could successfully for example do an nslookup -h www.google.com over IPv4.

I took your technique from this thread and of course it worked fine with sed as is, but when I tried to adapt it to use a $variable inside "double quotes" (I did at least remember double quotes) I then kept getting a failure of the regex matching. My regex prowess is reasonably good as I use them all the time, but I’m wondering if there’s some feature support thing in that version of sed (within BusyBox?) that I’m unaware of, or maybe something like (is it) grep -e needed to enable the good stuff to be turned on? I wanted to extract just the modem number, rather than altering the string as you did, ie pull out the third byte nn from eg 192.168.nn.1, so that I could use it elsewhere. I was trying to use sed to do the following
   match= '^.*(\d+)\.\d+$' ; replace with= $1  ie the () group. But it’s ages since I’ve used sed, and I’m unsure about things such as backslash-escaping needed where the first regex is inside single quotes as in 's/^.*(\d+)\.\d+$/whatever/' and also I forgot what the syntax is for the replace-with when you wish to specify a reference to a group in the first regex. And sleep overcame me at that point, so off to google it.

[Weaver]

I later found that, following some changes, I had disturbed the pattern that the call to awk relies on, as the layout of the text output by ip route differed for me compared with what I and Mr Johnson had seen. I tried using ifconfig instead as I according to experiments its output syntax is more fixed. So I did an "fconfig br0" as that seemed to be the most suitable i/f on my modem.

Try as I might I can’t get the following sed to work, even though I’ve validated the regex against the captured exact text input in question using the (superb, highly recommended) web-based tool at https://regex101.com/, which is my regular workhorse when debugging. That tool also has various switches to control its operation including dialect switching. It may very well be that I should have been using some of these but I’m not sure how, although I have intentionally eschewed any regex features I thought just possibly might be unavailable in that sed version.

My sed is now as follows. I skip 0..n lines until I see a line starting with "inet addr:" and then an IPv4 address, and then I grab the third byte, skipping all of the rest.

ModemNumber="$(ifconfig br0 | sed 's/^(?:.*\n)*\s+inet addr:\d+\.\d+\.(\d+)\.[\s\S]+$/$1/')" ; echo $ModemNumber

And here’s the input I have to work with:

br0       Link encap:Ethernet  HWaddr 04:BF:6D:AF:16:08 
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: 2001:8b0:1ce::2/64 Scope:Global
          inet6 addr: fe80::6bf:6dff:feaf:1608/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1508  Metric:1
          RX packets:65930 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1070874 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4437519 (4.2 MiB)  TX bytes:99986240 (95.3 MiB)

So if any Linux gods would cast an eye over it to see if there’s anything obviously dodgy.

[burakkucat]

I create a file named zxc with the following contents --

Code: [Select]

br0       Link encap:Ethernet  HWaddr 04:BF:6D:AF:16:08
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: 2001:8b0:1ce::2/64 Scope:Global
          inet6 addr: fe80::6bf:6dff:feaf:1608/64 Scope:Link
          UP BROADCAST RUNNING ALLMULTI MULTICAST  MTU:1508  Metric:1
          RX packets:65930 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1070874 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4437519 (4.2 MiB)  TX bytes:99986240 (95.3 MiB)

To mimic your command sequence I then execute --

Code: [Select]

ModemNumber=$(cat zxc | sed 's/^(?:.*\n)*\s+inet addr:\d+\.\d+\.(\d+)\.[\s\S]+$/$1/')

Followed by --

Code: [Select]

echo $ModemNumber

The above echo command line returns --

br0 Link encap:Ethernet HWaddr 04:BF:6D:AF:16:08 inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: 2001:8b0:1ce::2/64 Scope:Global inet6 addr: fe80::6bf:6dff:feaf:1608/64 Scope:Link UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1508 Metric:1 RX packets:65930 errors:0 dropped:0 overruns:0 frame:0 TX packets:1070874 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4437519 (4.2 MiB) TX bytes:99986240 (95.3 MiB)

If I make use of awk --

Code: [Select]

ModemNumber=$(cat zxc | awk -F. '/inet addr:192.168/ { print $3; exit}')

The an "echo $ModemNumber" returns the expected digit "1".

[johnson]

+1 for awk, long regexes make my head hurt!   

[burakkucat]

I am surprised that an "ip addr show br0" command cannot give the required information but an obsolete "ifconfig br0" command provides the required output. 

[johnson]

I am surprised that an "ip addr show br0" command cannot give the required information but an obsolete "ifconfig br0" command provides the required output. 

Maybe my suggested ip route differs in output order on weavers modem, with a default route set it outputs 2 lines. In any case this:

Code: [Select]

ModemNumber=$(ip addr show br0 | awk -F. '/inet / {print $3}')
echo $ModemNumber

Outputs the expected value '1', on my modem at least.

[Weaver]

The reason that Johnson’s awk 9 solution worked then failed on my modem was because another line of text had appeared at the start, something that mentioned the newly added default ipv4 route.

I eventually worked up why I was stuck. In this sed (iirc) version, the common abbreviation \d for [0-9] doesn’t work, and that was what was driving me up the wall until I realised.

I’m ashamed of my own effort, cooked up while our efforts were crossing over. Burakkucat’s solution was I think posted during my Swedish class on Monday evening. But it works and is here for your amusement

Code: [Select]

echo 'br0       Link encap:Ethernet  HWaddr 04:BF:6D:AF:16:08  ' >f.txt
echo '          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0' >>f.txt
echo '          inet6 addr: 2001:8b0:1ce::2/64 Scope:Global' >>f.txt
echo '          inet6 addr: fe80::6bf:6dff:feaf:1608/64 Scope:Link' >>f.txt
 
ModemNumber="$( cat f.txt | sed -r 's/^\s\s+inet\s+addr:?\s*[0-9]+\.[0-9]+\.([0-9]+)\..*/\1/' | grep '^[0-9]' )" ; echo $ModemNumber
I like Burakkucat’s second solution a lot. It’s short and looks robust in respect of future-proofing considerations in case a later build should ever change the layout of the text in the tiniest of details such as whitespacing.

Many many thanks to both of you. The value of this little journey is not just in the end-product, I now have the long-term ability to detect ‘which modem am I?’, but there’s also the huge amount I’ve learned in a couple of days and I would never have got there without your help.