Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: Fun with AWK and GAWK  (Read 2991 times)

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Fun with AWK and GAWK
« on: June 04, 2019, 05:04:55 AM »

I had a go at writing my first ever AWK program. This is a tool that is a bit like the C preprocessor where it handles #if’s bug applied to XML Firebrick config files.

I have invented a system where significant comments are used, comments that begin with a special very long distinctive marker and the rest of the comment contains some declaration or command. This can be a #if-like command which causes the following section to be deleted or left in.

The reason for this is that the Firebrick FB2700 and above support USB NICs for 3G and 4G (aka ‘dongles’) but the FB2500 has no usb interface. Unfortunately the FB2500 gets upset if you even mention usb or dongles, so you cannot use a single config file with both models. I have one of each, and I don’t want to have the nightmare of maintaining two files and having to keep them both up to date.

So using the #if-like conditional section markers I can hide FB2700 usb- and dongle-related stuff from an FB2500 by automatically removing the marked section, and if possible I can put in alternative sections too. So there are FB2700-only sections.

Anyway to attempt to cut a long story short, wrote this in AWK, worked fine. It preprocesses an input FB2700-format Firebrick XML config file, does the right thing with it. Alternatively it can optionally convert FB2700 format into a format suitable for upload into an FB2500, ie edited to be without mentions of usb and dongles.

However when I copied the AWK program to a new Raspberry Pi system running Ubuntu v16.xx, AWK spat out a big load of syntax errors. After tearing my hair out for ages, I worked out that this was a completely different AWK interpreter from whatever was on the other Pi. Installed GAWK and then the program worked fine. Whatever AWK executable I had been running before was lacking some enhanced facilities, the lack of support for the GAWK switch statement was the main casualty as far as I could tell.

So what should I do in general? Change the first line to be #!/bin/gawk -f ? To show that it really needs GAWK ?

What should I do about installing the program - it needs to handle its dependency- so needs to do an apt-get install -y gawk or some such ?

My other problem : Isn’t apt-get a flavour/distro specific command anyway? Is there something more portable solid command that I can use for installing such a package?
Logged

burakkucat

  • Respected
  • Senior Kitizen
  • *
  • Posts: 38300
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Fun with AWK and GAWK
« Reply #1 on: June 04, 2019, 05:52:22 PM »

So what should I do in general? Change the first line to be #!/bin/gawk -f ? To show that it really needs GAWK ?

Yes. That would be the most sensible.

Quote
What should I do about installing the program - it needs to handle its dependency- so needs to do an apt-get install -y gawk or some such ?

My suggestion would be to use --

aptitude install gawk
Logged
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.

petef

  • Reg Member
  • ***
  • Posts: 135
Re: Fun with AWK and GAWK
« Reply #2 on: June 06, 2019, 08:30:03 PM »

If you are working with XML then XSLT would be a more natural fit. XSLT has similarities with AWK being based on pattern matching but using XML syntax.

Another alternative is Python + lxml.
Logged

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Fun with AWK and GAWK
« Reply #3 on: June 07, 2019, 01:39:48 AM »

I have done a huge amount of XSLT. I’m writing it all in AWK as a learning project. I might have a go at writing the tool in D as D has a very powerful regex engine and I need that for part of it.

The awk code is just a state machine and the D code would have exactly the same structure.

It would also give me a chance to improve my D, which is sorely needed.

I started out debugging the state machine and getting it right, using AWK. I then ported the AWK to the iOS Shortcuts language is that now I have the preprocessor tool running in my iPad too. iOS Shortcuts is such a nightmare and you can easily get lost so having something already thought out, which you can just follow robotically, really helps.

I have got Python for the iPad but I wouldn’t even know where to start. It looks incredibly powerful.
Logged
 

anything