Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: Design muddle - versioning  (Read 860 times)

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Design muddle - versioning
« on: December 19, 2018, 01:02:47 PM »

I have got my brain into a muddle and could do with some kindly help.

I wish to have automatic version labelling of the content of Firebrick router config files. So when the content is changed some number gets updated automatically, and I cannot forget to do it. Either an integer, or multi-part dotted number eg v3.11, or even an iso date e.g. 2018-12-01T00:31:35.

I am using the latter at the moment as I am not sure the other versions are as useful. The other options either have too little or too much additional informational content. Don’t know what you think.

Anyway, the point is how to get these version numbers updated automatically and not forget. I need to do this on the iPad, although I suppose I could write a version server for my remote Raspberry Pi.

I upload the config XML files to my Firebrick and in fact it automatically version stamps the result with a lot of info, the IP address uploaded from, the username, a date and an incrementing integer.

That’s superb but too late. I want version numbers on the saved source config files ideally. As it is, I have no idea how to mate up the timestamps showing inside the Firebrick, in an already uploaded configuration, with source file versions, so I have no idea what has been uploaded by just looking at it.

Currently I am putting my own additional version timestamps in the config files before uploading them.

However there are a lot of inadequacies. If I don’t automate this updating, I could forget to update the version timestamps, obviously. If I do update it, the timestamps could get updated when the content has not changed, say if I upload a version twice. I’m then left wondering what has changed. I don’t want to have to download things back again and have to diff the results on a wild goose chase.

Some suggestions on how to proceed?

Really, I need something that can check what her a source (XML config) file has really changed. And then, and only then, a version timestamps gets applied to it.

Detecting difference, I suppose I could hash the file, or really difference it.

I could filter the file first - strip comments and whitespace so as to kick out differences that I really don’t care about. And I should remove the embedded version timestamp itself so that a situation where there is a (mistakenly) edited and updated timestamp within the file, and that is the only difference, does not count as ‘different’ content.

I’m not sure about hashing the file contents. Not sure what tools I have on the iPad for this, I’d have to look. Differencing the entire filtered content might be no less hassle.

But I’d have to keep either the previous file content or the previous hash stored persistently on the iPad. If I write the code for this in iOS Shortcuts, my tool of necessity, since I have not yet learned Python or Objective C, then it makes this all file io really awkward but I think I can crack it.

I still have one problem I cannot currently fix. I can’t get the stupid iOS Shortcuts library to let me read from, or most importantly write to the source XML files, so I can’t write a version timestamp into the saved source even if I compute one, or compute a conten-same/different flag.

Any help deconfusing me?

Perhaps at the moment all I can do is change version numbers manually and have an alarm which tells me when I have forgotten to change the version timestamp yet the content has (significantly) changed. A basic diff program, with a Boolean output. Would that make sense?




Logged

Weaver

  • Senior Kitizen
  • ******
  • Posts: 11459
  • Retd s/w dev; A&A; 4x7km ADSL2 lines; Firebrick
Re: Design muddle - versioning
« Reply #1 on: December 20, 2018, 04:22:30 AM »

I managed to struggle with the insanity of iOS Shortcuts’ file io and get a routine written that stores something persistently, in a file, and then later you retrieve it and match it against a new given input parameter value. This value may or may not be a match against the stored value, the routine returns true or false depending on equality. The routine finally stores a ‘different’ or ‘new’ value, read to be matched next time. So this routine is an ‘Is Different()’ or ‘HasChanged()’ routine returning a Boolean, a change detector.

A lot of the crazy behaviour of iOS Shortcuts’ file-io seems to be to do with the extensions you choose for filenames, that appears to be one thing that triggers bizarre buggy behaviour. I renamed a file which is stored in "iCloud::/Shortcuts/whatever" to have a .txt stuck in the end and it started working. That’s one thing. Also I found that I could save and load JSON text into data structures with a bit of random messing about, none of this being documented anywhere, so no one knows what ‘correct’ behaviour even is. It’s like it’s oerhapsmautomatically trying to parse the content of files that it opens, without permission. If that is what’s happening then it might explain some of the random successes/failures seen in the past with it, so whether things work or not could be file content dependent too, as it certainly depends on filenames and file locations are (sometimes) restricted too, maddeningly.

So I could just now use this as an alarm as discussed, the difference detector reminds me that I not set a source code version stamp, even though the code has actually changed. I have to now arrange for this routine to get called unavoidably, in some circumstance or other, as a nag reminder.

I’m wondering whether to hash the files down or store and compare the entire files in each new-vs-previous comparison pair, about 50k of text because of vast comments, including XML comments that document change history. That is what bloats the size. I can’t see it matters that much either way as the files are not that enormous and what you save on one operation you lose on another, it seems to me. I would in any case strip out all comments and non-significant white space but that’s not to do with performance, that’s irrelevant here. Stripping out irrelevant changes just reduces the noise level, prevents version number increases for no apparent reason, given that we decide that changes to comments or to insignificant whitespace do not count at all.

Trying to write code on the iPad using this both powerful and yet crippled and frustrating shell scripting language is a crazy thing. A mixture of enormous productivity one minute and then crushing limitations and mad omissions the next. A lot of complex programs that I have written tend to have dozens of regexes in them. But some normal loop constructs are missing, so one has to either use recursion instead, which is very inconvenient, or other kinds of strange tricks to getaway with other types of loops. I wrote a very simple sort routine and had to use dirty tricks to cope with the lack of a normal while loop for this case. Luckily, I knew the maximum possible number of times round this particular loop, and so when the real loop condition triggered and we should be all done, it then turns things of with an embracing if-statement which makes any final unneded iterations just fly past and it goes round the embracing n-times-max loop for a certain number of final left-over iterations skipping over the whole body within each time so it gets out in almost no time. Would not get away with that if the maximum loop count were unknowable or enormous. Madness.

But having automation available in an iPad right there where you want it is very good. They horrible UI doesn’t help though, as I have way over a hundred programs and subroutines, can’t see all the names and can’t find anything and can’t organise them, into favourites and standalone programs vs functions that only make sense when called by other code, not really intended to be called from the top level by hand. You can call call the routines from Siri but I would never be able to remember the names, which are all long and descriptive. Also the ui layout is a disaster in that the end of names of programs and functions get chopped off so you can’t see the entire names. Desperately need to be able to organise them in directories and modules. Currently you can’t see what needs what other routines as there’s no way of seeing who calls what from where and so if you were to try to copy some program or function to somewhere else you would never know how many associated required functions you also need too, not until some day when you might or might not get a runtime error. It doesn’t forewarn you when you start a routine that requires (external) procedures that are missing, you just run into an error or not depending on the later path you take. If you delete a routine as well, it then corrupts all the routines that called it. Apple needs to get a lot more serious about this and get some serious manpower thrown at sorting these ridiculous things out. Having a bit of respect for users.
Logged