Kitz Forum
Broadband Related => Broadband Hardware => Topic started by: hopkins35 on February 20, 2021, 07:59:45 PM
-
Of interest for those Firebrick users with bonded lines from Andrews & Arnold, the latest alpha releases feature the ability to auto detect the uplink speed of your lines:
Release notes from Alpha release 1.55.112 to Alpha release 1.55.113
PPPoE
New option to pick up speed from connect message to set egress rate on PPP (ideal for bonding)
Note that because this is still in alpha you need support to enable alpha upgrades on your brick or wait until it makes it to beta!
-
Thank you for the note. I'm sure it will interest our friend Weaver.
-
I got a message from AA (https://aastatus.net/39082) about this. Now when you PPP-connect, the connect response message that you see says "BBEUnnnnnnnn 1234567/123456" which is downstream/upstream speed, whereas before it was just the BBEU id. The speed units are sync rates presumably, will have to check that.
It mentioned that the Firebrick will have some new XML attribute which just means work out the upstream speed limits as x% of the upstream rate value seen in the connect response message. I’m assuming that I would perhaps have to rate conversion from sync rate to real IP rate and then further downrate that to taste, say 95% as I do currently. The first multiplication factor might either be 1 or the protocol efficiency factor (0.884434 for my situation with 32 bytes worth of overhead, ATM, and 1500 bytes PDUs), this multiplied by the 95% or whatever. But I’m assuming you just supply the combined multiplication factor, not the speed itself, and as the speed varies after any resync so the firebrick varies the rate limiting which will be superb.
This has been a long long time coming, but needed some imagination and changes to multiple systems. It’s going to really improve performance and reliability.
Do you know where there is any documentation for the XML for the new Firebrick software release?
I have already had to adjust some software I wrote in the light of the connect response change so as to strip off the down/up rates and leave just the BBEU id. The BBEU id is used to check whether or not the cables are plugged into the right lines, which it does by comparing the reality observed against a database which contains the correct expected values.
I think I will wait until it’s a factory full release before I change things. I will have to change some more software of mine. I have an iOS Shortcuts program for the iPad which does surgery on a Fredrick XML config file, generating a snippet containing <ppp /> elements that contain the the correct upstream rate limiter speeds, and inserting that snippet into the middle of the XML config before uploading the customised result. Currently this program gets the true current upstream rate by querying the modems and converting sync rates into IP PDU rates. It will need to be simplified to add in the new speed multipliers, whatever they are, and forget about the current "speed=" attributes setting absolute speeds. It can forget about querying modems and the only dynamic variable will be what I term the "modem loading factor (MLF)" which is the down-rating to taste, currently 95% and I’m assuming that will also need to be multiplied by the protocol efficiency (=IP PDU rate/sync rate) and that product of the two values put into some new attribute.
-
I always assumed PPP must have a data rate to avoid trying to push more data down the link than is possible and was always puzzled why no routers seem to reveal this useful piece of information, let alone use it for setting QoS. Although how does that work if you connect to PPP over a network with unknown capacity?
Then again, they also have the DSL sync rates and don't bother using those either.
-
Do you know where there is any documentation for the XML for the new Firebrick software release?
Documentation https://download.firebrick.co.uk/fbsoftware/2901/V1.55.124/FB2900/V1.55.124-2901-FB2900-Hamman+.html (https://download.firebrick.co.uk/fbsoftware/2901/V1.55.124/FB2900/V1.55.124-2901-FB2900-Hamman+.html)
XML https://download.firebrick.co.uk/fbsoftware/2901/V1.55.124/FB2900/V1.55.124-2901-FB2900-Hamman+.xsd (https://download.firebrick.co.uk/fbsoftware/2901/V1.55.124/FB2900/V1.55.124-2901-FB2900-Hamman+.xsd)
-
I can see something called "auto-percent" defined as a ubyte in units of percent. Do we think that’s the new attribute we need?
And the next question is do I set it to 95% my choice of PPP IP modem loading factor (where 100% means flat out, the max the modem can take), or to 95 * 0.884434 including the protocol efficiency factor which is used to down-convert from sync rate to IP rate in my situation?
-
Do we think that’s the new attribute we need?
I think that's a fair bet judging by the description. I'll be setting mine to 98 as I currently just multiply the uplink sync rate reported by my modems and/or the A&A control pages by 0.98.
I'm waiting for it to make it to beta before testing, I previously used one of the earlier betas that had a number of voip changes and had no problems with it. I left my brick set to auto update to factory releases and manually flashed the beta, then a couple of weeks later it flashed to the factory version once it was released with no drama
-
Is the 98 the protocol efficiency for VDSL for you ?
I am using the protocol efficiency figure calculated for ADSL/ATM which for me is (1500+32 bytes ) / 48 = rounded up to 32 ATM cells, so 32 * (48 + 5) bytes per cell, so the protocol efficiency is 1500 / (32 * (48+5)) = 0.884434, which would correspond to flat out, so the new attribute max for me would be 88% and I normally run at 95% of flat out, which would be 84% = 88%* 0.95
-
Is the 98 the protocol efficiency for VDSL for you ?
I can't remember how I came by that number, it's been a few years now, it's entirely possible that I plucked it out of thin air
-
This is my assumption either flat out = auto-percent=100, or else auto-percent=protocol_efficiency, where protocol efficiency is the conversion factor from sync rate to IP rate. And I favour the latter, where auto-percent=88 for flat-out adsl or 97 for flat-out VDSL. If you were using 98% before, then I assume you need 98% * 0.88 for adsl or 98% * 0.97 for ADSL.
Make sense? Let me know what works for you. But I would be surprised if the same number is right for auto-percent because I’m assuming that different units are being used there between sync rate and IP PDU rate. But I could easily be wrong; AA could already be including an intelligent conversion factor like my ‘protocol efficiency’.
I will test it by starting with my favourite 88% (=assumed 100% * protocol efficiency for ADSL) and then increasing it a lot in order to watch and see if performance gets faster or else no change or even gets worse because I’m overdriving the links.
-
I emailed AA to request clarification and the reply was that my opinion earlier is correct. Their recommended value auto-percent=88 for flat-out ADSL.
Alternatively, I would think auto-percent=97 for flat-out FTTC/VDSL2.
From AA’s reply to my email:
88% I think was calculated as an optimum level to shape to account for overheads and reduce packet loss.
I was talking about ADSL/ATM specifically in that context.
So again if you’re using VDSL2 it would I think be correct to use auto-percent=97 for flat-out, or say (97 * 0.95) or a bit higher, as a first experiment, which will avoid the risks of packet loss.
-
Thanks @weaver, that's very helpful
-
I’ve just upgraded my FB2900 to v1.56 Jacoby and am now using the new auto-percent attribute, set at 84%, for ADSL2, assuming flat-out=88.4%, calculated from the protocol efficiency in terms of byte bloat, and then taking 95% of that, as I mentioned when we were talking about this before.
If I use 100% of 0.884, then the links are broken! It seems that that represents a massive overload. So 95% seems like a very good guess. The https://speedtest.aa.net.uk results are disappointing though, with upstream at around 1.0 Mbps :-( as opposed to an expected 1.25-1.6 Mbps (varies wildly, don’t know why, and that’s even with conducting multiple tests and taking only the max, to get rid of statistical noise or other users being active).
BTW I got lost trying to find the release notes for this latest factory FB2900 release (Jacoby). Can anyone help me find them?
-
BTW I got lost trying to find the release notes for this latest factory FB2900 release (Jacoby). Can anyone help me find them?
It appears that the website (https://www.firebrick.co.uk/support/software/?PRODUCT=2900) has not yet been updated, as the "Release notes from Factory release 1.54.101 to Factory release 1.55.111" are shown as current.
Factory release 1.55.111 was given the name "Hamman".
-
Ah, look under the "Factory and Beta" tab on the website (https://www.firebrick.co.uk/support/software/?PRODUCT=2900) page, where "2021-03-11 Latest Beta release 1.56.000 (Jacoby)" is listed.
Release notes from Factory release 1.55.111 to Beta release 1.56.000
* Fix a bug in the flash logging, which could cause logging to stop working after a while
CQM
* Graphs used to show a damping level even when damping not in use (i.e. l2tp damping not set), removed
DHCP
* Added "circuit" to the matching rules for DHCP server IP pool (circuit being Agent Info option 82 circuit sub option 1)
ETUN
* Add tx/rx packet stats
IPsec
* Additional logging and status information for roaming pools
* Add manually triggerable IKE clearing
L2TP
* Issue with DOS limit on outgoing L2TP fixed
PPPoE
* New option to pick up speed from connect message to set egress rate on PPP (ideal for bonding)
VoIP
* Additional debug
Web control pages
* Setup wizard bug when IPv6 defined
-
Ah, thank you. Does this mean that 1.56 Jacoby is now a factory release, not a beta? Because I told myself I would not risk betas.
I’d be interested to know what upstream speeds anyone else is getting before and after changing to auto-percent.
-
Ah, thank you. Does this mean that 1.56 Jacoby is now a factory release, not a beta?
Your guess is as good as mine. Perhaps the page has not yet been updated? ::)
-
My FB2900 is set to not update its software automatically; it requires a manual download and update. It tells you in its status webpage when there is a new release available. I seem to remember something about allowing beta releases to be downloaded, or allowing only full releases. I don’t know about the ‘new release available’ displayed in the UI, whether or not that shows you beta releases. Or alternatively maybe the allow-beta thing is only concerned with automatic upgrades. When will it prompt you about availability of a beta, does anyone know?
I do want to know whether or not I’m downloading a beta, as I am definitely not a risky early adopter.
Btw, as an experiment, I changed my ‘modem load factor’ (ie. auto_percent = modem_load_factor * protocol_efficiency; protocol_efficiency=0.8844 for my ADSL2) from 95% to 98%, and the system wasn’t happy at all. So (95% * protocol_efficiency) seems to be a really good choice for the auto-percent attribute, pretty close to the limit. I’m not sure, would have to check again, but I wrote down a note for myself that A&A’s protocol_efficiency value is 87.9%. Guesswork: The difference might be to do with their assumptions about typical PDU size, based as I do on the MTU, and they might be using an assumed PDU size = MTU = 1492 in their calculations rather than 1500 as do I. AA’s figure might be more realistic, for whatever reason, and mine might be optimistic.
-
Guesswork: The difference might be to do with their assumptions about typical PDU size, based as I do on the MTU, and they might be using an assumed PDU size = MTU = 1492 in their calculations rather than 1500 as do I. AA’s figure might be more realistic, for whatever reason, and mine might be optimistic.
You may well be right but, given that A&A are strong advocates of 1500 as the MTU, I am a little surprised that they appear to use the smaller value.
-
If they use the smaller value, it could be the precautionary principle; it does no harm if a rate is a little bit under the critical value but things collapse if you push a link too hard.
-
Ah, yes. A good point and, probably, the fundamental reason.
-
This got me curious, checked the pfSense logs and both my lines show:
ppp[20345]: [wan] Bundle: Status update: up 1 link, total bandwidth 64000 bps
This is especially interesting as Plusnet portal says:
Current Line Speed (Download): 68.8Mbps
I measured actual throughput of 66Mbit a few days ago with the modem straight into my laptop and Windows PPPoE, that strangely reduced to 64Mbit through pfSense (which starts to get curious).
Am I clutching at straws or is there some correlation here? (although I seem to only be pulling 61Mbit today from each line)
-
I tried the assumption that 68.8 is the sync rate and so roughly 68.8 * 0.9669 = 66.5 Mbps would be the IP PDU rate, so could one system be talking in different terms/units? Quoted figures will be reduced a lot further still if it’s TCP SDU rate, ie. payload that is being quoted. The figure of 0.9669 was something I wrote down as the overhead for VDSL2 with a normal level of error correction, and some users will have a much worse figure of 0.91 if using G.INP=high. I believe I got these numbers from Kitz, so I’m hoping I remembered correctly.
-
Does this mean that 1.56 Jacoby is now a factory release, not a beta?
Jacoby is still in beta. If your brick is set to auto update to factory releases only it will still prompt you that a beta release is available and does label it as such and you have to manually initiate the update which is what you must have done.
The new setting itself looks to work as expected and sets the egress rate to the same number that I would have manually calculated
-
Ah. I was fooled by the prompt as I don’t do automatic updates, so I have indeed unwittingly installed a beta, it seems. >:(
-
I believe it is possible to roll-back to the previous image . . . assuming I am remembering correctly what I read in the FB2900 documentation.
-
I think so indeed and I remember that it will roll back automatically under certain conditions if it crashes. I’m going to stick with it now though because I’ve just changed a lot of iPad software that I wrote in iOS Shortcuts so that it works with the new auto-percent feature and no longer queries modems to get sync rates and derive speed="xxx" attribute values inserted into the config. I could roll such Shortcuts changes back but it’s a minor hassle. One example is a config upload routine that adapts the ppp elements in the config XML according to my config preferences before upload. I wrote the upload adaptation routine for the new auto-percent thing a while ago but couldn’t test it; luckily it worked first time when I switched it in. Now I actually don’t need the adaptation feature at all but kept it in in a new form because I can use it to experiment with tweaking the modem load factors (= fraction of max flat-out upstream speed) that I use to set the auto-percent values without editing the config XML by hand.
-
I see. That all make good sense. :)
-
I’ve been using the new “auto-percent” attribute feature for a while now and it works very nicely. Zero hassle from now on; no more having to keep an eye out for changes in links’ upstream sync rates after which I then run my program that queries the modems, recalculates the upstream rates, inserts them in the <ppp /> elements in an XML config file template and uploads the derived config to the Firebrick. And the beta o/s has not given any problems. I will definitely upgrade to the next ‘factory’ release asap though, just to be on the safe side.
-
I’ve been using the new “auto-percent” attribute feature for a while now and it works very nicely. Zero hassle from now on; no more having to keep an eye out for changes in links’ upstream sync rates after which I then run my program that queries the modems, recalculates the upstream rates, inserts them in the <ppp /> elements in an XML config file template and uploads the derived config to the Firebrick. And the beta o/s has not given any problems.
That's good to know and anything which minimises what you have to do is a definite bonus.
I will definitely upgrade to the next ‘factory’ release asap though, just to be on the safe side.
If I have fully understood the Firebrick (or, more specifically, the Adrian Kennard) concept of alpha, beta and factory releases, the next factory release will be the beta version you have currently installed just renamed . . . sed 's/beta/factory/'
Assuming, of course, that there are no recently discovered bugs that require squishing.
-
And they might release a later beta I suppose. I wonder if I should simply do nothing and ignore later betas, or install them. If they stick to my understanding of ‘beta’, which is that a beta release is feature-complete and contains no known bugs, then later betas will contain only bug fixes and no other changes.
-
Didn't I read that the Firebrick have firmware "slots" so rolling back is as simply as a reboot? Seems not much to lose from updating.
-
It does have firmware slots. I believe that if a newly installed o/s crashes under certain conditions, then it reboots and switches back to the earlier version. The boot loader and associated systems seem rather well-designed.
-
b*cat nods in agreement. That is what I recall reading, some time ago.
-
I read one has 10 slots which seems seriously overkill but I guess if you have the NAND space then why not. That may have been the datacentre-grade model though.
I'm really surprised they haven't released new models by now though as 750Mbits/sec seems a bit low now Gigabit is spreading across the country.
-
I'm really surprised they haven't released new models by now though as 750Mbits/sec seems a bit low now Gigabit is spreading across the country.
That is something to which CarlT occasionally alludes . . . end-users' CPEs incapable of switching Gigabit Ethernet connections.
-
> I'm really surprised they haven't released new models by now though as 750Mbits/sec seems a bit low now
I agree. A faster processor or maybe more accelerator hardware might be needed. The FB2900 is the upgraded version of the FB2700 which can route around 350Mbps. I don’t know what the speed of the FB2900’s big brother the FB6000 series is, but that’s horribly expensive at something like £12k [!] and so that’s only for ISPs and corporates, and I don’t think even that is fast enough for 10Gbps flows.
I really wish they would add QoS support into the Bricks. They get away with not having it by having a heuristic which is that small packets have priority, so eg TCP ACKs, VoIP or streaming video packets, and DNS lookups get prioritised.
-
Its the FB2900 that says approximately 750Mbits on the homepage.
Even the FB6000 range only have 2xGigabit ports, presumably why AAISP only allow L2TP up to 100Mbit as you don't get a whole lot of customers out of each box.