Kitz Forum
Internet => General Internet => Topic started by: Weaver on August 11, 2015, 10:12:43 PM
-
In the days of dial-up modems, some ISPs, Demon being an outstanding example, offered all the fancy features that the PPP protocol suite provides, including header compression for IP and TCP headers to more or less make them go away, very powerful data compression too.
* I wonder if any of this goodness has survived into the modern era?
* Do any ISPs have kit powerful enough to do the sexy things we used to enjoy?
* Would this be constrained by any need to remain within the boundaries of what BT can understand?
IPv6 header compression - yum, yes please. And TCP header compression, still relevant.
Obviously the ISP is hardly going to be motivated if no CPE is available that can handle this yummy goodness at the user's end.
I noticed that our cheerful Scot from BT - speaking at a recent UKNOF iirc, his name has just gone - mentioned how they would like to get rid of PPP and do IPoE.
Who ordered that? Yet more bloat from the arguably useless Ethernet headers, but I can see the attraction for multiplexing and getting strange functions via VLANs that caould make ISPs money from non-Internet services (and start to get up to all kinds of disturbing tricks too which may be hard for us the users to monitor and control). And more chaos and wasted money in the transition too.
-
I very much like your curiosity! ;D
Unfortunately on this occasion I cannot be of much assistance, other than to say I will be watching this thread with a keen interest ;)
-
I have failed in my attempts to google this subject, its the time/era thing that Google doesn't seem to be capable of understanding.
-
the problem is these days compression at the network level now has more limited benefits, because things like gzip compression on http pages e.g. is now the norm.
As for your curiousity tho I can give you some stats in regards to network level compression the next time I enable my VPN to watch a game, it does get reasonably impressive compression levels.
However I expect if isp's were to do this network wide there would be massive cpu usage across the board.
-
> massive cpu usage
and a lot of totally incompressible data too. Email and html aren't significant in term of the fraction of bandwidth they represent out of the total.
But crazy CPUs should be able to do things such as header compression, Moore's law and multicore can take care of the stresses placed upon these nodes surely.
I'd be interested to know how important very short messages are. VoIP is short? DNS queries (not enough of them). Very short IP packets over IPv6 might be candidates for a big percentage decrease in size? I'm not sure.
What's a high rate, short packet protocol?
-
Those wouldnt be worth bothering with, dns lookups slow you down due to the high exchanges of packets. Compression would do very little if anything to speed up that process.
So a 10ms 500kbit line will do dns lookups faster than a 100ms gigabit line. With DNS bandwidth isnt the issue, latency is.
-
@Chrysalis - agreed. :-)
-
Only just seen this. Please sir I think I may know why. :waves hand in the air:
First a wee bit of a background into compression:
Compression is primarily designed to make file sizes smaller. There are various compression algorithms which are effectively judged by how much they can reduce the file by. The most 'efficient' algorithms are not necessarily the quickest to parse. The larger the file size the longer it takes to scan through looking for duplicates. Compression is processor hungry and it takes time to parse a file. Certain 'fast' algorithms such as CBR (audio) and DCT (jpg) wont do the job for data because they are lossy. Network data compression has to be lossless and it can be slow to parse. Grab any decent size file and see how long it takes to zip/unzip it even on a fast PC.
The most common algorithm used network data is Lempel Ziv Stac (LZS). My dissertation was in compression and although I've forgotten most of the in-depth stuff now, I've certainly not forgotten how its full name can be difficult to say. Seriously try pronouncing it several times - I'm not alone in this - and in some circles it was shortened to "Stac".
As mentioned Lempel Ziv Stac is heavy on processing for the parse, the larger the file the longer it takes to go through the loop and theres many, many loops of them inside the algorithm, trying to find duplicates. Also bear in mind that you cant compare a pre-processed compressed file stored on a server for download to having to it continually on the fly for real time data throughput. Ok that out of the way... the reasons why its not used.
-----
Packet size: Dial up = 576 bytes. DSL = 1500 bytes. So header is comparatively small compared to useful data for DSL packets. Therefore not as large of a saving to be made.
Processor intensive: Already covered above, the faster the speed the more data having to be compressed on the fly.
Computing power @ home: With dial up the compression tool sat on the local machine as the modem was inside the PC. With DSL & a NAT network its the router that would be having to do the processing. Your average DSL routers aren't really equipped to handle this sort of high power processing. They already juggle acting as multiple 56k modems (DMT), then stuff like FEC, Interleaving and now G.INP.
Lag: The processing causes lag - you may not notice say 1ms on 56k, but as data bandwidth increases, the greater the delay.
Computing power at the ISP: In days of 56k, the ISP customers were in the 100's or 1000's. The ISPs didn't have that many EU's hooked up to one of their routers, but to cope with compression, they needed an additional co-processor specifically to do the compression/decompression. Today's ISPs have 100's of thousand of EUs - some millions. There is no way that the ISP gateway routers could handle processing 10's of GBs of data. I cant remember now but I seem to recall even with co-processing is (was) limited to something like 8Mbps before it caused any noticeable slow downs. This is 8Mbps for all their customers so it probably worked well if you had 1000 x 56kbps users.
Back in the days of dial up ekeing out a few extra kbps made a big difference. Similar to the early days of DSL how we would tweak MTU to get a few extra kbps, but once the speeds get faster no-one seems to bother MTU tweaking unless its impacting something other than speed.
PPP header compression was ditched with DSL as just not worth it for very little gain. Its not an ISP specific thing - its a worldwide industry thing.
I have failed in my attempts to google this subject, its the time/era thing that Google doesn't seem to be capable of understanding.
For the reasons I mentioned above, try searching for 'Stac' as it pronounced by those that actually used the technology. Unfortunately it really does seem to be one of those things whereby if you didn't already know, then there aren't any back links!
-
https://en.wikipedia.org/wiki/Stac_Electronics
-
Remember LZ/W too?
No I was very much if the opinion that IP packet payload (an L3 SDU) is not worth compressing, with the exception of TCP headers possibly, but TCP packets are often huge so the gains in TCP header compression are relatively insignificant. IP (+TCP) headers, esp. IPv6 headers would be the only thing I could possibly think of which could have any potential benefit in some circumstances.
(Dial-up Demon used to offer Microsoft something compression for IP payload, iirc.)
-
It would probably get even worse if you take the fixed size ATM cells (the A in PPPoA) used on ADSL into consideration. 53 byte ATM cell, consisting of a 5 byte header and 48 bytes of payload, or payload plus padding to make up the 48 bytes. If compressing a higher level header only saved 10 bytes, that could just end up as 10 extra bytes of padding in one ATM cell, not actually saving any bandwidth.
I think the biggest overhead on ADSL is the 5 byte ATM cell header in each 53 byte ATM cell. So the biggest gain would come from dealing with that.
-
@ejs - this argument works both ways of course. A saving of one byte could mean a saving of an entire unneeded ATM cell.
[I discuss this very point in the wikipedia section on PPPoE overheads. (Really bad article, still.)]
Of course losing ATM altogether would be a fine thing, reclaiming about 10% complete waste. (Well 5/48 overall, and the fixed 8 byte trailer per packet )
But failing ditching it altogether, you can’t compress ATM cell headers. (i) They're at a lower and inaccessible layer in the xDSL protocol stack. The PPP entities don't know about the existence of ATM. (ii) They have an error checking role, which you would lose. This is what a HEC error is. Bloating the ATM byte stream in this way makes any errors more detectable.
-
it may help pppoe actually now I think about it.
If the compression would allow pppoe to use 1500byte mtu packets (without jumbo frames) then it would be worth looking into.
-
@Chrysalis - that's a good point. I hadn't thought about that.
Yes, we just need to claw back 6+8 bytes, but reliably. Sometimes something, sometimes nothing is the usual order of the day, and the great length of the data won't help us because we can't exploit IP payload, as it might be random.
-
Yes, good point about the fixed size ATM cells working both ways.
I think with a Broadcom based DSLAM and CPE, there's something proprietary called Nitro which is apparently a "ATM header compression scheme". Sky might use it (http://www.ispreview.co.uk/index.php/2013/10/sky-broadband-upgrade-unbundled-uk-network-boost-performance.html). Not on BTWholesale ADSL equipment of course.
-
@ejs - thx very much for that tip about Nitro. This is indeed outside the scope of this thread, which is about PPP.
From what I read, it's a Broadcom-to-Broadcom non-standard feature. Very interesting. deserves and needs a separate thread, and so if anyone finds out some good stuff on it, please contribute to the thread below:
http://forum.kitz.co.uk/index.php/topic,15947.0.html
-
separate thread re Nitro at
http://forum.kitz.co.uk/index.php/topic,15947.0.html
-
here stats from 2 hour'ish session mostly streaming.
Statistics
TUN/TAP read bytes 134876428
TUN/TAP write bytes 3293776427
TCP/UDP read bytes 3458901056
TCP/UDP write bytes 313423933
Auth read bytes 3293777099
pre-compress bytes 4480330
post-compress bytes 4495858
pre-decompress bytes 9993975
post-decompress bytes 13070650
-
I hear what people said about processing of user data, L4 payload (L4 SDUs or L4 PDUs) by a PPP compressor on DSL. It would cost a lot in CPU cycles and the nodes in question do not have access to long enough buffers full of ingress data, not without introducing latency and guzzling ram per flow a setup that doesn't scale especially because of the ram. In any case that's what servers can and do sometimes do, eg http servers with gzip and a terrifying amount of ram. The access to buffer length thing was true for dialup modems. PCs and ISPs’ PPP servers as compared to dialup modems had more ram, dialup was slower so ingress queues were long enough so that they had access to more ingress data to consider so allowing compression a good longish context to work on in each buffer full.
However.
IP and TCP and TCP+IP header compression ought to be doable as far as I can see.
In fact, leaving aside PPP for a moment, in the case of the stupid PPPoEoA stack you could build a PPPoEoA-speaking compressor in a DSLAM that would knock out 24 - 36bytes of PPPoEoA crap completely. In some cases this would save a whole ATM cell, but not in the particular IP PDU 1500 case unfortunately.
Combining it with saving of part of an IP header or IP+TCP header then you would certainly save a cell with nearly 40 + 24 bytes saved for IPv4, and for IPv6 60+24 bytes which nearly saves you 2 cells (nearly at 96 bytes payload) so sometimes you would save two cells. So for some short packets you would halve the size of the data, for other sizes you would cut it down to one third the size on IPv6. Wouldn’t help applications much because they are often concerned about the turnaround time in a ping-pong/stop-and-wait protocol, or about one-way latency. But it would conserve bandwidth available to other flows where there is a substantial stream of short L3 packets. Perhaps that is too much of a niche.
Combined PPPoEoA + IP (esp IPv6) + TCP gives you an extra ~3% speed with IP 1500 byte PDUs and a massive saving (size cut down to 50%) on each TCP ack.
This kind of thing was found to be worthwhile before with dialup, and the opportunities available with IPv6, ATM and PPPoEoA are even greater.
Does make me wonder, anyway.
-
This seems to be a theme: vastly overcomplicating.
PPPoA isn't going to be an issue much longer as it's almost entirely gone from UK networks as indeed it has almost everywhere - IP everything, ATM is yesterday's or, more accurately, last century's, technology.
PPP shouldn't be an issue too much longer. 802.1ad allows for replacement. The drive is to reduce the workload on equipment and present a software defined, unified architecture, not start messing around with L3 / L4 proxying to mitigate extra overheads from encapsulation.
-
i actually think pppoa is ok, its pppoe thats horrible, and only one uk vdsl isp doesnt use it and its also used for g.fast
-
PPPoE is horrible.
-
PPPoA is, again, a product of its time. Made sense when DSLAMs were backhauled by ATM, however redundant once ATM is removed.
PPPoE I think got a bad reputation here from when ATM was a thing and PPPoEoA was possible. It had its place but both can be retired in favour of using 'regular' Ethernet mechanisms.
-
PPPoE is horrible.
Can my VDSL2 Annex B use IPoE with just configuration changes (at vmg1312-b10a / service provider) if it always worked on PPPoE? ( :baby: :fingers:)
-
It can, but it needs the ISP to implement it.
In the UK most ISP's use PPPoE.
TalkTalk use IPoE.
-
j0hn, is that TalkTalk ADSL on LLU, straight IP in ethernet frames inside RFC 2684 is side AAL5?
What about TT FFTC?
-
They use PPPoA on ADSL and IPoE on FTTC. TalkTalk Business wholesale customers use PPPoE.
-
It can, but it needs the ISP to implement it.
In the UK most ISP's use PPPoE.
TalkTalk use IPoE.
So 2 uk isps I guess.
Sky with DHCP and TT with IPoE.
-
DHCP just provides IP address, etc, need something to get the DHCP and other packets there and back?
Pretty sure Sky use IPoE with authentication via DHCP options? TT use DHCP but don't worry about requiring a username and password I believe, Openreach populate options fields to allow them to tell which customer should be connecting.
-
yeah you right did a bit of digging and is IPoE with DHCP auth. I just knew it wasnt PPPoE.