Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Weaver]

I was just thinking, my 2.5 Mbps downstream per line is roughly fifty times faster than my dialup in 2003. Fifty times! That’s incredible, such a phenomenal increase, yet still I’m actually moaning and wanting much faster speeds. I’m sure some of our number are on at least 50 Mbps downstream with FTTx and are moaning a little bit. So given that 50 Mbps is twenty times faster downstream than one of my lines, another incredible difference.

How did we get to this situation? Let’s disregard TV for the moment because in fact we don’t need TV to be delivered over the internet, it’s just that the on-demand thing is extremely attractive, but for the purposes of argument let’s put that aside for a moment. Why are we getting uploads and downloads that are so huge that the transfer time is a pain in the backside ? We managed somehow in the days of dialup and I manage somehow to do most things even with my 7.8 Mbps downstream + 1 Mbps upstream. One thing I sometimes can’t do at all is have both Janet and I myself watching streaming video at the same time; sometimes that works, sometimes not, it’s marginal. Consumes around 3 Mbps as measured by looking at clueless.aa.net.uk CQM graphs of live streaming sessions. But other than that we actually do manage to do most things, it’s just slow sometimes. Upstream is a real pain; doing backups is awful, can take 10-30 mins to back up an iPad to the Apple cloud.

How did files get so huge? Streaming video is a new thing, huge by its very nature, despite enormous compression still huge. In the early 1990s we just didn’t do streaming video, couldn’t it over dialup apart from very small windows with low x times y and poor quality. I used to occasionally watch the TV program Big Brother’s live streaming feed over dialup, in a really small window and with terrible quality due to truly vicious, massive compression. I say decent video was a new thing because it was a dead loss in the early 1990s so proper video counts almost as a new data type now.

The other thing that has changed is the arrival of multiple users per dwelling. Businesses were of course always multi-user, but in the 1990s a lot of domestic dialup internet connections were just into the one computer so one user only, not into a LAN with multiple active users. Now the requirement in the case of residential domestic connections is to accommodate multi-user households, with say two to four or even five users in some situations. That’s why G.993.2 / FTTC isn’t fast enough, but I don’t know how/why, given that 50 Mbps is twenty times faster downstream than one of my lines.

Is some of this because software developers and web developers have become lazy and have got used to thinking that huge data bloat is normal? My brethren software developers have been inflicted by madness with instances of bloat that is beyond comprehension. I recently saw a modem’s config file that was all in XML and had numbers in ASCII text. The XML tags were quite unnecessary; as an example, could have made each number into a byte or word and could just could have just packed them into a C struct. I’m aware of the problems associated with this but it’s worth dealing with them imo. There are vast improvements in speed associated with not being lazy as well: no parsing, no decimal ASCII conversion, massively reduced i/o time, all good. But anyway, it’s easy to see five or ten times bloat factors, maybe more. Could do like Microsoft did with the recent office XML ZIPped up document formats, but then you still have the processing time of compressing/decompressing and the large time wasted in XML parsing when reading files. Anyway, I think it’s complete madness and I would be ashamed about releasing software that is associated with huge data file bloat in the case of non-trivial files. Web development - don’t get me started. The convenience of being able to edit files with plain text editors because they are in ASCII is ridiculous: could just supply a text-to-binary io format converter program and deal with the convenience issue that way.

Anyway, in general, how did we get to where we are? What’s going to happen in the future with file sizes, data types and use-cases, and what will happen with possible demand for ever faster speeds?

[Alex Atkin UK]

Bloat is definitely a factor, especially adverts.  But websites in general were much more primitive back then.

For me the bigger problem is not being able to buy most things on physical media any more, plus wanting to trial TV shows and movies before committing any cash.  They've literally pushed us to using more bandwidth because its the ONLY way to obtain a lot of content.

[Weaver]

Remembering back to the 1990s images were not used on websites much, and background images were often absent compared with today. Today huge images and photographs are common, covering the whole window background or a large part of a window.

Web browsing can’t consume much bandwidth though, because it’s initiated by the user, not a continuous flow of data like movies or file downloads/uploads.

How many people who get FTTP will choose the higher speeds?

[meritez]

Dial up was efficient as it was symmetric.
I only have 330/50 for the upload speed.

[displaced]

There’s digital distribution of software that wasn’t so common then.  And 60GB for a PC or modern console game is pretty average.  I don’t even think this is laziness on the part of the programmer — it’s the sheer level of detail involved in modern games.  Of course, that doesn’t make a game *good*, but it is what it is.

I think the arrival of reliable always-on data connections essentially opened the floodgates.  It meant that digital distribution was possible (even on single-digit megabit lines), no matter how slow.  So once it became feasible, people wanted it to be faster whilst the sheer size of the data being distributed has grown

I don’t tend to look at formats such as XML and JSON as bloat.  The concept of maintaining human-readability (whilst preserving structure) in data is appealing to me. 

[Alex Atkin UK]

Dial up was efficient as it was symmetric.
I only have 330/50 for the upload speed.

Dialup wasn't symmetrical in the later years, it was "up to" 56K down, 33.6K up.

I don’t tend to look at formats such as XML and JSON as bloat.  The concept of maintaining human-readability (whilst preserving structure) in data is appealing to me. 

Especially as those things can be compressed, if you write your server-side application accordingly to buffer output.

[Weaver]

The human readability which is important can be achieved by having an encoder/decoder app which converts packed unfriendly data files into man-readable unicode text. That’s having your cake and eating it. Having worked on small machines with very slow processors thr requirement for high efficiency of data formats was always overwhelming, and human readability was a nice-to-have. You’re absolutely right that XML is highly compressible, which is the definition of bloated in fact. I like XML a lot, and I think there should be a high efficiency binary format that is a one-to-one mapping to XML as we know it. That would have to do better than current general compressors otherwise there’s little point.

I can’t see that the web is the real reason why people crave more speed. And TV over the internet gets more and more efficient all the time (quality of Netflix at 3 Mbps is an impressive achievement), so TV doesn’t seem to be making increased demands for speed, rather the reverse.

[Alex Atkin UK]

I can’t see that the web is the real reason why people crave more speed. And TV over the internet gets more and more efficient all the time (quality of Netflix at 3 Mbps is an impressive achievement), so TV doesn’t seem to be making increased demands for speed, rather the reverse.

Sadly not, personally I'd like to see it reach parity with Bluray/UHDBluray.  Instead we've gone backwards as Netflix reduced bitrates due to some bizarre EU request for home workers during the pandemic, which they haven't reversed (the UK got bundled in even though our ISPs mostly have plenty of capacity so didn't need it).  It might be impressive, but on a 55" a soft image with macro-blocking during motion is ugly.

On the flip side I just watched Ghostbusters Afterlife on UHD Bluray and my goodness that's as amazing picture.

It boggles my mind that as TV display technology is getting better and better, broadband is getting faster and faster, subscription prices go up each year and compression techniques are making the picture quality worse.

[Weaver]

I was watching Netflix on a large iPad, and was thinking about what can be done in very meagre bandwidth. You’re absolutely right: there should be user choice, and you should be able to get the effectively uncompressed video if you can afford the number of bytes. The bitrate I was getting was very welcome though as 3 Mbps is low enough that I can now just about get away with running two Netflix user sessions simultaneously. Now that I am down to only three lines not four after last summer, I have currently only about 7.8 Mbytes downstream and 1 Mbps upstream. The downstream sync rates are exceptionally good at the moment presumably due to the cold weather.

[Alex Atkin UK]

The Netflix situation is particularly stupid as it doesn't seem to impact 4K streams.  So I pay the premium for 4K but still get degraded quality from anything not available in 4K. :/

You'd think anyone paying for 4K which its base bitrate being MUCH higher than the top-end 1080p was before the capping, would at least get uncapped 1080p.

[grahamb]

So you're saying that if I watch a film or tv series that's labelled as Dolbyvision/Ultra 4k HD on the Netflix app on my 4K OLED tv, and I'm paying for the premium account, I'm not actually watching that film or tv series in 4k resolution?

Edit: Ha, no, I've just read your first sentence properly... 

[Alex Atkin UK]

Edit: Ha, no, I've just read your first sentence properly... 

I suck at coherent sentences, hopefully made it clearer now.

I'm actually not 100% certain 4K is full bitrate, but my eyes say it probably is as I haven't noticed any issues whereas the 1080p streams its really obvious, at least on some titles.

[craigski]

An interesting read:

Netflix https://netflixtechblog.com/optimized-shot-based-encodes-for-4k-now-streaming-47b516b10bbb

The last example, showing the new highest bitrate to be 1.8 Mbps, is for a 4K animation title episode which can be very efficiently encoded. It serves as an extreme example of content adaptive ladder optimization — it however should not to be interpreted as all animation titles landing on similar low bitrates.

[Alex Atkin UK]

An interesting read:

Netflix https://netflixtechblog.com/optimized-shot-based-encodes-for-4k-now-streaming-47b516b10bbb

There's no denying Netflix have developed some amazing technology in their back-end.  Now if only they applied the same theories to sound, some movies on Netflix sound absolutely terrible though their premium 4K content usually is excellent on picture and sound, so not sure where the discrepancy comes in.

There have been times where I have a movie on Bluray but decide to watch on Netflix instead, almost immediately getting up and putting the Bluray on because it sounds flat and lifeless.  Were not even talking "in comparison", it just sounded bad period.

[Weaver]

Will we just want bigger numbers, that being the only reason for shelling out for ever faster data rates ? Or will there be a genuine driver for higher data rate traffic in the future that will take us to 10G and then maybe past it some day? Ispreview had a piece about an alt net in   the Crewe and Nantwich area of Cheshire who offers 2G and iirc that’s symmetric but maybe I remember that wrongly.