Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: ISP BT Trials New Way of Checking Customer Broadband Line Speeds  (Read 1028 times)

Bowdon

  • Content Team
  • Kitizen
  • *
  • Posts: 1918

https://www.ispreview.co.uk/index.php/2019/01/isp-bt-trials-new-way-of-checking-customer-broadband-line-speeds.html

Quote
Consumer UK ISP BT has begun to trial a new way of checking customer broadband speeds, which sounds like it will require special code (firmware) to be added on to the end-users router (Smart Hub etc.) in order to test and monitor their line. This could help them to comply with Ofcom’s forthcoming changes.

Last year the telecoms regulator, Ofcom, announced a new Voluntary Code of Practice for Broadband Speeds (CoP), which among other things would provide UK consumers with more information about their estimated line performance and make it easier to exit your contract if related problems cannot be resolved within 30 days.

Crucially this CoP, which is due to be introduced from 1st March 2019, added a requirement for ISPs to deliver normally available speed estimates based on peak time speeds. Effectively this meant that member ISPs might have to test the actual speeds of a statistically meaningful panel of customers on each broadband package during peak time, which at the time was neither cheap nor easy to do.

Ordinary web-based speedtests are simply not up to the task because they are too unreliable and easily influenced by issues such as local network congestion, limitations of end-user devices and slow WiFi etc. The alternative of using lots of custom routers from SamKnows (Ofcom and Virgin Media use this for their speed reports) has also been rejected by other ISPs as being too expensive or tedious to implement.

Alternatively ISPs have also been exploring different methods, such as the possibility of extracting more performance data from wholesale suppliers (there are some big limitations to what can be done here) or building special connection monitoring code into the end-user’s router (e.g. ASSIA’s TruSpeed). The latter would be difficult to develop if starting from scratch.
Logged
BT Infinity 2 - Smart Hub 6 - ECI Cab

ejs

  • Kitizen
  • ****
  • Posts: 1959
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #1 on: January 07, 2019, 07:06:02 PM »

I wouldn't have thought it would be particularly difficult to develop some code within a firmware to do that. Not for a firmware developer at least.
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 32077
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #2 on: January 07, 2019, 11:45:00 PM »

Someone kindly gave a link to the OFCOM file which gives more information about the tests

https://www.ofcom.org.uk/__data/assets/pdf_file/0025/111697/annex-5-testing-principles.pdf

Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 7338
  • Retd sw dev; A&A; 4 × 7km ADSL2; IPv6; Firebrick
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #3 on: January 19, 2019, 01:52:34 AM »

That document - stupid using TCP, as then the results are affected by the vagaries of different TCP implementations. Software design vs network conditions has nothing to do with fair assessment of link capacity or network congestion. They should be using a simple UDP-based test application with a fixed protocol which is designed specifically for highly accurate speed testing and which carefully maxes the link out in one, then the other then both directions. Perhaps something like iperf.
Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1121
  • Next generation network design and deployment
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #4 on: January 19, 2019, 11:01:59 AM »

TCP is most representative of user experience. The point is to reflect Quality of Experience.
Logged
-----
Deploying better networks, not just faster ones.

CarlT

  • Kitizen
  • ****
  • Posts: 1121
  • Next generation network design and deployment
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #5 on: January 19, 2019, 11:27:18 AM »

Should also mention that I immediately thought of a way to ensure test results from anything UDP-based flatter my network without any QoS I would have to put in my Key Facts Indicator: adjust queue length / buffer size to a value that's as small as feasible. Under congestion this would mean higher loss which would throttle TCP sessions but UDP would barely notice.

UDP testing in the presence of 5% packet loss will deliver 95% of peak speeds. TCP will be like wading through molasses.

EDIT: Also trying to test both directions simultaneously is a bad idea on everything bar DSL and point to point fibre.
Logged
-----
Deploying better networks, not just faster ones.

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 7338
  • Retd sw dev; A&A; 4 × 7km ADSL2; IPv6; Firebrick
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #6 on: January 19, 2019, 01:12:53 PM »

You are of course quite right about the ‘representative of user experience’ thing. But then you would be measuring the performance of a pair of tcp stacks of random type, and which one and also making assumptions about applications. You would not be measuring the link alone but a complex combination of: link + software components + app + use-case scenario. And to me that would be arbitrary and, while important and interesting, it would not be measuring the link +/- isp alone which was, I thought, the requirement.
Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1121
  • Next generation network design and deployment
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #7 on: January 19, 2019, 02:40:35 PM »

It's measuring performance from the router to the test server end to end which is intentional.

Measuring the access layer link performance is easy, it's already done by the kit either side, and meaningless to customer experience. It sets an upper bound only.

The TCP stacks aren't random. One side is controlled by whomever is administrating the tests, the other by the provider of the CPE, so it's a known environment.

The point of having the software running on the router is to remove the end user's local network, including software, from the equation. More reliable than having the end user send in speed tests carried out on their own machines.
« Last Edit: January 19, 2019, 02:43:20 PM by CarlT »
Logged
-----
Deploying better networks, not just faster ones.

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 7338
  • Retd sw dev; A&A; 4 × 7km ADSL2; IPv6; Firebrick
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #8 on: January 20, 2019, 06:53:00 AM »

No, I mean that in the customer’s experience, they could have a variety of TCP implementations at both ends. But the tests just pick one. Having a test that reflects customer experience is good but seems to me to be off-topic when they are supposed to be measuring the ISP, not software.
Logged

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 32077
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #9 on: January 20, 2019, 11:20:27 AM »

Its not measuring the software.   The 'software' runs a test between the CPE and OFCOMs servers.   Thereby testing the SP's routing when it comes to things like peak time congestion.   The ISP is not allowed to give any preferential QoS to OFCOM. 
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

CarlT

  • Kitizen
  • ****
  • Posts: 1121
  • Next generation network design and deployment
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #10 on: January 20, 2019, 04:22:06 PM »

So most kit uses whatever TCP implementation is built into the operating system it runs on. Anything reasonably modern shouldn't have any issues if implemented properly.

As you'll be aware the devices we all use most call libraries to open sockets and send their data to them, passing them to the kernel libraries to handle. To write data directly onto the wire requires raw sockets which in turn require applications to be running as a superuser.

Customer experience isn't a per-application thing as multiple applications are using bandwidth at once. It is about getting as close as possible to that experience in order to produce a single set of figures. The best way to do that and, indeed, only way is using generic TCP, UDP and IP stacks.

If they run the full range of tests the SamKnows hardware can there are a few different things covered. Most common denominator, eliminating user-side statistical noise is about the best that can be done.

At some point I'll find some material that was produced on this by a company that sells traffic monitoring and management software.
Logged
-----
Deploying better networks, not just faster ones.

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5742
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #11 on: January 21, 2019, 02:59:02 PM »

Given all the issues now with wifi, power plug networking etc. I think this is a logical step forward for isp performance diagnosis.  Even ethernet can be the cause of problems, e.g. on realtek cards on linux/bsd OS they dont play well when checksum offloading is enabled.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 7338
  • Retd sw dev; A&A; 4 × 7km ADSL2; IPv6; Firebrick
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #12 on: January 21, 2019, 06:05:57 PM »

Agreed that realistic whole-experience testing is very worthwhile. I was approaching the ‘link-alone’ thing as a different goal, also worth doing separately imho.

I agree that Realtek NICs and drivers can be horrible. I’ve had dreadful ones. Put a posh Intel NIC on a card into one old machine once to use instead of the crappy Realtek that was on the motherboard.
Logged

CarlT

  • Kitizen
  • ****
  • Posts: 1121
  • Next generation network design and deployment
Re: ISP BT Trials New Way of Checking Customer Broadband Line Speeds
« Reply #13 on: January 22, 2019, 11:17:21 AM »

The broadband link alone is easy to approximate and pointless.

On a shared media it's a simple calculation. If bandwidth available > (user rate limit - current usage) performance = rate limit. Else performance = (bandwidth available - rate limit + usage) + rate limit.

Say there's 500Mb free on my 24 downstream channels right now and I'm using a megabit. (500 - 402.75 + 1) = >=402.75
300Mb free (300 - 402.75 +1) + 402.75 = 301.

This can also be used to apply to hybrid media - use the point to point segment to produce your rate limit then run the calculation above - it basically states if there's zero congestion on the backhaul you should get your rate limit, whether that's at your own CPE or the DSLAM, else you are nominally assigned the bandwidth you are using + remaining bandwidth on the circuit.

On point to point throughout it's just deduct the overheads.

Without using software to measure the other option is to look at the QoS in use to share bandwidth then calculate each CPE's 'share' either per device or per IP flow. In the case of DOCSIS at least there's a minimum speed set as well as a maximum and queuing tends to be simple FIFO so you're at the mercy of the applications in use and how aggressively they retransmit, you can only guesstimate.

Doing anything else will 'pollute' the numbers as you're going to be using software to measure.
Logged
-----
Deploying better networks, not just faster ones.