Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
   Glossary   Glossary
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Author Topic: 2Wire firmware upgrade and Impulse Noise  (Read 8261 times)

waltergmw

  • Kitizen
  • ****
  • Posts: 2776
2Wire firmware upgrade and Impulse Noise
« on: December 23, 2011, 10:31:28 PM »

Gentlefolk,

Some of you will probably have observed that the BT Auto-update on 2700 HGV modems has distributed version 6.3.9.63-plus.tm

Amongst other things the diagnostics page has been changed to include Impulse Noise protection. The picture link illustrates the new format and also illustrates the transfer from ADSL1 to ADSL2 on one of the lousy lines on the wrong side of Cobblers Brook where we know there is substantial clicking probably from electric fences. You will see that the modem has detected the noise and suggests increasing the protection. Once we had negotiated the knowledge barrier of the first line support staff, some pertinent information has been obtained. Contrary to previous suggestions for noisy lines to be retained on ADSL1, there now seems to be a good reason for ensuring all lines are configured to ADSL2 as Impulse Noise Protection is only available on ADSL2 services.

http://www.scribd.com/doc/76394055

With ADSL2 configurations only, various INP levels can be selected; however these are dependent on capping your sync at varying levels.

160-288 - INP0 or INP8
160-8128 - INP0 or INP1
160-24384 - INP0 or INP1
288-576 - INP0 or INP8
576-1152 - INP0 or INP4
1152-2272 - INP0 or INP4
1472-3072 - INP0, INP4 or INP8
2272-4544 - INP0 or INP4
3328-6656 - INP0, INP2 or INP4
4864-9728 - INP0 or INP2
7168-14336 - INP0, INP1 or INP2

If the minimum sync is below the achievable level the connection fails.
Logged

waltergmw

  • Kitizen
  • ****
  • Posts: 2776
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #1 on: January 10, 2012, 02:17:45 PM »

Gentlefolk,

Just in case you haven't been poking around with a tone tracer, this is a typical result on a long line D side 100 pr cable in our "Horseyculture area" where there are masses of, sometime faulty, electric fences.
In this recording, and sorry for the traffic noise, you can quite clearly hear two fences racing each other.

http://audioboo.fm/boos/617750-cobblers-brook-impulse-noise

Kind regards,
Walter
Logged

roseway

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 43467
  • Penguins CAN fly
    • DSLstats
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #2 on: January 10, 2012, 02:26:33 PM »

Fascinating, thanks Walter. :)
Logged
  Eric

waltergmw

  • Kitizen
  • ****
  • Posts: 2776
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #3 on: January 12, 2012, 05:44:56 PM »

Gentlefolk,

I have some very intriguing data and I am perplexed !

Differences observed on 2Wire 2700HGV before and after Impulse Noise profile changed from 1152-2272 to
Rate Band Downstream: 1472-3072
Rate Band Upstream: 32-Uncapped
Interleave Level (INP) Downstream: High { 8 }
Interleave Level (INP) Upstream: On
Target Margin Downstream: 6dB
Target Margin Upstream: 6dB

This has resulted in a reasonable improvement in the modem's statistics BUT has actually reduced the throughput speed marginally.
The CRC eror rate is under 150 /hour so it's not retransmissions causing the drop.

I've extracted the figures from the two diagnostic pictures attached below.
Note that I happend to be accessing the modem remotely just before the profile change when I obviously lost communications.
We can probably assume that the actual noise conditions will not have varied very much between the two pictures.

Downstream Rate Cap changed from 1152 kbps to 3072 kbps (1152 should have been 2272)

Downstream Attenuation at 300 kHz changed from 75.7 dB to 74.3 dB

Uncancelled echo changed from -13.5 dB to -14.2 dB

VCXO Frequency Offset changed from +2.8 ppm to -7.4 ppm

Final Rx Gain remains at 34.5 dB

Excessive Impulse Noise remains at 0

Downstream Required Impulse Noise Protection changed from 9 to 3

Downstream Impulse Noise Protection changed from 4.74 to 8.00

Downstream Delay of Latency Path changed from 7.70 ms to 16.00 ms

Upstream Impulse Noise Protection changed from 2.00 to 2.01

Upstream Delay of Latency Path changed from 7.80 ms to 7.85 ms

Framing mode remains ATM

Downstream Rate increases from 1152 kbps to 1520 kbps

Downstream Max 1 increases from 1344 kbps to 1525 kbps

Downstream Max 2 & Max 3 increase from 1593 kbps to 1809 kbps

Downstream Margin 1 decreases from 7.4 dB to 6.5 dB

Downstream Margin 2 decreases from 7.2 dB to 6.5 dB

Downstream Attenuation  decreases from 73.5 dB to 72.0 dB

Downstream Power increases from 16.3 dBm to 16.5 dBm

Upstream Rate decreases from 525 kbps to 522 kbps

Upstream Margin 1 decreases from 6.3 dB to 6.1 dB

Upstream Margin 2 decreases from 6.5 dB to 6.3 dB

Upstream Attenuation  increases from 40.2 dB to 40.3 dB

Upstream Power increases from 12.4 dBm to 12.5 dBm

The bit loading diagram also shows quite a few increases in tones and bit numbers.

All comments are welcome.

The TBB speed tests yesterday are all after the profile change.

Kind regards,
Walter

[attachment deleted by admin]
Logged

asbokid

  • Kitizen
  • ****
  • Posts: 1286
    • Hacking the 2Wire
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #4 on: January 13, 2012, 02:26:13 AM »

Fascinating, thanks Walter. :)

It is very interesting, Walter

What would Impulse Noise look like on the QLN and Hlog graphs of the Huawei?

cheers, a
Logged

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #5 on: January 13, 2012, 07:27:14 AM »

Hi asbokid,

I'm not hijacking this thread of Walter's, but as you asked:-


What would Impulse Noise look like on the QLN and Hlog graphs of the Huawei?



For anyone who may be interested, I have started a FTTC thread here:-
http://forum.kitz.co.uk/index.php/topic,10526.0.html

I have located a document that describes impulse noise & lists and qualitatively compares various initiatives that have been suggested in the VDSL2 standard body to improve the protection of VDSL2 services against impulse noise:-

http://www.google.co.uk/url?sa=t&rct=j&q=inpulse%20noise%20protection%20vdsl2&source=web&cd=2&sqi=2&ved=0CC4QFjAB&url=http%3A%2F%2Fwww.ncc.org.in%2Fdownload.php%3Ff%3DNCC2008%2F2008_B2_3.pdf&ei=KtUPT9nqHZOA8gPTr5HHAw&usg=AFQjCNHzsrdfZ81IRjPXaGhb4TxhGOaW1Q

My graphs below show the regularly changing interleaving levels for my connection that are automatically applied by DLM.
Plusnet tell me that unlike ADSL connections, they have no control over interleaving levels for VDSL2 connections.

The blank space at the left hand side of some of the graphs are purely due to the fact that I wasn't harvesting those particular stats at that time.


It is such a long time ago that my connection performed at a more stable & higher speed (before I had full access to my stats), that I don't recall 100%, but I'm almost sure that ping times from speedtest.net were usually around 5ms.
Since I have been recording all my stats, it has been around 15ms when the interleaving depth (D:) was below 500 & 25ms when it was over 1600.


When Interleaving was below 500,

These were the other DS stats:-

INP:      3.00
PER:      3.38
delay:      8.00
OR:      56.65


& when over 1600:-

INP:      8.50
PER:      3.59
delay:      16.00
OR:      53.41

I have only reported DS values as Interleaving is ALWAYS off for my connection (a D: value of 1)

Paul.


EDIT: My QLN & HLOG graphs always look pretty identical to the graphs shown below, regardless of INP or Interleaving levels.

[attachment deleted by admin]
« Last Edit: January 13, 2012, 08:12:35 AM by Bald_Eagle1 »
Logged

asbokid

  • Kitizen
  • ****
  • Posts: 1286
    • Hacking the 2Wire
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #6 on: January 14, 2012, 01:11:55 AM »


I have located a document that describes impulse noise & lists and qualitatively compares various initiatives that have been suggested in the VDSL2 standard body to improve the protection of VDSL2 services against impulse noise:-

http://www.google.co.uk/url?sa=t&rct=j&q=inpulse%20noise%20protection%20vdsl2&source=web&cd=2&sqi=2&ved=0CC4QFjAB&url=http%3A%2F%2Fwww.ncc.org.in%2Fdownload.php%3Ff%3DNCC2008%2F2008_B2_3.pdf&ei=KtUPT9nqHZOA8gPTr5HHAw&usg=AFQjCNHzsrdfZ81IRjPXaGhb4TxhGOaW1Q


The paper is very interesting. Thanks for posting it.  There's another paper on Impulse Noise Protection from 2005 written by a team at 2Wire [1] the manufacturers of the 2700 and 2701 BT Business Hub, and there's the well-cited PhD thesis of Nedko Nedev, on the subject of the "Analysis of the Impact of Impulse Noise in Digital Subscriber Line Systems". [2]

It's not clear from any of the modem manufacturers what method of impulse noise protection is actually being used. And how is the impulse noise measured and detected by the CPE modem and/or DSLAM?

To be honest, I've listen to the audio tracks of Walter's videos and I can't hear the noise that Walter is reporting. Presumably it's a repetitive clicking sound?  Is it definitely caused by an electric fence, though?  Could it be from the ignition system of a vehicle, an all too common culprit?

Quote
My graphs below show the regularly changing interleaving levels for my connection that are automatically applied by DLM.
Plusnet tell me that unlike ADSL connections, they have no control over interleaving levels for VDSL2 connections.

The blank space at the left hand side of some of the graphs are purely due to the fact that I wasn't harvesting those particular stats at that time.

It is such a long time ago that my connection performed at a more stable & higher speed (before I had full access to my stats), that I don't recall 100%, but I'm almost sure that ping times from speedtest.net were usually around 5ms.

Since I have been recording all my stats, it has been around 15ms when the interleaving depth (D:) was below 500 & 25ms when it was over 1600.

When Interleaving was below 500,

These were the other DS stats:-

INP:      3.00
PER:      3.38
delay:        8.00
OR:      56.65

& when over 1600:-

INP:      8.50
PER:      3.59
delay:   16.00
OR:      53.41

I have only reported DS values as Interleaving is ALWAYS off for my connection (a D: value of 1)

By that you mean the interleaving is always turned off for the upstream?

Quote
EDIT: My QLN & HLOG graphs always look pretty identical to the graphs shown below, regardless of INP or Interleaving levels.

Here's my layman's take on things..

In ideal circumstances, the xdsl equipment would identify narrowband transient bursts of noise (which we would expect to see as spikes in a Quiet Line Noise (QLN) graph that plotted noise in dB against frequency). During the training phase, and periodically during showtime,  the bit allocation algorithm would ensure that the tones which are most affected by that noise were masked out, and left unused.  But that would rely on the sampling of Quiet Line Noise to be carried out over a suitably long period of time, and also frequently enough to capture the impulse noise, to identify its characteristics.  Noise sampling that was carried out, for example, for a duration of 500ms would probably miss the impulse noise from an electric fence if its HT discharge occurred only for a 50ms burst every 10 seconds. Similarly, if the sampling was done every 10ms, but the noise burst lasted only 500us, then the sampling would likely miss it. Also, as the authors of those academic papers all point out, the impulse noise must be isolated from background noise.   It would be positively harmful to the performance of the xdsl connection if the effect of the impulse noise was simply added to some average calculation for overall noise over the whole bandwidth.  For a repetitive impulse noise, part of the analysis would have to be done in the time domain. Frames or cells that were transmitted at time t, 2t, 3t, 4t, 5t, etc. and which were all found to be damaged would, if correctly identified, cause every affected frame at time n.t to be dropped. Therein introducing another host of problems. For a crude piece of equipment like an electric fence, it probably uses a very primitive oscillator circuit that is almost impossible to synchronise against. Operating in the wilderness of Kentish country, the discharge circuitry is likely susceptible to wide variations in time, due to fluctuations in outside temperature and humidity, etc.  Today the fence will fire its bolt of energy every 9.862 seconds, but tomorrow, with the sun out, the frequency could be every 10.216 seconds!  How can a CPE modem capture the characteristics of that, in order to counteract its effect?

At college, we only very briefly studied the different types of noise and how they can affect digital communications. It certainly wasn't my favourite topic! ??? Noise is surely a very difficult thing to model. Especially with something like impulse noise. By the time you've set up the kit to measure the noise, it's probably gone, at least until the next day, when Farmer Giles rises at the crack of dawn to fire up his antiquated three-phase milking machine!

cheers, a

[1] http://mrsrl.stanford.edu/~sedarat/papers/2005c_ICASSP_ImpulseNoiseProtection.pdf
[2] http://www.era.lib.ed.ac.uk/bitstream/1842/1366/1/Nedev.pdf
« Last Edit: January 14, 2012, 01:21:44 AM by asbokid »
Logged

Bald_Eagle1

  • Helpful
  • Kitizen
  • *
  • Posts: 2721
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #7 on: January 14, 2012, 08:44:36 AM »


By that you mean the interleaving is always turned off for the upstream?


For my connection, since I have been logging Interleaving levels, yes.

There was a period back in July/August when Plusnet's logs reported that Interleaving was HIGH for my upstream.
I do not have any of my own collected data from that period, other than speed test results.

At that time my US was capped at Plusnet's standard 2Mb level. Usually US speed tests reported 1.67Mb, but for a while it reduced to 0.67Mb. It was only at that time that US Interleaving was reported as HIGH by Plusnet.

Since various "repairs" have been carried out on my D-side cabling, my US speeds have not been affected by Interleaving.
The cap on my US is now 10Mb (arranged by Plusnet as a trial), but I only achieve around 4.8 to 5 Mb throughput.
Attenuation on my US appears quite high at:-
8.3dB    - U0
52.8dB  - U1
64.1dB  - U2
N/A      - U3


@Walter,

I harvest some of my connection's stats every minute 24/7 & some only "as & when" - on an ad-hoc basis.

From my 24/7 logs, I notice that Interleaving levels for my VDSL2 connection only ever change when the connection re-syncs.

I have not been logging INP levels 24/7, but any changes do "appear" to coincide with re-syncs & changes in Interleaving levels.
Neither have I been logging QLN levels 24/7.

I had assumed that re-syncs at lower speeds were initiated by some of the incredibly high & sudden bursts of errors on my connection (sometimes 2 Million or more CRC / RSUNCorr errors within a one minute sample period).

It may be that the error bursts (presumably caused by increased noise levels) suggest a need to increase INP levels, which in turn suggests a need to increase Interleaving levels, the only way to do this being during a re-sync.

Following a re-sync, I have noticed that sometimes error counts start to mount almost immediately, & sometimes it may be a couple of hours or so before they start to mount significantly.

It appears that when INP & Interleaving levels are higher, the error counts are reduced.

Now that you can monitor INP levels etc. you may see a pattern developing over a prolonged period.
Are you able to capture any useful data with RouterStats or similar tools for the connection(s) you are monitoring?

My connection appears to show higher error counts during the period between 22nd December & 8th January.
(Christmas lights perhaps?)

Unfortunately we suffered a few power cuts during that period which may have also had some effect on matters.

FWIW, My connection has now been up for around 130 hours & error counts are vastly reduced in both frequency & level.

In your case, I presume the electric fence noise is fairly constant 24/7. Is that the case?
From your audio sample, I could hear a constant "buzzing" sound, with some "clicking".
Are you saying the "buzzing" is normal & expected, & that it is the "clicking" that is causing the issue?

Also, is the issue purely low speeds, or is it also frequent re-syncs (not initiated by the user).

I try to stay connected 24/7.
The vast majority of my connection's re-syncs occur "on the fly", are completed within around 16 seconds, & are not reported in Plusnet's logs as the PPP session & dynamically allocated IP address is maintained.
Until I was able to collect my stats & generate relevant graphs, Plusnet believed my connection to be very stable.

For curiosity, who is the ISP that provides the service for the particular connection you mention, & do they consider it to be stable?


Extract from BT's SIN 498 doucument.
It refers to VDSL2 connections, but may also be relevant for ADSL2?:-

2.1.6 Downstream shaping

The CP is expected to shape the downstream traffic to match the actual VDSL2 line
rate in order to avoid excessive traffic loss.

CPs should be aware that the mechanism for reporting the downstream and upstream
line rates relies on a line re-train causing the CP, or the CPE, to initiate a new PPP
session or a new DHCP request. The success of this is down to the CP's choice of
timers used around PPP / DHCP handling.

If the PPP/DHCP survives a re-train, then the CP will be unaware of any change in
the line rate and will not be able to shape appropriately.

The line re-train time for VDSL2 can be anywhere between 10 and 60 seconds, with
typical values in the 20-30 second range.

As DHCP typically uses lease timeouts in the order of days rather than seconds, CPs
intending to use DHCP are advised to consider the impact of downstream line rate
changes on their service and any strategies they could adopt if they wish to shape
downstream traffic.


Paul.

P.S. I'm still trying not to hijack your thread, with the thought that some of my own connection's details may assist some way in comparing matters (even though mine is atually a VDSL2 connection).

EDIT (1):
Walter, are you suggesting that 2wire 2700 HGV users now have the ability to adjust INP levels themselves, the modem does it automatically, or just that the updated firmware is now reporting INP levels, requiring ISP manual intervention to manually adjust them in a similar manner to how they can adjust SNRM levels?


EDIT (2):
Some of us have noticed that the TBB speed tests appear to occasionally under-report throughput speeds.
Apparently it is something to do with them being HTTP speed tests?
We have found the speedtest.net speed tests to be generally more consistent & reflective of true throughput speeds
« Last Edit: January 14, 2012, 09:01:32 AM by Bald_Eagle1 »
Logged

waltergmw

  • Kitizen
  • ****
  • Posts: 2776
Re: 2Wire firmware upgrade and Impulse Noise
« Reply #8 on: January 14, 2012, 10:05:04 AM »

Hi Paul,

Firstly there have been overnight developments and a rate cap has been removed. This has resulted in short-term disaster !
More on this subject later when the line will hopefully be much less noisy.

Quote
Are you able to capture any useful data with RouterStats or similar tools for the connection(s) you are monitoring?
I have the capability but my laptop is being deployed elsewhere at present.

Quote
In your case, I presume the electric fence noise is fairly constant 24/7. Is that the case?
Yes, but it does vary with the weather.

Quote
Are you saying the "buzzing" is normal & expected, & that it is the "clicking" that is causing the issue?
Buzzing (Probably @ 50 hZ) might well be normal for unshielded twisted pair cable but could well be highly damaging to throughput.
However we must not forget that a tone tracer is a bit of a blunt weapon and is no substitute for a spectrum analyser.
The tracer has a "Volume control" which can amplify all sorts of noises, some even just mechanical contact.

Quote
For curiosity, who is the ISP that provides the service for the particular connection you mention, & do they consider it to be stable?
It is your favoured supplier PlusNet and we all agree that we're between a rock and a hard place with our glorious last mile supplier's usual antics.
Just to whet your appetite the following picture is the situation I'm dealing with this morning !

However Nil Desperandum !

Kind regards,
Walter

EDITS

Quote
Walter, are you suggesting that 2wire 2700 HGV users now have the ability to adjust INP levels themselves, the modem does it automatically, or just that the updated firmware is now reporting INP levels, requiring ISP manual intervention to manually adjust them in a similar manner to how they can adjust SNRM levels?

The new firmware does indeed report more detail. There are rather limited INP "profiles" available, as recorded at the beginning of this thread, which must be applied by the ISP.

Quote
Some of us have noticed that the TBB speed tests appear to occasionally under-report throughput speeds. etc. ....

I am aware that TBB produces slower figures than Speedtest.net but both are not as slow as the BT speed test.
This topic is an old chestnut but I don't want to restart that conversation here.
Suffice it to say that all these things are statistical sampling tools which bear little relation to real-life transactions. Lies, damned lies & ....
They are however useful if the results are used for comparison over extended periods of time.

As a typical example you'll see that BT make you agree to having no other activity on your line.
I regularly run BT tests whilst monitoring the activity remotely and it seems to have no effect whatever on the results when compared to running the same test only locally.
Furthermore, I believe many speed tests use priority packets for their data thus accentuating the positives. You might try chatting to Seb @ TBB.

K R
Walter

[attachment deleted by admin]
« Last Edit: January 14, 2012, 10:27:58 AM by waltergmw »
Logged