Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Weaver]

Agreed. I suggested to AA today that we both keep an eye on the ripening, and give it time.

A while ago I improved the reporting in my HCD detection code so that it now outputs the vertical drop distance in dB from the midpoint of a chord to the graph below. It also indicates which of the two tested chords it’s referring to; that is either chord #1, the chord between tones 40-85, or chord #2, between tones 40-60. It now gives me a concrete badness figure and saves me having to squint and use a ruler.

[Alex Atkin UK]

Maybe OR should hire you to rewrite the ADSL DLM?    Or at least the fault detection code.

[Weaver]

I hear you Alex. Has anyone apart from me ever seen this anywhere? I never saw this before Jan/Feb 2020 (can’t remember when it was exactly).

Stupid question of mine, getting confused. Does the DLSAM/MSAN have any access to the SNRM-vs-tones data that I can see in my modem at this end?

[burakkucat]

Has anyone apart from me ever seen this anywhere?

No, not I.

Does the DLSAM/MSAN have any access to the SNRM-vs-tones data that I can see in my modem at this end?

The xTU-C and xTU-R do exchange data. Exactly "when, how and what", I know not. (Or if I did know, I've now forgotten.    )

[jelv]

I hear you Alex. Has anyone apart from me ever seen this anywhere? I never saw this before Jan/Feb 2020 (can’t remember when it was exactly).

No, not I.

Before Weaver posted about HCD, I wonder if you would have noticed? Now I guess, being aware, it's something you'll (maybe subconsciously) look out for.

[burakkucat]

Before Weaver posted about HCD, I wonder if you would have noticed?

A good question. My answer has to be probably not.

Now I guess, being aware, it's something you'll (maybe subconsciously) look out for.

Now with a typical VDSL2 circuit, rather like that of your own, it's not something that could be easily seen.

[Weaver]

Indeed, it would be hard to translate this concept into VDSL2 with its multiple d and u bands. I don’t know why my experience of it is limited to tones 40 - 85 or maybe just possible 40 - 100.

Have agreed with AA that we will keep an eye on it as the fault develops.

The HCD detection code reports:
    *❗ DSL link #2 defect detected: so-called ‘hollow curve disease’ in the downstream bit-loading. (-5.149 dB at tone 62 below chord  1: tones 40-85) ❗ Serious fault 🔺

I see two overall measures of how bad the fault has become. One is the above 5 dB, the drop below the chosen approx ‘midpoint’ of a certain chord.

The other measure compares what is with what used to be, as follows: (I’ve actually cheated and compared the afflicted line 2 with the health line 4) The total hollow curve disease peak-to hollow-depth is now 14dB; that is the amount that the SNRM of the former best downstream tone has gone down, forming the bottom of the ‘hollow curve’ of SNRM vs tones. The best tone was approx tone 44 at around 41dB SNRM; now the local minimum is somewhere around tone 62 at 27dB. So either the noise has gone up on those frequencies, or the susceptibility to noise has, or the signal strength has gone down, or the line attenuation has gone up on those tones.

[Weaver]

Yesterday, the downstream CRC rate and ES rate were both steadily rising, also being higher at night as might be expected. The SNRM downstream was a mere 1.8 dB rather than the target of 3dB. In order to cool the rate of errors, which had risen to roughly one CRC error ever two seconds, I forced a resynch. The period after the resynch is also visible at the end of the graph below:




Here is the usual HCD picture in its current state of ‘ripening’ :



       Summary of DSL links’ wellbeing and error counts
     ────────────────────────

* There are 3 modems in total. They are:     #1, #2 and #4
* The active, contactable modems are:        #1, #2 and #4
* The modems successfully queried are:       #1, #2 and #4

       ───────────────────────
       ***                                                                           ***
       ***     There is some SERIOUS BADNESS !          ***
       ***                      All is not well !   😦                       ***
       ***                                                                           ***
       ───────────────────────

--
* Sync rate: The following link has a really low downstream sync rate, below min:
   Link #2 downstream: current sync rate 2380 kbps is below minimum expected downstream rate (2800 kbps)  ❗Line #2 fault  🔺

--
*❗Link #2 defect detected: so-called ‘hollow curve disease’ in the downstream bit-loading. (-5.426 dB at tone 62 below chord  1: tones 40-85) ❗ Serious fault 🔺

--
* ES (less serious): The following links have a few CRC errors, at lower error rates, where the ES rate < 60 ES / hr (†):
   Link #2 downstream:        latest period: ES per hr:  13.02, mean time between errors: 276.5 s, collection duration: 553 s
   Link #2 downstream: 'previous' period: ES per hr: 24.00, mean time between errors:  150 s, collection duration: 900 s

──────────────────────────────
(†) The duration of the ’latest‘ errored seconds (ES) collection
bucket is variable, with a _maximum_ of 15 mins. The buckets’
start times are always 15 mins apart. A ‘previous’ bucket's
duration is a fixed 15 mins. An ES is a 1 s time period in which one
or more CRC errors are detected. A CRC error is a Reed-Solomon
coding-uncorrectable error, ie. corrupted data is received that
cannot be recovered.
──────────────────────────────      
                                              ◅ ◅ ◅◊▻ ▻ ▻

[Reformed]

xDSL needs nuking from orbit, just to be sure. What a horrendous waste of time, energy and effort chasing down these constant faults over 7km of rusting wires precariously jointed is.

Anything further on being rid of this nonsense and being able to replace it with something that just works? There is certainly no reason why a remote pizza box OLT cannot be installed in the area.

[Weaver]

It wasn’t like this before early 2020. Yes there were plenty of faults but not the same fault again and again, never getting fixed. Line 1 hasn’t failed in I don’t know how long. This forum serves as a record I suppose, but it would be a pain going back through old threads trying to find the most recent line 1 fault. I could go back through AA’s records more easily but there’s a lot of noise in there.

As for nuking from orbit don’t I wish. I don’t have the money nor the physical abilities to be able to take up some of the options available that would truly nuke the problem from orbit. FTTP is getting close now after 18 years of waiting, and the system as a whole is indeed highly reliable because the bonding covers up all known faults. Even with one line down it’s still fine and my wife is able to work with no problems. We also have 3G as a transparent failover should three copper lines fail and 4G is available to us too. If I didn’t monitor the system closely I wouldn’t even know that a line has gone down/up, or down and staying down. My wife gets an SMS (email optional too) to tell us that a line has gone down. Ten years ago, before I spent money in starting up the business, I definitely could have nuked the thing from orbit.

My aims in boring you all with documenting the history of this are (i) to use your collective brain power considering how weird it all is (ii) to serve as a historical record for me too, (iii) to be of interest to a small minority.

As for your point about the waste of time, you would thing Openreach would have some escalation department for repeat faults. They ought to know about this, and if not then they could be reminded and then they should either review the entire cable bundle or simply replace the whole lot with fresh copper as part of decent customer service.

I could ask AA to try and escalate this within OR and/or BTW. Presumably if I email OR they’ll tell me to contact my ISP.


Could you put your thinking caps on (or keep them on) and please, electronic engineering people, please let me know how we can have either a frequency-selective noise susceptibility and/or signal attenuation problem so that it amounts to reduced SNRM that isn’t truly narrow-band and which can somehow get fixed without us knowing how.

[burakkucat]

There is certainly no reason why a remote pizza box OLT cannot be installed in the area.

I would suspect that either the garage or the stable at "The Weaving Shed" could be made available to Openreach for such a purpose. All of Heasta could then be provisioned from that OLT.

The free provision of a fibre from that OLT to an ONT located in Senior Management's office would be a fair exchange. ("Senior Management" == "Finance Director" == Mrs Weaver.)

[burakkucat]

As for your point about the waste of time, you would thing Openreach would have some escalation department for repeat faults. They ought to know about this, and if not then they could be reminded and then they should either review the entire cable bundle or simply replace the whole lot with fresh copper as part of decent customer service.

sed 's/lot with fresh copper as part/lot with an optical fibre link as part/' 

[Weaver]

Indeed. Agrees with your sed. OR could save themselves money or maybe it’s BTW or AA paying for all the stupid engineer call-outs.

Before 2020, when a fault was fixed that was an improvement in the quality of the line. A fault fixed stayed fixed. It was one less, and enough call-outs and we would ultimately end up with a line with zero bugs. Twenty+ years ago there was a situation like this with repeat problems and it was fixed by replacing a piece of copper that was lying in the road and being driven over by vehicles every day. Instead the cable was replaced and buried properly. End of faults.

In my opinion, ideally when a fault is ‘fixed’ it ought to remain fixed. Clearly OR can’t fix faults that vanish the moment they arrive, but in that case they need to capture the badness without disturbing the system too much. Ghostbusters. A tall order for them. I need to find out whether or not the HCD faults survive the action of unplugging a modem from the line and then plugging it back in, the modem remaining powered up.

I wish AA had their own engineers, now I come to think of it. Don’t know how the practicalities or financial side of that would work but I’m going to ignore that for the moment. That would be fantastic. My own Black Sheep. Either on the end of a phone line or here in Heasta in person or someone remotely accessing kit over the internet (over 4G in this case).

[Reformed]

Replacing the copper would be a waste of money and time. Almost certainly not going to happen. Overbuild with fibre delivers a faster, more reliable and better service and is happening sooner or later.

I need to make some calls tomorrow and see what I can find out. Nostalgic as it is seeing weird faults on really long copper loops it is 2022 and we've the technology to do far better.

[Weaver]

We have the technology, but I don’t have the cash, nor the physical abilities that most of us have (as I’m stuck in bed much of the time btw).

Agreed that replacing the copper would be a waste of money and time. BT has spent 15 years saying that, while they continue to waste more money and my time on repairs. They are expected to have a commitment to a certain level of customer service.

[A polite reminder to all. Alternative link technologies have been discussed elsewhere already and are off-topic for this thread. This thread is about documenting faults and diagnosis musings.]

Good point about my garage. The stable isn’t suitable, and nor is the garage as there was some rodent damage in the garage some years ago, maybe before Thomas turned up though. Might be much better to use a different building, one that is imaginatively called "the Big Shed" which I used to use as a warehouse for electronics hardware and is now a residents’ lounge for Shepherd Hut guests. Has mains, but has no advantage over just using Janet’s office though, the latter being much much more secure and has much more room.