Kitz Forum

Broadband Related => ADSL Issues => Topic started by: Weaver on April 10, 2020, 02:01:00 PM

Title: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 10, 2020, 02:01:00 PM
This is the odd-shaped SNR-vs-tone graph that I get from my modem #4:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28426;image)

The others have the normal overall smooth curve apart from the usual notches at the top end, but anyway, without the depression at the very top seen here. Why do you think that depression might be there?



This is the graph for modem #3, for comparison purposes - the normal shape :

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28428;image)



Here are the complete stats for modem #4:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 442 Kbps, Downstream rate = 2940 Kbps
Bearer:   0, Upstream rate = 525 Kbps, Downstream rate = 2785 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    2.4       5.8
Attn(dB):    64.5       40.9
Pwr(dBm):    18.4       12.4

         ADSL2 framing
         Bearer 0
MSGc:      59      11
B:      28      62
M:      8      1
T:      3      1
R:      16      12
S:      2.6243      3.7736
L:      756      159
D:      1      8

         Counters
         Bearer 0
SF:      5746059      541776
SFErr:      994      5
RS:      140060197      3638675
RSCorr:      351555      22698
RSUnCorr:   1016      0

ReXmt:      448050      0
ReXmtCorr:   447986      0
ReXmtUnCorr:   1122      0

         Bearer 0
HEC:      1140      1
OCD:      2      0
LCD:      2      0
Total Cells:   606046574      114767360
Data Cells:   34016038      5929808
Drop Cells:   0
Bit Errors:   56760      1666

ES:      954      4
SES:      0      0
UAS:      53      53
AS:      92574

         Bearer 0
INP:      28.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      15.99      16.03
OR:      32.51      8.48
AgR:      2805.74   532.16

Bitswap:   21234/21234      105/105

Total time = 1 days 1 hours 43 min 47 sec
FEC:      351555      22698
CRC:      994      5
ES:      954      4
SES:      0      0
UAS:      53      53
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 15 minutes time = 13 min 47 sec
FEC:      5613      188
CRC:      23      0
ES:      22      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      6079      648
CRC:      20      0
ES:      20      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 1 hours 43 min 47 sec
FEC:      37897      2090
CRC:      148      0
ES:      140      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      313658      20608
CRC:      846      5
ES:      814      4
SES:      0      0
UAS:      53      53
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 1 days 1 hours 42 min 53 sec
FEC:      351555      22698
CRC:      994      5
ES:      954      4
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0




Summary of all modems - Live sync rates:
  #1: down 2898 kbps, up 529 kbps
  #2: down 2807 kbps, up 553 kbps
  #3: down 3066 kbps, up 396 kbps
  #4: down 2785 kbps, up 525 kbps
Title: Re: Odd shaped SNR-vs-tone graph
Post by: burakkucat on April 10, 2020, 03:45:41 PM
That's a strange looking plot. How does the equivalent Hlog (logarithmic representation of the channel characteristics function) plot look?  :-\
Title: Re: Odd shaped SNR-vs-tone graph
Post by: Weaver on April 10, 2020, 04:01:10 PM
The hlog for modem #4 looks very unremarkable to me :

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28436;image)
Title: Re: Odd shaped SNR-vs-tone graph
Post by: burakkucat on April 10, 2020, 04:04:27 PM
Hmm . . . Nothing odd there. I'm baffled.  ???
Title: Re: Odd shaped SNR-vs-tone graph
Post by: Weaver on April 10, 2020, 04:23:36 PM
For some reason, the upstream was missing for modem #4 in the first shot, and now it has reappeared, so here’s modem #4 again:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28440;image)
Title: Re: Odd shaped SNR-vs-tone graph
Post by: Weaver on April 10, 2020, 04:46:54 PM
The line has gone bad again. It just didn’t feel right, so I did a copper line test and the test failed, twice, with error message:

BT Test xDSL Copper Test:Inconclusive More diagnostics required Line test failure. Algorithm unable to continue.Inconclusive More diagnostics required Line test failure. Algorithm unable to continue. DS05:Test Failure - Please Retry - if unsuccessful report problem to OR via a Trouble Report

Third time lucky: the test completed with this fault message:

BT Test xDSL Copper Test:Pass Openreach network fault found.Pass Openreach network fault found. T023:FAULT - Battery Contact

So I reported this to AA, but they won’t be working today I presume. They do work on Saturdays though - they did submit a fault to OR for me on a Saturday before.
Title: Re: Odd shaped SNR-vs-tone graph
Post by: burakkucat on April 10, 2020, 05:03:37 PM
I wonder if a joint closure has recently been opened, to fix a different fault in a cable pair, and the disturbance has impacted on your line #4 circuit.
Title: Re: Odd shaped SNR-vs-tone graph
Post by: Weaver on April 10, 2020, 06:16:21 PM
Good point. Makes sense.

It’s a shame I don’t have any data going back further. If OR did some work on something, as Burakkucat suggests, then we might see some change in a measurable stat at a certain time then.

I tested the other lines with the ‘copper line test’ and they look ok.



Question: on the CRCs vs time graph and FECs vs time graph what are the units on the axes ?

Recap: I apologise because I have asked this question before, I’m just rechecking my understanding and failing memory: Iirc, ‘FECs’ is the count of errors detected and successfully fixed by using forward error correction maths, whereas ‘CRCs’ is the number of errors detected but which could not be successfully corrected?
Title: Re: Odd shaped SNR-vs-tone graph
Post by: burakkucat on April 10, 2020, 08:23:46 PM
Question: on the CRCs vs time graph and FECs vs time graph what are the units on the axes ?

I shall have to let the maestro explain, as I do not use the firmware image.

Quote
Recap: I apologise because I have asked this question before, I’m just rechecking my understanding and failing memory: Iirc, ‘FECs’ is the count of errors detected and successfully fixed by using forward error correction maths, whereas ‘CRCs’ is the number of errors detected but which could not be successfully corrected?

Yes that's good enough explanation, for me.  :) 

I usually explain FECs (for those who are worring about the numbers noted) as "CRCs that did not occur due to the successful intervention of the (forward) error detection and correction mechanism".
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 10, 2020, 08:30:58 PM
Here’s a CRCs-vs-time graph for modem 4, to illustrate my question about axes and units :

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28442;image)

I’m wondering what Δn is in Δn/Δt, the count per unit ‘what time intervals’ - the units up the y axis?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 10, 2020, 09:00:03 PM
Ah, I see.

I believe that the y-axis is the number of CRCs per minute, while the x-axis is time.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 14, 2020, 12:43:44 AM
What time interval, do you think, from a guess from the numbers?



Can’t tell if OR has been out today (Monday) or not. I’m assuming not, as there are no relevant records in clueless.aa.net.uk. I can’t see any engineer’s notes. It was an English bank holiday for AA, although not for us up here.

The downstream sync has become very bad now, down to 2.5M Mbps d/s @ 3 dB SNRM. That’s much worse than it has been; the only sign of any problem originally was that funny shaped noise graph and then the battery contact fail test result.

Waiting on faultbtconfirm   Wait customer confirms fault closed PSTN Fault - NSY   aa@a
11 Apr 10:12:11      Track PSTN Fault 5-7-201625816677 Notes Field: **Line Stability:**Network Stability:**Test Outcome:Fail**MFL:LN**Term Statement:LINE TERMINATION DETECTED**Line Signature:**Distance to Fault:0**Cable length:8.11**Test Start Time:2020-04-11T11:12:03**Test Stop Time:2020-04-11T11:12:03   bt
11 Apr 10:12:11      Track PSTN Fault 5-7-201625816677 Informational Message: 4465 Please refer to the Notes field for the actual message   bt
11 Apr 10:12:11      Track PSTN Fault 5-7-201625816677 Estimated Response Time: 2020-04-14T23:59:59   bt
11 Apr 10:12:11      Track PSTN Fault 5-7-201625816677 Status: Open - Associating Info   bt
11 Apr 10:11:25      Track PSTN Fault 5-7-201625816677 Informational Message: 4040 Notification only - Estimated Response Time.   bt
11 Apr 10:11:25      Track PSTN Fault 5-7-201625816677 Informational Message: 3100 Trouble Report Accepted   bt
11 Apr 10:11:25      Track PSTN Fault 5-7-201625816677 Estimated Response Time: 2020-04-14T23:59:59   bt
11 Apr 10:11:25      Track PSTN Fault 5-7-201625816677 Status: Open - New   bt
11 Apr 10:11:14      Track PSTN Fault 5-7-201625816677 Message Informational 9323 The asset care level 2.5 will be applied rather than the specified service level.


I notice it said “Estimated Response Time: 2020-04-14T23:59:59” so they have all of Tuesday; perhaps that means an engineer might be sent out tomorrow. Perhaps they were all busy, but I would have thought it would have been Monday seeing as it isn’t a Scottish bank holiday, and I have discovered faults at the weekend recently and had the engineer out on Monday.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 14, 2020, 01:20:50 AM
What time interval, do you think, from a guess from the numbers?

I believe the data is harvested once a minute. So the quantity on the y-axis is the "number of events in the last minute" and the x-axis is every minute.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 14, 2020, 02:13:37 AM
I got fed up with typing 192.168.n.1 all the time, so I defined some domain names m1.example.com, m2.example.com etc - something really short anyway - and it works very nicely because the browser remembers those domain names in its history and if you just type m4… say, it completes the rest.

For obvious reasons, in general you’re not supposed to define A or AAAA records with RFC1918 private target IP addresses, and I got a warning from AA’s clueless.aa.net.uk server’s DNS record setup function and it coloured my new entry yellow, but it sensibly let me go ahead anyway.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 14, 2020, 11:57:22 AM
Engineer has come out and just finished. I have the notes :

===Point Of Intervention notes===

Between this point...
- Location: In garden of clisham
- Work Point: JB26
And this point...

Plant details...
- Plant affected: JNT100
- Plant type: MC32

Multiple Intervention?: N
===Point Of Intervention notes ends===
(No manually entered closure notes)

=== QBC Summary Start ===

Customer Report: End customer advised the line was noisy.

Actions to resolve: Engineer has resolved the fault located at the D-side including aerial cables / lead-in / block terminal. There was a fault outside the end customer's curtilage caused by general wear and tear. The fault was fixed by clearing in joint.

Additional information: Engineer has not visited end customer premises.

Final alternate test results: Final FastTest performed from the customer premise.The test passed on 14/04/2020 11:33:48.
=== QBC Summary End ===
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 14, 2020, 09:35:04 PM
So a bad joint has been fixed. Exactly where that joint is located . . .  :shrug2:

But all is now well, once again.  :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 15, 2020, 10:31:44 PM
The SNRM-vs-tones graph is still approx. unchanged - still has that depression in the top of it. The downstream sync rate is still quite poor by the usual standard of things. I’m not sure what to do about it:


xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 436 Kbps, Downstream rate = 2828 Kbps
Bearer:   0, Upstream rate = 525 Kbps, Downstream rate = 2578 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.1       5.7
Attn(dB):    64.0       40.7
Pwr(dBm):    18.3       12.4

         ADSL2 framing
         Bearer 0
MSGc:      54      11
B:      28      62
M:      8      1
T:      3      1
R:      16      12
S:      2.8343      3.7736
L:      700      159
D:      1      8

         Counters
         Bearer 0
SF:      367531      367760
SFErr:      25      0
RS:      8269456      1955956
RSCorr:      16479      1951
RSUnCorr:   26      0

ReXmt:      17649      0
ReXmtCorr:   17648      0
ReXmtUnCorr:   31      0

         Bearer 0
HEC:      29      0
OCD:      0      0
LCD:      0      0
Total Cells:   35782273      7306913
Data Cells:   858511      170651
Drop Cells:   0
Bit Errors:   1104      0

ES:      100191      59
SES:      74      3
UAS:      4209      4136
AS:      5897

         Bearer 0
INP:      28.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      15.94      16.03
OR:      30.10      8.48
AgR:      2597.91   532.16

Bitswap:   431/431      12/12

Total time = 5 days 38 sec
FEC:      7699356      101286
CRC:      209426      134
ES:      100191      59
SES:      74      3
UAS:      4209      4136
LOS:      8      2
LOF:      62      2
LOM:      0      0
Latest 15 minutes time = 38 sec
FEC:      98      0
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      2580      156
CRC:      4      0
ES:      4      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 38 sec
FEC:      98      0
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      94368      22784
CRC:      3002      111
ES:      194      42
SES:      63      3
UAS:      4096      4034
LOS:      7      2
LOF:      53      2
LOM:      0      0
Since Link time = 1 hours 38 min 17 sec
FEC:      16479      1951
CRC:      25      0
ES:      25      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0

Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 15, 2020, 11:03:45 PM
I complained to AA about the slight downstream sync speed reduction : 2.5Mbps down from 2.8Mbps. Did a copper line test, which failed to complete, the test algorithm failed three times and finally cam back with a result of ‘line test OK’. Seems very fishy, it failing repeatedly like that - just as it did going way back to before the repair.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 15, 2020, 11:46:49 PM
If you would perform a harvest of the data, for line #4 only, and then send a copy of that data to the usual e-mail address, I'll create the four snapshot graphs locally.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 16, 2020, 02:01:55 AM
I have now taken a look at the data and created the usual four snapshot plots.

The "sag", seen in both the Bit Loading and the SNR plots, is puzzling. It's not like any perturbations usually seen when circuits are degraded. See the montage, attached below.

Any opinions, please?  :-\
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 02:10:56 PM
After Openreach turned out to make repairs on my line 4, the downstream sync rate dropped by between 11-17% to 2.51 Mbps d/s sync from a former typical 2.8-3.0 Mbps sync rate. The upstream is unchanged. The downstream sync rate has been around 2.8-3.1Mbps consistently for the last four years, and was at 3.0-3.1Mbps this winter. Is there anything I can do to get things back to the way they were?

I complained to AA and got a very unhelpful reply which appeared as if they might be saying I didn’t know what I was talking about. Seeing as they have a published no "bullsh*t (https://www.aa.net.uk/support/)" policy, I could have a go at them on that basis, but feeling charitable and working in “assume good faith”, a fairly rare thing for me, I am proceeding on the basis that this has been a misunderstanding based on definitions of rates; I quoted synch rates to them, while they could have been thinking about their egress rates ie rate limiters’ rates which are = sync rate * 87.91% approx iirc. Or they could have been thinking about IP + TCP SDU (ie TCP payload) rates at 84.2 - 86.1% of synch rate. Those differences are large enough to cover the gaps between differently defined sets of rates and could make it just miscommunication. Despite feeling miffed at the possibility I had been brushed off I noticed that AA tech support have set up regular capture of the synch rate on a repeating timer, every hour a server remotely queries my modem via BT. That’s good because then they will be seeing synch rates not one of the other measures. And it means they are doing something.

Just recently I have watched the SNRM very carefully, waiting until such a time as the downstream synch rate may happen to go above 3.15 Mbps. Now 3Mbps is the downstream SNRM target, and a +0.15 Mbps additional gap seems to be sufficient such that if you choose a moment when the synch rate is at about that level and then force a resynch then you will then end up with a higher downstream synch rate. I’ve mange to push it up by small amounts three times by this feeble method, in total from about 2.51 to 2.667 Mbps d/s synch. It’s not much but it is a sizeable chunk of what I lost now regained (went down from a former 2.8-3.1 Mbps downstream sync rate to 2.51 Mbps at lowest). I managed to creep up in steps: from 2.51 Mbps to 2.564 to 2.61 to 2.667 Mbps. However I feel that there is no reason to assume that this method will get me very far long term though, perhaps I’m wrong. I assume that the further up I manage to crawl, the more infrequent the opportunities may become, also if I were to get things wrong one time then I think I could then easily lose the entire total of such feeble gains, and possibly even more, in one single large drop. Anyway, I don’t see that there’s anything sustainable about these tactics.

There’s also the question of the strange shape of my SNRM-vs-tones graph nowadays. As mentioned earlier, Line 4 has a weird dent or depression or ‘sag’, as Burakkucat said, in the top of its graph as mentioned earlier in the thread, for reasons unknown.

Does this odd-looking feature have anything to do with my downstream sync rate loss problem?

Since it’s a depression in such a graph then this could either mean (i) more noise at the middle frequencies or (ii) less signal at those frequencies. Usually without additional information it’s impossible to tell which of the two it is because all you have in SNR is a ratio derived from two quantities. In this case though the second option seems totally unbelievable; I can’t see how there could be reduced signal levels only in middle frequencies, lower than the higher signal levels at frequencies both below and above. So we must be left with the first option: higher noise in a selective frequency band, which is fairly wide, from > tone 40 to < tone 80 on the SNRM-vs-tones graph, but on a bit loading graph the depression is from tone 48 - 78 inclusive and is 1 bit deep. So that’s (78 - 48+1)=31 bits per symbol period lost ie a rate reduction of 31 * 1 * 4k = 124 kbps. So if that were filled in it would mean 2.667 + 0.120 = 2.791, and if the top were rounded off into a more realistic-looking hump, following the general SNRM-vs-tones curve rather than being sliced flat, that might mean another ~ 60kbps, which would take us all the way back to roughly where we should be. So the SNRM curve depression does fit in with the theory of a noise source - it’s about the right size so that it fits. Another way in which this fits is that the dent/sag is in the downstream tones only and only the downstream synch rate is affected; the upstream rate is not.

Another question: looking at the stats regularly, I noticed my downstream ‘attainable rate’ has gone up too when the synch rate has gone up. Could someone remind me what this is all about as I’ve forgotten completely? Does it mean I have a chance of getting to that rate if I do <something>? - no idea what though.

Right now the d/s stable rate is 2908 kbps @3.1 dB SNRM - and current sync rate 2667 kbps.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 17, 2020, 03:34:20 PM
There’s also the question of the strange shape of my SNRM-vs-tones graph nowadays. As mentioned earlier, Line 4 has a weird dent or depression or ‘sag’, as Burakkucat said, in the top of its graph as mentioned earlier in the thread, for reasons unknown.

Does this odd-looking feature have anything to do with my downstream sync rate loss problem?

Yes, I believe it does.

Quote
Since it’s a depression in such a graph then this could either mean (i) more noise at the middle frequencies or (ii) less signal at those frequencies. Usually without additional information it’s impossible to tell which of the two it is because all you have in SNR is a ratio derived from two quantities.

Hence my request for the opinions of others in my post, above.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 04:02:13 PM
Agreed. I think we need the collective wisdom of the high council.

Thinking more about a realistic shape for a hump, taking two 1-bit thick horizontal slices through the top of such an imagined curve, we get the total area in bits*tones = ( 80 + 1 − 38 ) × 1 + ( 79 + 1 − 51) × 1 = 72 bits so we have 72 * 4 kbps= 288 kbps and the lowest sync rate 2510 kbps + 288 k brings us right back up to where we should be in full. To get to our former 3.0 - 3.1 Mbps we might need some cold weather to get better conductivity and reduce the attenuation/loop loss. Maybe those latter figures, which I often achieve, are winter-only though.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 05:22:48 PM
This is what the historical SNRM vs time graph (not tones, ie frequency distribution) looks like now:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28486;image)

Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 17, 2020, 05:28:07 PM
I trust your arithmetic and so accept that the "sag" does account for the "missing" synchronisation speed.  :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 05:38:34 PM
And here FYI is the current bit loading ie. bits-per-bin assignments graph:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28492;image)

A less optimistic assumption about area still gives something like bits * tones = 30 * 1 * 2 = 60 which gives A rate increase of 60 * 4 kbps = 240 kbps which is still good enough, so the exact details don’t seem to matter too much.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 05:49:45 PM
I wonder where the extra noise came from; be it crosstalk or whatever but it’s band-limited that’s certain so we can multiply those bin numbers by 4.3125 kbps (iirc) to get the frequency range. And the usual mysteries: why this modem and not the others, and why now?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 10:07:33 PM
I just noticed that in the space of a few hours, now the sun has gone down (it’s now 21:05 UTC) the bit loading has changed a little. Compare to the preceding bit loading graph (https://forum.kitz.co.uk/index.php/topic,24614.msg414491.html#msg414491). The top of the graph looks different, less flat. Unfortunately I stupidly cropped off the legend at the bottom below the horizontal axis, so this should read "Tone". So here’s bit-loading II:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28495;image)

Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 17, 2020, 10:30:29 PM
The very latest SNRM-vs-tones graph looks slightly different from that posted at the start of this thread (https://forum.kitz.co.uk/index.php/topic,24614.msg414054.html#msg414054) too; if my eyes do not deceive me, the top portion of it is all about 1dB lower. So it looks as if either the attenuation has increased - which doesn’t seem likely at all - or the noise has increased (much more likely) because it’s now night time (21:20 UTC).

So here’s the second SNRM-vs-tones graph:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28497;image)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Chrysalis on April 18, 2020, 09:59:47 AM
I have no idea why you have that shape, but I can tell you in the multiple years I had ADSL, I had multiple of those on my line and were there for the full duration.

The noise bursts I used to get during office hours were shaped like that as well but much more severe.  They came and went in an instant.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 18, 2020, 03:07:03 PM
Thanks Chrys ! It’s the same general shape on a much wider scale, due to your use of a vastly greater range of tones. When was that snapshot taken btw?

(Btw, I can see just how incredibly good my line is, yours being 15 dB attn better yet having such a low bit rate downstream, yours was only 3.7Mbps d/s sync and my line 3 is 3.05 Mbps currently, so mine is much much cleaner just very weak signal. It’s as I thought, supports the idea that my lines have very thick copper (from Black Sheep) and extremely low noise due to their isolation from human noise sources.)

So the mystery remains, why not before and why now and why only on this line not the others?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Chrysalis on April 19, 2020, 12:00:16 PM
Many years ago, its been a long time since I was on ADSL.

The normal sync for the line if on a 6db fast path profile was around 6mbit, but it had huge issues with sudden large bursts of noise, when they occurred it generated those curves eating into the signal, the only way the line could stay online was when I had SRA.  The issue persisted from the day the line was activated until the day it was ceased year's later (moved to cable).

Since the problem is E side, thankfully its gone on VDSl.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 20, 2020, 10:51:02 PM
Could anyone comment on the error rates ES and CRCs per unit time ? Good/bad?


xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 436 Kbps, Downstream rate = 2680 Kbps
Bearer:   0, Upstream rate = 525 Kbps, Downstream rate = 2667 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):     2.4        5.6
Attn(dB):    64.5       41.0
Pwr(dBm):    18.4       12.4

         ADSL2 framing
         Bearer 0
MSGc:      57      11
B:      28      62
M:      8      1
T:      3      1
R:      16      12
S:      2.7403      3.7736
L:      724      159
D:      1      8

         Counters
         Bearer 0
SF:      18161933      720074
SFErr:      6951      46
RS:      429075683      1032124
RSCorr:      1903856      70687
RSUnCorr:   7172      0

ReXmt:      2278836      0
ReXmtCorr:   2278204      0
ReXmtUnCorr:   7855      0

         Bearer 0
HEC:      8894      25
OCD:      0      0
LCD:      0      0
Total Cells:   1856628001      367910954
Data Cells:   112535746      16779885
Drop Cells:   0
Bit Errors:   410203      9838

ES:      108636      117
SES:      75      3
UAS:      4432      4359
AS:      296759

         Bearer 0
INP:      27.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      16.18      16.03
OR:      31.13      8.48
AgR:      2686.98   532.16

Bitswap:   68212/68213      452/452

Total time = 11 days 9 hours 12 min 37 sec
FEC:      10380348      235257
CRC:      218468      208
ES:      108636      117
SES:      75      3
UAS:      4432      4359
LOS:      8      2
LOF:      62      2
LOM:      0      0
Latest 15 minutes time = 12 min 37 sec
FEC:      6968      158
CRC:      25      0
ES:      25      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      7285      166
CRC:      52      0
ES:      51      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 9 hours 12 min 37 sec
FEC:      240569      10515
CRC:      919      5
ES:      870      4
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      619848      20663
CRC:      2674      7
ES:      2515      6
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 3 days 10 hours 25 min 59 sec
FEC:      1903856      70687
CRC:      6951      46
ES:      6556      35
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Chrysalis on April 21, 2020, 06:33:56 AM
the CRC per ES is good, thats almost 1 CRC per ES which is as low as you can get.

the amount of ES is hard to say as these are very lone ADSL lines, longer lines tend to have more errors.

Would need a baseline average of ES/hour figures so could compare to a reasonable expectation.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on April 21, 2020, 11:34:18 AM
I looked at line 3 for comparison, which is 3.066Mbps downstream, so much faster downstream, dreadful upstream, and much cleaner

Live sync rates:
  #1: down 2898 kbps, up 529 kbps
  #2: down 2807 kbps, up 553 kbps
  #3: down 3066 kbps, up 396 kbps
  #4: down 2564 kbps, up 522 kbps

Line 4 is running at a really, really low d/s SNRM at the moment, because I forced resynchs at opportune moments when the SNRM was well above the target. But I don’t like the number of CRCs I’m getting compared with the other lines, so I forced another resynch in the middle of the night to get it back on the target of 3dB d/s as opposed to being so very low at 2.4dB. That has reduced the downstream synch rate somewhat but no errors now.



Line 4 QLN is fairly rubbish too, compared to line 3. Here’s line 4 QLN:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28543;image)



This is line 3 QLN for comparison:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28545;image)

So much quieter.

All this extra noise - I wonder if line 4 is better at detecting it, it’s not that there’s more noise out there, surely? - apart from crosstalk from different neighbours, how can the noise environment be the different? The two conductors are in the same place, (approximately) in the same environment and should be exposed to the same noise, but the QLN says otherwise. Wild speculation: if there is a new non-linear joint in line 4, detecting noise, then that would make sense because whatever the cause of the phenomenon is, it is new: it’s only been around for a week or so; something was done to the line and this noise then appeared, and I lost 300-500k d/s sync which is around 10-15%.

Is that a sensible suggestion?

There’s no way that I can get BT to sort it out as the line is nowhere near faulty; I used to be incredibly lucky with it with extremely low noise and low attenuation for such a long line, and now I’m not quite so lucky any more. If it were to develop a fault then perhaps the repair process might somehow magically kick the current badness out the same way as it was introduced but in reverse. AA is doing their bit, constantly measuring the sync rate, collecting data.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Chrysalis on April 21, 2020, 04:39:10 PM
been able to run those long adsl lines at a 3db margin is very good, my line would vary upwards of 7-8 db on some tones during a typical 24 hour period. 
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on April 21, 2020, 05:55:00 PM
Wild speculation: if there is a new non-linear joint in line 4, detecting noise, then that would make sense because whatever the cause of the phenomenon is, it is new: it’s only been around for a week or so; something was done to the line and this noise then appeared, and I lost 300-500k d/s sync which is around 10-15%.

Is that a sensible suggestion?

Yes, that would fit the observed facts.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 05, 2020, 11:39:52 PM
Latest figures - Live sync rates:
  #1: down 2807 kbps, up 519 kbps
  #2: down 2700 kbps, up 544 kbps
  #3: down 2964 kbps, up 389 kbps
  #4: down 2187 kbps, up 532 kbps

All of the lines were faster downstream before, the others by approx 100k very roughly but line 4 downstream is down by 377kbps, if my arithmetic is right, line 4 was 2564 kbps in the earlier post (https://forum.kitz.co.uk/index.php/topic,24614.msg414778.html#msg414778).

Latest line 4 detailed stats:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 452 Kbps, Downstream rate = 2344 Kbps
Bearer:   0, Upstream rate = 532 Kbps, Downstream rate = 2187 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.0       5.7
Attn(dB):    65.5       41.5
Pwr(dBm):    18.2       12.4

         ADSL2 framing
         Bearer 0
MSGc:      54      12
B:      14      62
M:      16      1
T:      5      1
R:      14      12
S:      3.4499      3.7267
L:      589      161
D:      1      8

         Counters
         Bearer 0
SF:      406574      396020
SFErr:      141      0
RS:      7623276      2830906
RSCorr:      20520      2109
RSUnCorr:   145      0

ReXmt:      45128      0
ReXmtCorr:   45119      0
ReXmtUnCorr:   166      0

         Bearer 0
HEC:      155      0
OCD:      0      0
LCD:      0      0
Total Cells:   34060189      8333379
Data Cells:   419321      118148
Drop Cells:   0
Bit Errors:   8245      0

ES:      448610      610
SES:      194      8
UAS:      5415      5234
AS:      6641

         Bearer 0
INP:      27.00      2.00
INPRein:   0.00      0.00
delay:      8      7
PER:      16.17      16.77
OR:      29.68      8.58
AgR:      2208.24   538.85

Bitswap:   1444/1444      35/35

Total time = 57 days 10 hours 19 min 9 sec
FEC:      36764065      1604333
CRC:      658801      903
ES:      448610      610
SES:      194      8
UAS:      5415      5234
LOS:      19      2
LOF:      142      2
LOM:      8      0
Latest 15 minutes time = 4 min 9 sec
FEC:      760      73
CRC:      4      0
ES:      4      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      3279      307
CRC:      20      0
ES:      21      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 10 hours 19 min 9 sec
FEC:      115341      20594
CRC:      912      13
ES:      873      9
SES:      0      0
UAS:      60      60
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      305956      33338
CRC:      3853      24
ES:      3655      13
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 1 hours 50 min 41 sec
FEC:      20520      2109
CRC:      141      0
ES:      137      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 06, 2020, 07:04:35 AM
In view of the fact that now, at downstream SNRM=3dB, the downstream speed is sufficiently slow that it is a FTB, I have talked to my ISP AA about it, having a minor moan. Since it’s an FTB, does that mean that OR has to do something about it ?

Anyway if so, I’m hoping AA will get it fixed for me.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 08, 2020, 03:05:28 PM
AA has booked an SFI engineer, coming out tomorrow morning (Tuesday).
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 09, 2020, 11:46:32 AM
Openreach engineer has been and gone. Replaced some connectors. Bits per bin and SNR vs frequency curves look fine now, completely normal shape with no depression in the region of tones 40-80. Downstream synch rate is now fine at 2700 which is still a tiny bit low but that’s only because the current SNRM is a bit high still at 3.6 dB and settling it down to 3.0 dB means it will go a little faster yet, so a perfect result. OR SFI man was excellent.

Looks like a normal overall shape. Tiny wiggles at the very top are too small to do anything about. Downstream Line attenuation is a bit high now at 66.5dB, which is 1.5dB.

Current synch rates:
  #1: down 2807 kbps, up 519 kbps
  #2: down 2700 kbps, up 544 kbps
  #3: down 2964 kbps, up 389 kbps
  #4: down 2700 kbps, up 535 kbps

SNR vs frequency curve attached:

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28777;image)

Line 4 detailed stats:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 458 Kbps, Downstream rate = 3020 Kbps
Bearer:   0, Upstream rate = 535 Kbps, Downstream rate = 2700 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.5       6.3
Attn(dB):    66.5       41.5
Pwr(dBm):    17.2       12.4

         ADSL2 framing
         Bearer 0
MSGc:      57      12
B:      28      62
M:      8      1
T:      3      1
R:      16      12
S:      2.7067      3.7037
L:      733      162
D:      1      8

         Counters
         Bearer 0
SF:      409669      394691
SFErr:      0      3
RS:      9678446      2808369
RSCorr:      336      3457
RSUnCorr:   0      0

ReXmt:      195      0
ReXmtCorr:   195      0
ReXmtUnCorr:   0      0

         Bearer 0
HEC:      0      11
OCD:      0      0
LCD:      0      0
Total Cells:   41879018      8304776
Data Cells:   311415      132106
Drop Cells:   0
Bit Errors:   0      481

ES:      453166      678
SES:      213      8
UAS:      7869      7678
AS:      6578

         Bearer 0
INP:      27.00      2.00
INPRein:   0.00      0.00
delay:      8      7
PER:      15.98      16.66
OR:      31.52      8.64
AgR:      2720.38   542.20

Bitswap:   92/92      45/45

Total time = 60 days 22 hours 38 min 21 sec
FEC:      37492020      1722719
CRC:      664401      995
ES:      453166      678
SES:      213      8
UAS:      7869      7678
LOS:      21      2
LOF:      159      2
LOM:      8      0
Latest 15 minutes time = 8 min 21 sec
FEC:      17      115
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      41      249
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 22 hours 38 min 21 sec
FEC:      207099      35728
CRC:      2576      47
ES:      1631      33
SES:      19      0
UAS:      2454      2444
LOS:      2      0
LOF:      17      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      189009      30725
CRC:      1041      20
ES:      1010      15
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 1 hours 49 min 37 sec
FEC:      336      3457
CRC:      0      3
ES:      0      2
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0


Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 09, 2020, 12:26:19 PM
Detailed stats after a forced resynch, which gave a very helpful improvement, synch rate downstream up to 2.86Mbps and for some reason the line attn downstream has gone back to normal and the 1.5dB has disappeared. The downstream tx power is up from 17.2 to 18.3dB and that must be contributing to the improvement too.

Lines sync rates now:

  #1: down 2807 kbps, up 519 kbps
  #2: down 2700 kbps, up 544 kbps
  #3: down 2964 kbps, up 389 kbps
  #4: down 2861 kbps, up 528 kbps


Line 4:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 496 Kbps, Downstream rate = 3148 Kbps
Bearer:   0, Upstream rate = 528 Kbps, Downstream rate = 2861 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.1       5.9
Attn(dB):    65.0       41.5
Pwr(dBm):    18.3       12.4

         ADSL2 framing
         Bearer 0
MSGc:      53      12
B:      32      62
M:      4      1
T:      3      1
R:      10      14
S:      1.4508      3.7561
L:      783      164
D:      2      8

         Counters
         Bearer 0
SF:      3599      3376
SFErr:      0      0
RS:      159260      60721
RSCorr:      0      8
RSUnCorr:   0      0

ReXmt:      2      0
ReXmtCorr:   2      0
ReXmtUnCorr:   0      0

         Bearer 0
HEC:      0      0
OCD:      0      0
LCD:      0      0
Total Cells:   392626      67582
Data Cells:   0      0
Drop Cells:   0
Bit Errors:   0      0

ES:      453166      678
SES:      213      8
UAS:      7928      7737
AS:      58

         Bearer 0
INP:      26.00      2.50
INPRein:   0.00      0.00
delay:      8      8
PER:      16.04      16.90
OR:      29.40      8.51
AgR:      2878.13   534.63

Bitswap:   15/15      0/0

Total time = 60 days 22 hours 57 min 27 sec
FEC:      0      8
CRC:      0      0
ES:      453166      678
SES:      213      8
UAS:      7928      7737
LOS:      21      2
LOF:      159      2
LOM:      8      0
Latest 15 minutes time = 12 min 27 sec
FEC:      0      8
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      59      59
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      0      0
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 22 hours 57 min 27 sec
FEC:      0      8
CRC:      0      0
ES:      1631      33
SES:      19      0
UAS:      2513      2503
LOS:      2      0
LOF:      17      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      0      0
CRC:      0      0
ES:      1010      15
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 57 sec
FEC:      0      8
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on June 09, 2020, 04:47:08 PM
That looks like a good result. I suspect it is all that the line can support.  :)

(Let's hope that the haggis do not gnaw on the cable sheath and pee in the joint closures.  ::)  )
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 09, 2020, 06:50:37 PM
It’s a very good result. The bits-per-bins are not as good as line 3, which shows that given a bit more work - which is never going to happen - it could give a bit more speed, another 100k or so, but that is just getting silly.

Latest line 4 bits-per-bins :

(https://forum.kitz.co.uk/index.php?action=dlattach;topic=24614.0;attach=28780;image)


One more important thing that I notice is that there are now zero ES d/s - back to what it should be. Before, there were almost always 2-3 ES per hour. That suggests to me support for the increased vulnerability detector of noise theory, where a joint had become a better detector or even demodulator?

Good job by OR and extremely helpful AA. Engineer’s notes are not back yet.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on June 09, 2020, 08:26:19 PM
One more important thing that I notice is that there are now zero ES d/s - back to what it should be. Before, there were almost always 2-3 ES per hour. That suggests to me support for the increased vulnerability detector of noise theory, where a joint had become a better detector or even demodulator?

Yes, that is possible. A joint showing semi-conductive tendencies.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on June 09, 2020, 09:22:25 PM
Remembering A-level maths and anecdotes about picking up Radio 4 on your metal teeth, demodulation of AM was what I was thinking about. I’m not sure of the mathematics of why nonlinearity helps with the kind of errors one sees in DSL specifically though.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on June 09, 2020, 10:19:03 PM
Keeping things very simple, we have two radio transceivers (of very low power) linked together by a metallic pathway that is only marginally AC balanced. The fact that xDSL actually works is just sub-miraculous. Inset imbalances, series or shunt resistances, series or shunt capacitances, couplings from other circuits & non-linear elements into that radio-frequency transmission line and errors have to be ever present.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 11, 2020, 01:11:37 AM
It’s back. Line 4 is showing a drop in downstream speed after a resync and the line 4 SNRM vs freq graph shows the characteristic dent / hollow between tones 40-80 approx.

(https://i.postimg.cc/zXHnK1S0/0-CE8-E26-B-F1-CC-4-A52-A6-E5-9576-BE2-A2-B42.jpg)

Live sync rates:
  #1: down 2701 kbps, up 509 kbps
  #2: down 2762 kbps, up 512 kbps
  #3: down 2883 kbps, up 376 kbps
  #4: down 2354 kbps, up 522 kbps

Line 4 downstream was the same as the other three until the resync.

AA reported the following:
Quote
Tx rate (adjusted) 2486358 to 2066526 (rx 522000)

- and I think those numbers are IP PDU rates, not sync rates; ‘Tx’ means downstream.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on July 11, 2020, 02:17:20 PM
Yet another mysterious event.  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 11, 2020, 07:16:08 PM
So either the (fairly) recent BT engineer didn’t fully fix the fault (in a lasting way), although the result was an excellent outcome, or otherwise there was more than one problem and the other latent fault was lurking, waiting to appear. Andrews and Arnold took a look at it yesterday. I would say that it isn’t slow enough yet for them to be able to justify getting an engineer out. It may deteriorate over time, as before. But nearly 420kbps is quite a big drop, a damned nuisance.

Is it possible for faults to get fixed and then become unfixed, just like that?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on July 11, 2020, 07:57:50 PM
Is it possible for faults to get fixed and then become unfixed, just like that?

I guess that anything is possible but without knowing exactly what was fixed, last time, it is really difficult to say.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: kitz on July 12, 2020, 10:14:30 AM
it may be crosstalk.   
 
Prior to vdsl and the availability of QLN graphs then a nice smooth dish shape on the SNR was about the only way to identify crosstalk.   On the SNR/tone graph & bitload graphs, crosstalk causes what I've always called a "shallow bowl" effect.   On QLN graph the curve shape is reversed and looks like a seagull wing.
 
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 12, 2020, 11:54:23 AM
Thanks Lesley, I didn’t know that.

A question for you all though. The thing is though, our BT man fixed something successfully last time, so, as far as I can see, strictly speaking it would have to be a susceptibility to crosstalk thing, no? Otherwise it wouldn’t be fixable and also we wouldn’t have an explanation as to why other lines are not getting it.

People talk about ‘balance’; is that right? I’m not exactly sure what that means as I don’t have an electronics degree, just a failed theoretical physics student. I’m assuming it means an undesirable dissimilarity between the two legs of a copper pair so that common mode rejection (which I believe is taking the difference between the voltages at the rx end) doesn’t work properly as the two sides aren’t equal, so their subtraction doesn’t cause cancellation of common noise. Is that right? Either that or else a failure of adequate twist in a twisted pair which amounts to the same kind of noise susceptibility problem.

It’s a damned nuisance, a 15% drop roughly, something like that, but not enough to count as a FTB

Do we have any guesses about why such a thing would suddenly start up ? - either the previous fault having been all good for a time, since it was last fixed, or alternatively this is a second separate fault, independent of the first one, which remains fixed.



Current detailed stats for modem 4. Pretty pictures below:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 439 Kbps, Downstream rate = 2540 Kbps
Bearer:   0, Upstream rate = 522 Kbps, Downstream rate = 2354 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.0       5.9
Attn(dB):    65.5       41.7
Pwr(dBm):    18.2       12.4

         ADSL2 framing
         Bearer 0
MSGc:      51      11
B:      27      62
M:      8      1
T:      3      1
R:      16      12
S:      2.9953      3.7975
L:      641      158
D:      1      8

         Counters
         Bearer 0
SF:      15597186      67600
SFErr:      11734      57
RS:      333389855      4151784
RSCorr:      1364442      118752
RSUnCorr:   12127      0

ReXmt:      2676494      0
ReXmtCorr:   2675741      0
ReXmtUnCorr:   12953      0

         Bearer 0
HEC:      13724      64
OCD:      0      0
LCD:      0      0
Total Cells:   1392268593      311379799
Data Cells:   60057824      16553586
Drop Cells:   0
Bit Errors:   731900      9624

ES:      11744      262
SES:      42      0
UAS:      199      157
AS:      252751

         Bearer 0
INP:      26.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      16.00      16.13
OR:      28.48      8.42
AgR:      2373.11   528.81

Bitswap:   54625/54626      305/305

Total time = 15 days 14 hours 19 min 10 sec
FEC:      1440420      126580
CRC:      12299      66
ES:      11744      262
SES:      42      0
UAS:      199      157
LOS:      1      0
LOF:      7      0
LOM:      33      0
Latest 15 minutes time = 4 min 10 sec
FEC:      1349      98
CRC:      6      0
ES:      5      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      5088      654
CRC:      36      5
ES:      35      2
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 14 hours 19 min 10 sec
FEC:      305710      21339
CRC:      2843      7
ES:      2684      3
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      473605      33180
CRC:      4220      21
ES:      3996      14
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 2 days 22 hours 12 min 29 sec
FEC:      1364442      118752
CRC:      11734      57
ES:      11099      41
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0




Current SNR vs tones for modem 4:

(https://i.postimg.cc/WpqCR9JD/563-F836-A-5-B21-49-BE-B250-1-AB53-F8-D1-CA5.jpg)



Bits per bin for modem 4:

(https://i.postimg.cc/J0X2bnW3/507-DC03-A-5819-4-E43-8-EE9-6-F589-D9626-F2.jpg)

Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 12, 2020, 04:22:01 PM
There is more weirdness going on too with modem 4: the numbers of FEC errors (corrected error events iirc?) and CRC errors (uncorrected errors) are radically different something like 10 times higher number of FEC errors, very roughly, if I can read the graphs correctly. Just look at the y axis scale and reel in amazement. It means that modem 4 is working really really hard, successfully correcting a lot of errors, whereas modem 3 is having an easier time of it. Pictures follow, comparing modems 4 and 3.

Modem 4 FEC event rates over time :

(https://i.postimg.cc/Y9wP1P7G/FA5-B16-B3-D838-42-DB-BA7-F-F98678-A78-E9-B.png)

I think we said this was events per unit time where the unit of time was 30 s ? Or was it one minute ? I think the former and I think our friend kitizen Johnson told me a little while ago, but I’ve managed to forget again thanks to all the NHS’s fine products. (Sorry Theo for being so dim.  :( )



This is modem 3 FEC count. See modem 4 had, what, 100-200 events / unit time rather than the 3-15 events we have for modem 3 here.

(https://i.postimg.cc/2541c1sc/E7-E58967-ED87-4-D4-F-A0-E6-73-A068-A1-C57-C.png)



Modem 4 CRC event count:

(https://i.postimg.cc/YCN48GKX/0-BA173-D5-E497-4-BCF-B7-C5-BB51-C3453-C21.png)



Modem 3 CRC count. Pretty much zero errors most of the time; compare this with the picture of several CRC errors every time period shown with modem 4 previously.

(https://i.postimg.cc/W4W06FgB/7983624-A-C8-BA-4699-9879-45-F37-E0-D3-C7-F.png)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: j0hn on July 12, 2020, 06:01:24 PM
Can you post the "info --stats" output of modem 3?
You've already posted modem 4 in the previous post.

There's definitely a noticeable difference between the 2 lines, particularly the rate of CRC
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 13, 2020, 01:51:00 AM
Here’s modem #3’s detailed stats for comparison with modem #34 (https://forum.kitz.co.uk/index.php/topic,24614.msg419225.html#msg419225) as you asked.

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 305 Kbps, Downstream rate = 3096 Kbps
Bearer:   0, Upstream rate = 376 Kbps, Downstream rate = 2883 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    2.7       6.3
Attn(dB):    65.5       41.3
Pwr(dBm):    18.2       12.4

         ADSL2 framing
         Bearer 0
MSGc:      52      11
B:      33      5
M:      4      16
T:      3      8
R:      10      14
S:      1.4841      8.0000
L:      787      110
D:      2      4

         Counters
         Bearer 0
SF:      20005969      375886
SFErr:      14      18
RS:      870259670      3664793
RSCorr:      23338      7194
RSUnCorr:   38      0

ReXmt:      2044      0
ReXmtCorr:   1927      0
ReXmtUnCorr:   41      0

         Bearer 0
HEC:      38      21
OCD:      0      0
LCD:      0      0
Total Cells:   2211224491      288296203
Data Cells:   94360009      14883067
Drop Cells:   0
Bit Errors:   2040      2344

ES:      134      64
SES:      0      0
UAS:      108      108
AS:      325158

         Bearer 0
INP:      26.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      16.13      17.00
OR:      28.74      8.00
AgR:      2899.49   382.50

Bitswap:   87169/87182      4126/4126

Total time = 16 days 18 hours 53 min 34 sec
FEC:      46536      40526
CRC:      211      72
ES:      134      64
SES:      0      0
UAS:      108      108
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 15 minutes time = 8 min 34 sec
FEC:      30      1
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      29      7
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 18 hours 53 min 34 sec
FEC:      3218      887
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      8376      2020
CRC:      14      8
ES:      2      8
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 3 days 18 hours 19 min 17 sec
FEC:      23338      7194
CRC:      14      18
ES:      2      17
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 27, 2020, 06:30:52 PM
It has gone incredibly bad again. Saturday morning everything went to hell. Have emailed AA who unfortunately want me to swap modems over, which I can’t do on my own of course. Have some gruesome screenshots from the modems via johnson easy stats-graphing. To tired to write further or post them up just now.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 28, 2020, 11:25:23 AM
Stupidly, I didn’t save any stats data just now and currently I can’t get any for your delectation. Janet has just swapped the modems 3 and 4 over at AA’s request, and now modem 3 - into line 4 - is down (line 4 is, that is). So just now I can’t ask the modem what the stats are as we’re not in sync in the showtime state. I will grab some for you when the line decides to come back up.

Scenes of horror from the last couple of days follow:



Look at this nightmare now - SNRM versus tones / frequency :

(https://i.postimg.cc/W1Y01dNX/588-F4066-DD85-406-E-82-DA-6-E433019-E7-C0.png)



SNRM - Upstream is in green, downstream in blue:


(https://i.postimg.cc/T1NwGDG6/89-EBBBDC-70-BC-41-A7-BE4-F-5812-AD670-C46.png)



FEC errors (corrected):

(https://i.postimg.cc/t4Kzp1CW/58-A41-C64-4047-4-E3-E-B7-AB-327990-A1959-C.png)



CRC errors (uncorrected):

(https://i.postimg.cc/SQVFR49J/9-E661548-EFB6-4390-A662-2-C23-B4-BC96-E8.png)



Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Alex Atkin UK on July 28, 2020, 05:34:22 PM
Yikes.  Its so weird that such extreme interference would happen to just one line.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 28, 2020, 08:03:40 PM
I’m thinking that the evidence of the SNRM-vs-tones graph with the hollow bowl where the hilltop should be means there’s an enhanced pickup of rf interference in those frequencies. Non linearity, poor balance I don’t know. What do our sages think?

AA is sending an engineer out tomorrow.

Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on July 28, 2020, 08:21:42 PM
I’m thinking that the evidence of the SNRM-vs-tones graph with the hollow bowl where the hilltop should be means there’s an enhanced pickup of rf interference in those frequencies. Non linearity, poor balance I don’t know. What do our sages think?

I am unsure. To be honest, I have never seen such an effect prior to your line(s) displaying such peculiar behaviour.

Also I can't think of any way to mimic such behaviour in my test set-up.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 28, 2020, 08:38:08 PM
I think something really odd is going on. "Line 3" is now down and line 4 is back up to full speed and it’s SNR vs tones graph looks normal. That says to me that Janet switched modems 3 and 4 over when asked by AA and hasn’t switched them back, but why "line 3" really line 4 was then down is a mystery. The test program that I write which queries the firebrick and gets the status of all the lines says that line 4 is the correct BBEU ID now though - it has a table of expected values and checks them off to detect modems plugged into the wrong wall sockets. So I’m totally confused. I asked Janet to put the modems back to the way they should be when AA’s test session had had a while to complete. It looks like i’m seeing line 3 into modem 4 - good SNRM vs tones curve and high speed and dead modem 3 line - presumed line 4, so they haven’t been swapped over, but then how can the BBEU check now be succeeding? The program has been tested and it has detected cross connected modems successfully before.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 28, 2020, 09:32:55 PM
Janet admits she knocked the power off on modem 3. now line 4 is all good - SNRM curve back to normal shape and sync rate high at 2.9Mbps d/s. What on earth?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on July 28, 2020, 10:21:21 PM
Totally puzzled and confused.  ???
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on July 29, 2020, 12:45:57 AM
The tones vs SNRM curve looks beautiful, and the speed is very good. I wonder if there is anything that AA might have unwittingly or otherwise done remotely, giving the line a tweak as part of a diagnostic procedure or some other procedure that can serve as a temporary bandaid and cure a fault for a while. (Thinks of wetting current making faults go away or at least covering them up - as the basis for an analogy, not literally.) But to cure a fault so completely … ?  ???   ???   ???
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 01, 2020, 03:15:51 AM
There’s that slight hollow back again now in the SNR vs tones graph. A couple of noise spikes sent the downstream target SNRM right up to 6dB and so it going fairly slowly downstream just now. It just seems to be susceptible to big spikes at the moment and every few days somethings happening with this line.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 19, 2020, 01:55:14 PM
It just goes on and on. Downstream at 2.12 Mbps this morning at SNRM 1.9 dB. This still qualifies as an FTR, and to their credit, AA are booking an engineer for me.  ;D ;D ;D
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on August 19, 2020, 06:35:50 PM
Downstream at 21.2 Mbps this morning at SNRM 1.9 dB.

Isn't that the best DS speed you have ever achieved in "The Weaving Shed"?  :-\
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 19, 2020, 11:51:51 PM
Oops. Should of course read 2.12Mbps.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Alex Atkin UK on August 20, 2020, 12:03:02 AM
I was thinking, they suddenly built a new cabinet/exchange for that line?  :lol:  (if only)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on August 20, 2020, 12:16:10 AM
Oops. Should of course read 2.12Mbps.

Ah, sorry. I didn't realise that the decimal point had been shifted to the right.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 21, 2020, 12:20:06 PM
AA has now booked an engineer - hurrah - at long last!  :(  :)  :(  Let’s hope this visit does the trick.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 22, 2020, 11:00:58 AM
Engineer turned up just after 08:00. Said he can see a fault on the line - hurrah :yay:  :thumbs:  :congrats:  and the fault was located; it is something bad at the top of the pole by the house. The pole leans a bit and has a red thing on it saying ‘dodgy’ or something- so Janet tells me. Our man did not have climbing gear with him, or some such thing so couldn’t finish the job. He said someone will be back on Monday or Tuesday.

I’m very pleased as loads of times either I or AA support have not been able to see any fail reported in the standard BTOR tests, although of course if they only were to look at the SNRM vs tones graphs they would see the problem reflected there.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on August 22, 2020, 05:24:06 PM
The pole leans a bit and has a red thing on it saying ‘dodgy’ or something- so Janet tells me. Our man did not have climbing gear with him, or some such thing so couldn’t finish the job. He said someone will be back on Monday or Tuesday.

That will be an embossed D on a red plate, nailed to the pole, signalling danger. The pole should eventually be replaced but, until that occurs, any pole-top access will have to be via a hydraulic lift.

Am I understanding correctly that it is the higher level pole, up at the house level, and not the lower level road-side pole?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault aga
Post by: Weaver on August 24, 2020, 01:33:06 PM
Ah, asked Janet which pole she meant. Today, Monday, our man went up the lower pole by the far side of the road, further away from the house. I think I misunderstood before; I was thinking that it cannot be the pole near the house as there are afaik no joints on it.

Finally, fixed !

Anyway, second man came out around midday and fixed the problem at that pole. The line 4 is now running at 2.832 Mbps d/s @ 3.2 dB SNRM, 550kbps upstream @ 5.8 dB SNRM. Here’s the beautiful modem 4 SNR vs tones graph:

(https://i.postimg.cc/Mp6rS1n4/AAEF7-BB5-3722-4-B2-E-8548-523-AF44-DF88-D.png)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault aga
Post by: burakkucat on August 24, 2020, 03:51:17 PM
Ah, asked Janet which pole she meant. Today, Monday, our man went up the lower pole by the far side of the road, further away from the house. I think I misunderstood before; I was thinking that it cannot be the pole near the house as there are afaik no joints on it.

Obviously I only have a "snapshot in time" to view and then only at the lower, road-side, pole. That is the one with a BT66 at the top, to house the joints between the two aerial drops (from "The Weaving Shed") and the tail, running down the pole, linking to the D-side cables.

Quote
Finally, fixed !

Anyway, second man came out around midday and fixed the problem at that pole. The line 4 is now running at 2.832 Mbps d/s @ 3.2 dB SNRM, 550kbps upstream @ 5.8 dB SNRM.

Excellent news. (Let's hope that the haggis do not decide to climb the pole and gnaw on the freshly re-made joints.  ::)  )
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 24, 2020, 04:12:31 PM
I’m afraid that in the past we have had evidence that haggises can climb poles.  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 25, 2020, 01:10:32 PM
Notes received by AA from first engineer, on saturday:

Quote
Note Engineer could not complete the work. As per his notes "I cannot complete this task because of a hazard owned by Openreach / third party. Unable to climb pole at dp72 as I am in a loan van and do not have my climbing harness is preventing further work. Please see additional information: Pair is faulty from joint feeding dp72 faulty towards Eu; unable to climb pole as I am in loan van and no harness to climb. Hence creating CSS Task and assigning the job to SN Team and completing action item. Review by date 26/08/2020."
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on August 25, 2020, 05:11:35 PM
Ah, so that was the reason and not a dangerous pole, as originally thought.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on August 25, 2020, 05:20:05 PM
The dangerous pole must have been Janet thinking about the upper pole - some confusion there anyway.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 09, 2020, 10:50:50 AM
And here we go again. The fix didn’t last; the fault has come back on line 4 (and line 2 - the most recently installed).


xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 515 Kbps, Downstream rate = 2552 Kbps
Bearer:   0, Upstream rate = 560 Kbps, Downstream rate = 2310 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    6.1       5.9
Attn(dB):    65.0       41.5
Pwr(dBm):    18.1       12.4

         ADSL2 framing
         Bearer 0
MSGc:      52      13
B:      26      62
M:      8      1
T:      3      1
R:      16      14
S:      2.9414      3.5402
L:      631      174
D:      1      8

         Counters
         Bearer 0
SF:      752848      719419
SFErr:      0      1
RS:      16374447      770625
RSCorr:      601      7297
RSUnCorr:   0      0

ReXmt:      707      0
ReXmtCorr:   707      0
ReXmtUnCorr:   0      0

         Bearer 0
HEC:      0      0
OCD:      0      0
LCD:      0      0
Total Cells:   65909686      15978087
Data Cells:   679324      183175
Drop Cells:   0
Bit Errors:   0      722

ES:      644552      1332
SES:      1771      1
UAS:      64990      63853
AS:      12095

         Bearer 0
INP:      26.00      2.50
INPRein:   0.00      0.00
delay:      8      7
PER:      15.99      16.81
OR:      29.01      9.03
AgR:      2329.95   567.23

Bitswap:   487/487      47/47

Total time = 74 days 13 hours 28 min 11 sec
FEC:      26852419      2238836
CRC:      947821      1308
ES:      644552      1332
SES:      1771      1
UAS:      64990      63853
LOS:      83      0
LOF:      438      0
LOM:      51      0
Latest 15 minutes time = 13 min 11 sec
FEC:      40      474
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      45      507
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 13 hours 28 min 11 sec
FEC:      9550      10960
CRC:      484      1
ES:      18738      1
SES:      10      0
UAS:      119      109
LOS:      1      0
LOF:      9      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      0      0
CRC:      0      0
ES:      25757      9
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 3 hours 21 min 35 sec
FEC:      601      7297
CRC:      0      1
ES:      0      1
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 09, 2020, 03:54:04 PM
I must be having another bad day.  :(

Please enlighten me by explaining what I am supposed to be seeing when I look at that latest statistics output. For example, I see --

ES:      644552      1332
SES:       1771         1

But that appears to be spread over --

Total time = 74 days 13 hours 28 min 11 sec
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 09, 2020, 04:48:58 PM
You are not the only one.  :)  What perplexes you? Are those values implausible?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 09, 2020, 05:11:24 PM
A block of data, such as the above, really tells nothing. Unless there is a reference data set with which it can be compared.

I presume you have looked at the various plots and are seeing an "eroded summit", once again? Do you experience a problem when just using the service?

Hmm . . . Not sure what else I can really type.  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 11, 2020, 07:48:14 AM
It’s back to the same hollow at the summit, yes. The loss of downstream sync rate an as bad as it as before (not yet anyway).

for comparison, here’s modem 3:

xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason:   8000
Last initialization procedure status:   0
Max:   Upstream rate = 312 Kbps, Downstream rate = 3200 Kbps
Bearer:   0, Upstream rate = 376 Kbps, Downstream rate = 2861 Kbps

Link Power State:   L0
Mode:         ADSL2 Annex A
TPS-TC:         ATM Mode(0x0)
Trellis:      U:ON /D:ON
Line Status:      No Defect
Training Status:   Showtime
      Down      Up
SNR (dB):    3.2       6.1
Attn(dB):    65.0       41.2
Pwr(dBm):    18.2       12.4

         ADSL2 framing
         Bearer 0
MSGc:      53      11
B:      32      5
M:      4      16
T:      3      8
R:      10      14
S:      1.4508      8.0000
L:      783      110
D:      2      4

         Counters
         Bearer 0
SF:      237508496      852465
SFErr:      4587      176
RS:      1919816364      4293165
RSCorr:      1011080      99952
RSUnCorr:   23782      0

ReXmt:      520206      0
ReXmtCorr:   473853      0
ReXmtUnCorr:   25343      0

         Bearer 0
HEC:      23603      170
OCD:      576      0
LCD:      576      0
Total Cells:   140982884      3404393386
Data Cells:   1332365539      178176483
Drop Cells:   0
Bit Errors:   662586      18580

ES:      2419      161
SES:      3      0
UAS:      53      53
AS:      3839699

         Bearer 0
INP:      26.00      2.00
INPRein:   0.00      0.00
delay:      8      8
PER:      16.04      17.00
OR:      29.40      8.00
AgR:      2878.13   382.50

Bitswap:   939067/939151      48693/48693

Total time = 44 days 10 hours 35 min 52 sec
FEC:      1011080      99952
CRC:      4587      176
ES:      2419      161
SES:      3      0
UAS:      53      53
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 15 minutes time = 5 min 52 sec
FEC:      3      0
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 15 minutes time = 15 min 0 sec
FEC:      5      5
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Latest 1 day time = 10 hours 35 min 52 sec
FEC:      691      285
CRC:      0      0
ES:      0      0
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Previous 1 day time = 24 hours 0 sec
FEC:      1955      1665
CRC:      5      6
ES:      3      6
SES:      0      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
Since Link time = 44 days 10 hours 34 min 57 sec
FEC:      1011080      99952
CRC:      4587      176
ES:      2419      161
SES:      3      0
UAS:      0      0
LOS:      0      0
LOF:      0      0
LOM:      0      0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0


Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 11, 2020, 07:57:25 AM
Here’s the modem 4 SNR-vs-tones graph :

(https://i.postimg.cc/cHTPLT0Z/2-CCC3-AC5-398-C-44-A4-82-F5-6825-D6-ADFDA4.jpg)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 11, 2020, 03:56:23 PM
I wish we could determine the actual, physical, cause of the "eroded summit" effect. Knowing the cause, even if it cannot be mitigated, would help alleviate so much of the head-scratching.  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 11, 2020, 11:55:47 PM
What we do know, for certain, is that BTOR can see the fault on their JDSUs or whatever, and can fix it successfully, but only for a while. When they have fixed it it just comes back at some point, but I’m not sure about the level-vs-time reappearance of the fault. I have never seen anything like it before this summer. Line 2 and line 4 are the two pairs in the same drop cable, btw.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 12, 2020, 12:14:54 AM
Line 2 and line 4 are the two pairs in the same drop cable, btw.

The newer aerial drop of the two which rise from the low-level, road-side, pole. (I am amused by drops which actually rise.) However I do not think that the two lines, which intermittently show the problem, being in the same drop cable is significant. That is to say I think it is just a co-incidence.

The head-scratching continues.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 12, 2020, 05:56:23 PM
I’m wondering about when to keep reporting these two lines; as they get slower there comes an arbitrary point where I choose to report them as ‘bad’ (sufficiently). I hate to keep on bothering AA again and again, like a cracked record. But the sooner I report them, the less time I have to put up with reduced downstream speed.

What do you think is reasonable ? Wait till a line becomes an FTB ?

I can’t remember what the percentage is - Kitz will tell me somewhere - but I don’t have to remember as clueless puts it up in your face - at the very top of its main system state display page it displays a list of any lines that are in breach of the fault threshold rate, so as to alert you.

My neighbour has built a house next door to me and has presumably put in phone lines. I wonder if that is the source of all of these problems?

How would you physically run new lines into a new dwelling?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 22, 2020, 11:35:22 AM
Here we go again: downstream sync rate of line 4 down to 1.278 Mbps. Have contacted AA


(https://i.postimg.cc/tTXXLdpm/B7-BCE253-2111-41-F9-BAE5-F5-EBF936-BF42.png)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 22, 2020, 12:20:15 PM
It wasn’t reliable at 3 dB SNRM downstream, so I changed it to 6 dB; that brought the downstream sync rate down to 0.963 Mbps  :( :'(

Code: [Select]
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 515 Kbps, Downstream rate = 1068 Kbps
Bearer: 0, Upstream rate = 560 Kbps, Downstream rate = 963 Kbps

Link Power State: L0
Mode: ADSL2 Annex A
TPS-TC: ATM Mode(0x0)
Trellis: U:ON /D:ON
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 5.7 6.0
Attn(dB): 65.5 41.6
Pwr(dBm): 18.0 12.4

ADSL2 framing
Bearer 0
MSGc: 59 13
B: 30 62
M: 4 1
T: 1 1
R: 16 14
S: 3.9576 3.5402
L: 283 174
D: 1 8

Counters
Bearer 0
SF: 11791 11483
SFErr: 12 2
RS: 191604 217969
RSCorr: 848 429
RSUnCorr: 12 0

ReXmt: 1652 0
ReXmtCorr: 1651 0
ReXmtUnCorr: 13 0

Bearer 0
HEC: 6 7
OCD: 0 0
LCD: 0 0
Total Cells: 433813 251913
Data Cells: 11612 7587
Drop Cells: 0
Bit Errors: 1023 0

ES: 805889 1593
SES: 1862 1
UAS: 131703 130487
AS: 194

Bearer 0
INP: 27.00 2.50
INPRein: 0.00 0.00
delay: 8 7
PER: 16.07 16.81
OR: 32.34 9.03
AgR: 990.67 567.23

Bitswap: 30/30 5/5

Total time = 87 days 15 hours 2 min 27 sec
FEC: 34198733 2870825
CRC: 1347699 1668
ES: 805889 1593
SES: 1862 1
UAS: 131703 130487
LOS: 90 0
LOF: 481 0
LOM: 51 0
Latest 15 minutes time = 2 min 27 sec
FEC: 666 417
CRC: 9 2
ES: 9 1
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 15 minutes time = 15 min 0 sec
FEC: 15648 872
CRC: 1335 4
ES: 540 3
SES: 11 0
UAS: 60 49
LOS: 1 0
LOF: 9 0
LOM: 0 0
Latest 1 day time = 15 hours 2 min 27 sec
FEC: 277085 31655
CRC: 12709 6
ES: 6854 4
SES: 21 0
UAS: 120 99
LOS: 2 0
LOF: 18 0
LOM: 0 0
Previous 1 day time = 24 hours 0 sec
FEC: 115311 63313
CRC: 637 28
ES: 656 22
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Since Link time = 3 min 13 sec
FEC: 848 429
CRC: 12 2
ES: 12 1
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 22, 2020, 05:41:31 PM
It wasn’t reliable at 3 dB SNRM downstream, so I changed it to 6 dB; that brought the downstream sync rate down to 0.963 Mbps  :( :'(

Remember that the default target SNRM is 6 dB, hence the circuit's degraded performance should be reported relative to that target and not with your tweaked setting.

Looking at it slightly differently, with the default 6 dB SNRM target the circuit is clearly faulty and so, hopefully, Donald, Hamish or Malcolm will resolve the problem quite quickly. (Until the next attack by the haggis on the cables or joints.)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 22, 2020, 09:55:28 PM
I usually keep everything at 3dB so as to compare like with like. Perhaps I should change to 6dB when reporting sync rates to AA.

It wasn’t happy at 6dB downstream SNRM either, not completely, so now it’s up to 9dB downstream and a miserable 598k downstream sync rate.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 22, 2020, 10:09:00 PM
I usually keep everything at 3dB so as to compare like with like. Perhaps I should change to 6dB when reporting sync rates to AA.

Yes, that would be sensible.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 23, 2020, 10:34:29 AM
AA got back to me. They are going to book an SFI engineer and get the drop cable replaced. Lines 4 and 2, the two problem lines, share that drop cable.

I have heard the term SFI engineer so many times, but nevertheless I have no idea what an SFI engineer does or what their expertise is. Can someone give me a definitive explanation ?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 23, 2020, 02:32:53 PM
Openreach coming on the 25th
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 23, 2020, 04:07:51 PM
AA got back to me. They are going to book an SFI engineer and get the drop cable replaced. Lines 4 and 2, the two problem lines, share that drop cable.

Ah, now that is interesting. I wonder if the fault is due to that very close lightening strike earlier this year?

Quote
I have heard the term SFI engineer so many times, but nevertheless I have no idea what an SFI engineer does or what their expertise is. Can someone give me a definitive explanation ?

Special Fault Investigation and Special Fault Investigation 2

Here follows a brief list (in chronological order) of forum threads where SFI (or SFI2) is mentioned --

May 03, 2015 (https://forum.kitz.co.uk/index.php/topic,15432.msg286935.html)
July 15, 2015 (https://forum.kitz.co.uk/index.php/topic,15758.msg293254.html)
August 17, 2016 (https://forum.kitz.co.uk/index.php/topic,18296.msg330732.html)
January 15, 2017 (https://forum.kitz.co.uk/index.php/topic,19198.msg340905.html)
September 08, 2020 (https://forum.kitz.co.uk/index.php/topic,25146.msg421871.html)

I think you might recognise the contributors to the above.  :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 23, 2020, 04:19:07 PM
My memory frequently embarrasses me; it’s shocking. The past is all simply - gone. Looking back, I didn’t understand our Sheep’s post.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 24, 2020, 04:50:13 PM
Looking back, I didn’t understand our Sheep’s post.

He was just detailing the basic steps which apply when visiting an end-user, on behalf of an ISP/CP who has ordered the SFI service.

One thought came to me overnight. I believe the "high level" pole was classified as dangerous at its last inspection (hence the D label affixed to it) and is listed for eventual replacement. As a consequence, any pole-top access will have to be done from the platform of a hydraulic hoist. So the replacement of the second drop cable will need to be a two person task.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 25, 2020, 07:42:41 PM
I didn’t understand what the skills and abilities of an SFI engineer are.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 25, 2020, 10:19:04 PM
Burrakucat- you were spot on about the D - the engineer moaned about that fact. Well why didn’t they give him a hoist? They must know. I have mentioned the D to AA in case it holds things up on some future occasion.

The moaning engineer said there was nothing wrong with the line. He put in his conclusion: "internet congestion". I’m lost for words!

 :drink:  However, he fixed both line 4 and line 2 some how (even though not booked to fix line 2). Speeds are now superb on both lines, and SNR curves are boringly normal. I shall await the notes coming back.

Live sync rates:
  #1: down 2765 kbps, up 522 kbps
  #2: down 2865 kbps, up 515 kbps
  #3: down 2979 kbps, up 379 kbps
  #4: down 2881 kbps, up 557 kbps
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 25, 2020, 11:14:23 PM
I didn’t understand what the skills and abilities of an SFI engineer are.

Oh, I see. The answer is quite simple . . . Just clone Black Sheep, divide the clone into two and then you have two SFI engineers.  ;)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 25, 2020, 11:32:15 PM
Here is another one of my "overnight" thoughts.

Certain lines are occasionally damaged by nearby lightning strikes. Just like the big zap, some months ago, which resulted in the replacement of (an) underground cable(s) at Harrapul. I believe one (or more) of your four lines required some of their joints to be remade, as the gel-crimps were smoke-blackened, nasty and defective.

We know that a basic NTE5 contains a series connected 1.8 micro Farad capacitor and 470k Ohm resistor as a shunt across the pair. When a line has suffered lightning damage, I would be very suspicious of the condition of both the capacitor and the resistor in the NTE5. So much so that I would be inclined to replace the NTE5 . . .  :-X
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 26, 2020, 11:24:07 AM
I understand. I can’t replace them myself and Janet’s hands aren’t good enough. Is that a job for AA?

I do get very good results on the lines. When they’re working properly, that is. Does that tell us anything regarding the state of the components?

I don’t know if a local electrician would want to touch the forbidden territory that is the NTE5?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 26, 2020, 06:36:04 PM
I only mentioned the line shunt components as they could, if damaged, cause peculiar circuit behaviour.

If all four lines are now well-behaved then it would be best to leave them alone. If, in the future, one becomes faulty then certainly ask A&A to request an NTE5 check as part of the fault-finding process.

To be absolutely correct, in terms of radio frequency transmission lines, you have no need for the shunt across the pair. The shunt is, er, detrimental. But it does provide a "signature" that can be "read" by a remote test set.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 26, 2020, 08:27:03 PM
Oh I see. And if the series DC blocking capacitor is dead, then you’d really know about it. Or is there a more subtle fault mode.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 26, 2020, 09:20:59 PM
I would expect the capacitor to go open circuit, probably by exploding, and the resistor to, perhaps, be vaporised at one extreme. The other extreme would be a hard short-circuit.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 27, 2020, 02:13:27 PM
That’s what I thought. Has any of our members had degraded performance or problems from a dodgy or blown NTE5 ?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 28, 2020, 03:34:14 PM
@Burakkucat : AA said that engineers will check NTE5s when they come into the house.

I still haven’t seen the engineer’s notes from that visit.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on September 28, 2020, 04:47:08 PM
AA said that engineers will check NTE5s when’s they come into the house.

Perhaps substitute a "should" for the "will" and an "if" for the "when's"?  :)

Quote
I still haven’t seen the engineer’s notes from that visit.

Maybe it was written in invisible ink.  :-\
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on September 28, 2020, 09:58:36 PM
In the past, engineers’ notes have always been posted onto AA’s clueless.aa.net.uk server and appear in the log for the appropriate line. I don’t know if this is automatic; I expect that it is; not sure exactly why I get that feeling.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on October 24, 2020, 11:33:04 PM
Here we go again; lightning strike 7 miles away and the four lines blipped and resynched at around 1620 UTC. I was asleep at the time they resynched and then we heard thunder and pulled the modem cables from the wall sockets. When I woke up later on, I noticed that downstream sync rates were really low on lines 2, 3 and 4. I performed a SNR reset on these lines using clueless.aa.net.uk and forced a resynch on each of the modems. The downstream speeds were still disappointingly low again.

And guess what, the culprit is evidently the hollow downstream SNR vs frequency phenomenon on lines 2, 3, 4. So I’ll have to contact AA.

Live sync rates, all at 3dB d/s 6dB u/s SNRM:
  #1: down 2876 kbps, up 529 kbps
  #2: down 2560 kbps, up 522 kbps
  #3: down 2080 kbps, up 386 kbps
  #4: down 2026 kbps, up 627 kbps

Notice the exceptionally high upstream speed of line #4 - weird. ???
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on October 27, 2020, 06:15:13 AM
Since then line 2 downstream rate has crashed to 178kbps :o

Lines 3 and 4 have both further deteriorated markedly.

Live sync rates:
  #1: down 2876 kbps, up 529 kbps
  #2: down 178 kbps, up 512 kbps
  #3: down 1867 kbps, up 389 kbps
  #4: down 1046 kbps, up 563 kbps
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on October 27, 2020, 04:22:24 PM
If only you could have electrically non-conductive links . . .  :-X
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on October 27, 2020, 05:20:05 PM
Indeed. I really would love to work out how to do the fibre media converter thing, but times 2 times 4 is a pain.

I pulled out the modems from the wall when I heard a thunder rumble; but I didn’t hear a beep from the hardware lightning detector on the wall rack by my bed - its beeper is far too feeble - a big design failing, and there’s no volume control either  :(  I missed the alerts from the app on my ipad because I was asleep, but I did see the app alert notification later on sitting there saying strike at 6.9 miles (or some such) but I didn’t carefully peruse the earlier alerts.

I ought to get rid of the OR Mk4 SSFP on line 2 (being the newest one) as that’s another thing to go wrong, can get cooked. Or failing that, have a spare on site.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on October 27, 2020, 05:41:59 PM
I really would love to work out how to do the fibre media converter thing, but times 2 times 4 is a pain.

I have been thinking about that . . . Ideally you would like the protection before the ZyXEL modems but the best I can come up with is between the modems and the Firebrick.

(I really need to document my current thoughts for you. It's the time that process will take . . . )
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on October 27, 2020, 05:49:39 PM
And the latter scheme would be more convenient and would protect the expensive kit, the ZyXEL modems being relatively cheap.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on October 31, 2020, 02:17:16 AM
AA sent two Openreach engineers on Friday morning. Lines 3 and 4 are now perfect but line 2 is not fixed - still slow downstream. They are just having an awful time locating these faults. The engineers’ notes aren’t back yet.

Live sync rates, all at 3dB downstream / 6dB upstream SNRM :
  #1: down 3015 kbps, up 522 kbps
  #2: down 2343 kbps, up 519 kbps
  #3: down 2844 kbps, up 386 kbps
  #4: down 3012 kbps, up 557 kbps



Line 2 tonight, after engineers’ visit:

(https://i.postimg.cc/Ssq27QRN/FC1849-BA-8588-453-C-A18-A-2-AA1816336-C7.png)



Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 02, 2020, 10:41:21 PM
BT-speak: could anyone tell me what LLUMS stands for ?

I just got the engineers’ notes back from AA; I can’t make anything much out of them, they seem to be fairly detail-free. This is mentioned in some correspondence with [?]BTW.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Alex Atkin UK on November 03, 2020, 12:31:50 AM
Just to pipe in, it wont let me respond to your PM it says:
User 'Weaver' has blocked your personal message.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 03, 2020, 04:55:04 AM
Oops, sorry about that.  :-[  Should be fixed now.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 03, 2020, 05:37:31 AM
AA staff have booked a second engineer. Hopefully a bit better ?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 03, 2020, 04:15:34 PM
BT-speak: could anyone tell me what LLUMS stands for ?

Not something that I have previously seen and it's not in the "L-section" of the Glossary (https://forum.kitz.co.uk/index.php/topic,4620.0.html).

Pure guess-work; something like "Local Loop Unbundled Management System" or "Local Loop Unbundled Maintenance System"?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 03, 2020, 10:22:02 PM
Engineer coming on Wednesday morning.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 03, 2020, 11:04:40 PM
Engineer coming on Wednesday morning.

Hopefully a SFI2 and not a plain network (i.e. telephony) engineer. No criticism of the last one to attend but his remit was just to prove the physical infrastructure was correct and that your modem achieved synchronisation with the DSLAM.

If only "The Cattery" was located in Harrapul . . .  :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 05, 2020, 12:11:20 AM
Weird. It was the same old engineer, who told Janet that there was absolutely nothing wrong with the line and that it was her modem (despite the fact that modems were swapped before the first callout). And guess what - the problem has now been fixed!

Live sync rates:
  #1: down 3015 kbps, up 522 kbps   3 dB downstream ; 6dB upstream SNRM
  #2: down 2577 kbps, up 529 kbps   6 dB downstream ; 6dB upstream SNRM
  #3: down 2844 kbps, up 386 kbps   3 dB downstream ; 6dB upstream SNRM
  #4: down 3012 kbps, up 557 kbps   3 dB downstream ; 6dB upstream SNRM

So line 2 will go a lot faster when I adjust the downstream SNRM to be 3dB, the same as the others.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 05, 2020, 01:55:37 AM
Yet again, engineers come out, they say they have found nothing and don’t claim to have made any repairs yet the problem goes away. Is there some action which is part of a test procedure that could fix a fault?

Line 2 SNRM-vs-tones post-fix:

(https://i.postimg.cc/HnSx8gTT/FE360-A0-C-9984-46-B8-B54-A-47-F6367163-E7.png)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 05, 2020, 11:03:24 AM
Better than we’ve been in a long while now: perhaps the summer winter thing is really kicking in now

All lines at 3dB downstream SNRM, 6dB upstream SNRM :

Live sync rates:
  #1: down 3015 kbps, up 522 kbps
  #2: down 2964 kbps, up 563 kbps
  #3: down 2844 kbps, up 386 kbps
  #4: down 3012 kbps, up 557 kbps
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 23, 2020, 12:35:56 PM
And guess what, in the last few days it has started up again. Within line 4 this time, which has dropped from d/s sync rate of 3 Mbps to 2.1. There was bad packet loss so I had to increase the d/s target SNRM from 3dB to 6dB which fixed the problem. Line 4 dropped this morning with a massive error burst downstream which knocked it out. I’ve told AA who are presumably overjoyed. SNR-vs-tones graph for line 4 looks sickly, the same as before, same downward hollow where the highest point downstream should be.

My lines must have got COVID-19.  ??? I never had any such faults before this year. Mind you, I also had that major lightning strike.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 23, 2020, 06:21:37 PM
My lines must have got COVID-19.  ??? I never had any such faults before this year. Mind you, I also had that major lightning strike.

  :(

[I wonder if that near strike, up on the high moor, both antagonised & super-charged the haggis and they have been gaining their revenge ever since.  :-\  ]
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 23, 2020, 08:25:40 PM
Sounds like a distinct possibility.  ;)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 25, 2020, 07:25:54 AM
Some pretty pictures to show the state of line 4. The link has now halved in downstream speed, from a miserable ~2Mbps to an appalling 0.97 Mbps because a burst of errors pushed the downstream SNRM up to something like 10.4dB and the link dropped. Here below is the SNRM history around the time of the link drop:

(https://i.postimg.cc/Nfh66f8Z/09-C651-B7-EF61-4-F07-9-FC2-C88-B886-BB730.png)



Here below is the CRC count per unit time vs time ie uncorrected errors [?] around the time of the link drop:

(https://i.postimg.cc/4x0NcnV9/93-ECDAEF-5-E55-41-BD-82-E8-813-C89-E30-B08.png)



Here’s the line 4 QLN which is horrible, with lots of abnormal noise spikes showing. I am assuming these are radio stations? The problem in the line is therefore a failure of noise rejection, is that correct?

(https://i.postimg.cc/NGNhFS1y/AE3-AAFCE-921-B-4997-8315-5-E7485-B36-F5-B.png)



Here’s line 3’s good QLN for comparison - all quiet.

(https://i.postimg.cc/W3CH6LSS/71-AADC5-C-5-DAB-4-B1-F-8-E27-45791263-EFFB.png)

So there we have some idea of what’s going on, but the question is what’s causing it?


Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 26, 2020, 06:30:01 AM
AA have booked an engineer for this morning.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 26, 2020, 06:27:00 PM
Very grumpy OR engineer turned up this afternoon having come all the way from Ùige and refused to do anything more than a voice line quality check, so a complete waste of time. Now I have an angry wife, blaming me, blaming AA and anyone else who’ll fit the bill. Between them AA, BTW and OR made a real mess of things. :-(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 26, 2020, 07:46:12 PM
Hmm . . . How about sending andrew-AAISP (https://forum.kitz.co.uk/index.php?action=profile;u=9597) a PM and ask him to review this thread?  :-\
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 26, 2020, 09:43:04 PM
I’m pretty sure a senior colleague has looked into this. I’ve emailed AA with a report, and an earlier email had all the graphs, and another had also a url pointing to Johnson’s github sources and binaries
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 26, 2020, 10:17:52 PM
Ah, I see. So let's strike out my suggestion.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 26, 2020, 10:33:23 PM
No, I’m not ignoring your suggestion, my friend.  :) In fact I think I still have Andrew’s email address.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 26, 2020, 10:39:04 PM
In fact I think I still have Andrew’s email address.

If you were to left-click (yes, I realise an iDevice is different) upon the "andrew-AAISP (https://forum.kitz.co.uk/index.php?action=profile;u=9597)" link you should see options for sending a PM and (possibly) also for sending an E-mail message.  ;)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 28, 2020, 10:47:34 AM
 :drink: :yay: Problem solved, once again, on second engineer visit.

Very knowledgeable and helpful engineer from Gleann Dàil (I think he said) arrived this morning and magically fixed the problem! But once again, we don’t know how: downstream sync rate was around 1Mbps (at I forget what SNRM) but DLM pushing the SNRM up may be part of the issue if I had not reset it recently.

Our engineer told Janet and me that FTTP is being provided in Skye now: in Dùn Tuilm and Bhatairnis at the extreme northern end of the Island and also on the mainland, in Gleann Seile, and he mentioned some thing about supply to 120 houses but I’m unsure where that is to be as my hearing is not good. It almost sounds as if sanity has broken out and the powers that be are doing those with the greatest need?

That has to be some of the best news in ages and he said it will certainly be coming to Heasta. He mentioned pole-mounted distribution nodes, I don’t know if that is instead of FTTP green cabs.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 28, 2020, 11:04:02 AM
Sync rates:
  #1: down 3015 kbps, up 522 kbps
  #2: down 2986 kbps, up 566 kbps
  #3: down 2935 kbps, up 389 kbps
  #4: down 3139 kbps, up 570 kbps

Look at line #4 go! This is a recurring pattern; when the faults get mysteriously fixed, the speed is very high downstream.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on November 28, 2020, 04:35:21 PM
Sync rates:
  #1: down 3015 kbps, up 522 kbps
  #2: down 2986 kbps, up 566 kbps
  #3: down 2935 kbps, up 389 kbps
  #4: down 3139 kbps, up 570 kbps

Look at line #4 go! This is a recurring pattern; when the faults get mysteriously fixed, the speed is very high downstream.

An excellent result. Let's hope it lasts.  :fingers:
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: neil on November 28, 2020, 06:57:46 PM
Sync rates:
  #1: down 3015 kbps, up 522 kbps
  #2: down 2986 kbps, up 566 kbps
  #3: down 2935 kbps, up 389 kbps
  #4: down 3139 kbps, up 570 kbps

Look at line #4 go! This is a recurring pattern; when the faults get mysteriously fixed, the speed is very high downstream.
wow  :o
looks like you are far way from your cabinet?
and you are using adsl
can you please tell me what software you are using for telnet?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 28, 2020, 10:03:45 PM
Hi neil, line 2 is on a cabinet but the other three are exchange-only (EO) ie direct into the exchange. The lines are 7300m long roughly - 4.5 miles.

To telnet into my modems I’m using the Prompt2 app on an iPad.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on November 30, 2020, 08:46:09 AM
I told my engineer about the weird mystery of how no-one knows what it is that is fixing my lines. He said he had seen the 3+ Mbps downstream sync rate when he first came on site, that is, he had not seen the low 1 Mbps reading.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 08, 2020, 10:22:25 AM
Fault on line 2 was reported to AA, they came back and told me immediately that they could see a battery contact fault. I could have checked for that myself but I didn’t think any more to it other than burned faceplate front or modem cable as seen with line 4. See https://forum.kitz.co.uk/index.php/topic,25430.0.html (https://forum.kitz.co.uk/index.php/topic,25430.0.html) This morning an engineer rang at 8 am; he was at the exchange or out on the moor locating the fault. Speed is already up to 2.9 Mbps, should be 3.1 Mbps and can see that the SNR vs tones curve is still not quite perfect but 93% of the way there. The bit-loading graph shows that it is 2 bits short of what it should be on the best downstream tones, should be 12 bits in places, otherwise 11 bits from tones 40 to 80, but in part of that region it isn’t yet even managing ten.

Anyway, a good job so far. Engineer still out there working on it. Really really nice guy; hellish job having to be up before 8am, ugh. Poor fellow, what a hero.



Erm, not such a good end to the tale: having been working, the line is now down!  >:( ???

Notes follow:

2020-12-08 11:11:24      Track PSTN Fault 5-8-183338846151 Informational Message: 4150 Response Required - Fault Report Cleared   bt
2020-12-08 11:11:24      Track PSTN Fault 5-8-183338846151 Estimated Response Time: 2020-12-08T23:59:59   bt
2020-12-08 11:11:24      Track PSTN Fault 5-8-183338846151 Clear: 82.2 In Joint AreaCable (Underground)   bt
2020-12-08 11:11:24      Track PSTN Fault 5-8-183338846151 Status: Open - Clear Unconfirmed

Track PSTN Fault 5-8-183338846151 Notes Field: **Line Stability:**Network Stability:**Test Outcome:Pass**MFL:OK**Term Statement:**Line Signature:**Distance to Fault:0**Cable length:8.16**Test Start Time:2020-12-08T11:11:08**Test Stop Time:2020-12-08T11:11:09
=== QBC Summary Start ===
Customer Report: End customer advised there is a broadband issue.
Actions to resolve: Engineer has resolved the fault located at the D-side including aerial cables / lead-in / block terminal.There was a fault outside the end customer's curtilage caused by general wear and tear.The fault was fixed by clearing in joint.The test passed on 2020-12-08T10:40:52.
Final alternate test results: Final FastTest performed from the customer premise. The test passed on 08/12/2020 11:01:28.=== QBC Summary End ===



The bit about "Final alternate test results: Final FastTest performed from the customer premise." is a complete lie as he never came to the house, or I would have been able to say, where’s my DSL gone?

Very nice guy though, very friendly, just a bit daft.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 08, 2020, 02:17:53 PM
Have notified AA that I now no longer have any DSL.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 08, 2020, 06:43:01 PM
 :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 08, 2020, 11:24:20 PM
AA just asked me to do a quick sanity test with a swap of modems, they said so "then I can confidently moan at them" [Openreach].
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 10, 2020, 01:00:35 PM
Engineer has arrived at the house just now, same, nice guy. He confirms there’s no signal at the house, has set off to NSBFD or out onto the wild moor



15:46
Modem is now showing 3026 kbps downstream, 532 kbps upstream.  ;D  Our friendly engineer has not reappeared yet and the light is going down. Not a nice day to be out on the tops at all.

Engineer reappeared at the house, making final checks. Nice guy. Literally wires crossed - lying touching another wire in a manhole, I think that’s what he said to Janet.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 10, 2020, 10:04:42 PM
The DS looks good, whereas the US seems to be a bit low?  :-\

But at least the circuit is now operational.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 11, 2020, 01:21:10 AM
@ Burakkucat- indeed, the upstream is a bit lower than it has been. This is because the upstream SNRM is currently 8.2 dB and line 2’s upstream has always gone up and down in an irregular square wave pattern, so the speed depends in where you are in the cycle when we resynch.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 11, 2020, 01:30:59 AM
I had forgotten about the cyclic "capabilities" of the line.  :-[ 
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 11, 2020, 01:38:24 AM
Yes, I keep forgetting. This is the only line showing this odd behaviour now. I can force a resynch and then it will be dropping from 6dB to 4dB upstream on the low part if the duty cycle. I might try that and see what the upstream error rate is like ?



After resynch: 3001 @3.1 dB downstream / 579 kbps @ 5.9 dB upstream. The SNRM-vs-tones graph looks perfect- no hollow, not even a hint of one.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 11, 2020, 01:58:04 PM
Engineer’s notes:

10/12/2020 16:46:00
------------------------
(No manually entered closure notes)

- Plant affected: JNT100
- Plant type: UCJ
Multiple Intervention?: N 2020-12-10T16:46:00 -
(no closure notes!)
What was the result of the initial pair quality test ? - Fail
Did you complete the Base module ? - Yes
Did you complete the Network module ? - Yes
Did you complete the Frames module ? - No
Did you complete the end customer Premises Wiring Module ? - No
Did you complete the end customer Premises Equipment Module ? - No
Have you successfully demonstrated connectivity to the NTE ? - Yes
Did all the connections, extensions and DSL filters meet the minimum standards ? - Yes
Have you conducted a sync test (modem light on) ? - Yes
===QBC Summary Start===
Customer Report: End customer informed us there is no broadband connection.
Initial test results: PQT test performed at NTE back plate. The test passed on 2020-12-10T11:46:19.
What was found: Crossed line identified.
Actions to resolve: Fault located in the D side underground network. This was caused by general wear and tear. The fault was fixed by re-terminating pair in joint.
Final test results: Final PQT performed at the NTE back plate. The test passed on 2020-12-10T16:14:08. Final FastTest completed. The test passed on 10/12/2020 16:44:57.
===QBC Summary End===
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 11, 2020, 06:01:53 PM
There are six interesting lines in those notes --

<snip>
- Plant affected: JNT100
- Plant type: UCJ
<snip>
Customer Report: End customer informed us there is no broadband connection.
Initial test results: PQT test performed at NTE back plate. The test passed on 2020-12-10T11:46:19.
What was found: Crossed line identified.
Actions to resolve: Fault located in the D side underground network. This was caused by general wear and tear. The fault was fixed by re-terminating pair in joint.
<snip>
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 11, 2020, 08:44:09 PM
What do you make from that little lot?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 11, 2020, 10:29:25 PM
Until such time as I am told otherwise, I'll hazard a guess that UCJ is an abbreviation for "underground closure joint" and JNT100 is the specific type.

"Crossed line [was] identified." So there was an incorrect electrical pathway through the joint. It was corrected by disconnecting the pairs and then making the correct connection to link you through to NSBFD.

"This was caused by general wear and tear." I am having trouble trying to picture a scenario which matches that description and the stated "crossed line identified".
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 11, 2020, 10:38:54 PM
No sounds very odd. He was talking to Janet about vermin getting into boxes and biting the cables to get a 50 V high, Janet’s words from him. I don’t see how vermin or haggis on the loose or whatever could cross wires? Wouldn’t crazed OR staff doing things wrong be an equally plausible explanation? The question is "was anyone doing any work on those lines when my line 2 went down?" answer, yes - this same engineer had come out and restored the speed of line #2 which had been down to less than half of what it should be, and during this, it cut out completely. All very odd.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 11, 2020, 11:12:44 PM
He was talking to Janet about vermin getting into boxes and biting the cables to get a 50 V high, Janet’s words from him. I don’t see how vermin or haggis on the loose or whatever could cross wires?

If the wee-beasties had gained access to the joint, and had developed a taste for the PVC insulation, I could just about believe that contact between two adjacent pairs might be possible. As the sharp incisors, whilst stripping off some PVC, might have pushed two wires into intimate contact.

Quote
Wouldn’t crazed OR staff doing things wrong be an equally plausible explanation?

I could not possibly  say . . .  ::)

The important point is that the service has been restored.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 12, 2020, 12:32:18 AM
Indeed, quite so.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 25, 2020, 08:07:09 PM
 :o

Today at 2020-12-25 13:59 my line #4 has failed again ! Downstream sync rate drops, a lot, and the SNRM-vs-tones graph had the usual hollow curve shape. I ran a ‘copper line test’ which failed with the error message:
    BT Test xDSL Copper Test:Pass Line test failed report fault to OR. Appointment advised. Pass Line test failed report fault to OR.
    Appointment advised. T107:FAULT - Dis In Network

I can’t remember what the error code means but it seems familiar

So here we go again. And unfortunately A&A staff aren’t at work this weekend so nothing is going to happen until Tuesday.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 25, 2020, 08:10:53 PM
I found this old reference to the error code https://forum.kitz.co.uk/index.php/topic,23742.msg401638.html#msg401638 (https://forum.kitz.co.uk/index.php/topic,23742.msg401638.html#msg401638)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: mofa2020 on December 25, 2020, 08:29:22 PM
Merry Christmas Weaver 🎄

DIS means a disconnection in the line somewhere,, unfortunately  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 25, 2020, 09:02:15 PM
Today at 2020-12-25 13:59 my line #4 has failed again !

<snip>

Appointment advised. T107:FAULT - Dis In Network

  :o  :(  :wall:
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 28, 2020, 07:31:51 PM
Weirdness. I decided I was going to do a 'prove-it' test for AA so that they could book an engineer asap
tomorrow (Tuesday) Such a test would prove that the problem is in OR's network not chez moi.
My wife is going to be out all day on Tuesday, so won't be able to do any modem-swapping 'prove-it' tests. So tonight we swapped modems 2 and 4's connections to the network. Line 4 was in fact down at the time of the test.
We swapped the modems 2 and 4's connections over and this is the report which we got from the Firebrick :

Link status report from Firebrick after 2-4 swapped
===================================================

Gave error messages: modem 2 is connected to line 4; modem 4 is connected to line 2

then:

*** One of the links is DOWN ***

--
Interface (of FB)    PPP     MTU    Uptime          Status     Phone line circuit ID
‒‒‒‒‒‒‒‒‒‒‒‒—————    ‒‒‒     ‒‒‒    ‒‒‒‒‒—          ‒‒‒‒——     —‒‒‒‒‒‒‒‒‒‒‒‒‒‒‒‒‒‒——
IF-MDM1              PPP1    1500   16d 16:32:01    Active     BBEU20700042 - line 1
IF-MDM2              PPP2    1500       -           Link down (Trying)
IF-MDM3              PPP3    1500       02:55:11    Active     BBEU20700055 - line 3
IF-MDM4              PPP4    1500       00:03:00    Active     BBEU29735631 - line 2

( PPPoE connect message)

--
Firebrick named “tg”, model FB2900, queried at address 81.187.---.---

===================================================

Actually this came from a program that I wrote which processes the output of the Firebrick's PPPoE status function so that it is decorated more elaborately and has various possible error messages.

As expected, the bad line 4, which had been down before, moves to i/f 2 as seen by the Firebrick. So this proves two things: that line 4 is (still) bad (when it is connected to modem 2), and proves that modem 4 is good as it works when it has to drive line 2. So all exactly as expected.

So having achieved what we wanted to achieve, we swapped things back. Then things got weird. Firstly line 4 came
back to life, and not only that but at full speed. Inspecting the SNR-vs-tones graph showed that the bad hollow curve shape of the graph as seen in every fault situation was now absent. So somehow, I fixed the problem, just like the OR engineers who have come out have done even though we don't know how.

So at the moment we possibly have no problem, because I have fixed it. There definitely was one today, from the evidence of the above report and from the evidence of the BT copper line test error report.

I am going to update AA as to what’s going on and ask them to do their own additional line tests to see if the line fault is still out there. I can’t see it on a recent second copper line test, so it’s one of those nightmare intermittent things.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 28, 2020, 10:38:41 PM
We swapped the modems 2 and 4's connections over . . .

I find that statement to be ambiguous.

First possible interpretation --

Were the two leads, currently connected to modem #2, disconnected at the modem; the two leads, currently connected to modem #4, disconnected at the modem. Then the pair of leads, originally connected to modem #2, were connected to modem #4. Likewise the pair of leads, originally connected to modem #4, were connected to modem #2?

Second possible interpretation --

Were the two leads, currently from modem #2, disconnected at the NTE5 #2 and the small switch LAN port #2, respectively; the two leads, currently from modem #4, disconnected at the NTE5 #4 and the small switch LAN port #4, respectively. Then the two pairs of leads were reconnected #2 pair to #4 and #4 pair to #2.

Further possible interpretations result in  ???  followed by  :swoon:

I am having great trouble in trying to identify the item that appears to be the source of the intermittent fault.  :(
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 29, 2020, 06:09:09 PM
I’m not sure exactly what Janet did, because I didn’t see. I think that the modem 2 was reconnected to a different NTE 4 and modem 4 connected to NTE 2. Does that help?

Update AA has taken a brief look at the line but they have not yet replied to my email which is as expected as there will probably be high priority jobs that have come up in the holiday period leaving users with lines down and even no service at all. AA’s tests found no faults.

I reran a copper line test on line 4 tonight and twice it came up with a test failure report
BT Test xDSL Copper Test:Inconclusive More diagnostics required Line test failure. Algorithm unable to continue. Inconclusive More diagnostics required Line test failure. Algorithm unable to continue. DS05:Test Failure - Please Retry - if unsuccessful report problem to OR via a Trouble Report

I have absolutely no idea at all what is going on.

Line 4 itself is now outstandingly fast and very healthy.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 29, 2020, 08:02:57 PM
I’m not sure exactly what Janet did, because I didn’t see. I think that the modem 2 was reconnected to a different NTE 4 and modem 4 connected to NTE 2. Does that help?

Ah, that is logical and is one of the possible interpretations that I did not try to spell out, above.

Quote
I reran a copper line test on line 4 tonight and twice it came up with a test failure report
BT Test xDSL Copper Test:Inconclusive More diagnostics required Line test failure. Algorithm unable to continue. Inconclusive More diagnostics required Line test failure. Algorithm unable to continue. DS05:Test Failure - Please Retry - if unsuccessful report problem to OR via a Trouble Report

All that the above tells us is that the auto-tester was unable to properly perform the test(s).
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 29, 2020, 09:38:35 PM
Does anyone know why DS05 test fail occurs?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 29, 2020, 09:52:38 PM
That program I wrote was designed to check for incorrect corrections, such as modem n plugged into line m. This can be discovered because the program has access to a database of expected config information. When a modem pppoe-connects, it gets an announcement which fortunately contains the BBEU ID for the line that it is connected to. There is a table of such expected responses in the site config database, together with a list of PPP i/f identifiers and line index numbers and all of this is cross checked to see that observed reality matches the config database and detailed diagnostic messages are spat out if not. It’s very useful in the case where Janet plugs some cable into the wrong i/f or wrong unit as it then says exactly what is happening and what should be happening instead. The program has a slight delay while it checks all this config expectation info before it then displays a status report of which lines are up and which are down, BBEU IDs and so on as shown in the excerpt in the earlier post.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on December 29, 2020, 10:03:05 PM
A very useful utility.  :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 30, 2020, 03:50:00 AM
There definitely was an OR network problem at one point because of the copper line test test failure report. I’m wondering if I should get rid of this particular modem, demote it to a backup unit.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: tubaman on December 30, 2020, 07:32:15 AM
I'm wondering if you have an intermittent modem cable, Do you know if you get a 'Dis in Network' fault if you unplug a modem from its line box? My logic is that as I believe you use a simple RJ11 (or RJ45?) socket as your line termination there is no capacitor and resistor across the line like there would be with the usual NTE box. If you had an open circuit modem cable there would then be nothing for the line test to see at the end of the line,
It's just a thought.
 :)
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on December 30, 2020, 12:06:54 PM
The DIS in Network thing has now gone away while the modem is plugged into the NTE5. It’s a brand new modem-NTE5 cable.



AA has told me that they cannot detect any other line faults with their own tests. So for now it seems that I’ve fixed it and we simply do not know how.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on January 02, 2021, 06:29:21 PM
Oh no !  :o :wall: Here we go again, but with line #3 this time. Same problem, on 2020-12-31.

Wondering how to proceed. Swapping modems might be an idea as it worked before, but we just don’t know how. Tests do not indicate there’s a problem in the OR network, but I’m not sure that that means there is not a fault in OR-land.

Any help?
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: burakkucat on January 02, 2021, 10:25:18 PM
I am completely out of suggestions.  :o  :(

In case it might help, I called the associated telephone number of each of the four lines. As discovered previously, the two numbers
returned NU-tone, whilst the two numbers
returned ring-tone.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on January 02, 2021, 10:52:48 PM
I’m thinking that this demolishes the earlier theory about burnout of modem cables or faceplates.

I think that I shall try swapping modem cables; swapping the modem ends of cables from NTE to modem over, at the modem end.
Title: Re: Odd shaped SNR-vs-tone graph - line #4 fault again
Post by: Weaver on January 03, 2021, 10:52:52 PM
I haven’t tried swapping cables yet but the hollow SNRM phenomenon has noticeably improved on its own. It was very cold today compared with the previous couple of days, max 1.5 C.

Here is a graph of the more recent SNRM vs tones taken on 2021-01-04 showing some marked improvement from 2021-01-01 but still quite sickly shape :

(https://i.postimg.cc/7Zzsz7ts/409-C0-E9-C-F565-43-B6-A63-E-6-D1-D032281-FD.png)


And detailed stats:

Code: [Select]
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 328 Kbps, Downstream rate = 3176 Kbps
Bearer: 0, Upstream rate = 402 Kbps, Downstream rate = 2751 Kbps

Link Power State: L0
Mode: ADSL2 Annex A
TPS-TC: ATM Mode(0x0)
Trellis: U:ON /D:ON
Line Status: No Defect
Training Status: Showtime
Down Up
SNR (dB): 3.3 5.9
Attn(dB): 63.5 39.9
Pwr(dBm): 18.3 12.4

ADSL2 framing
Bearer 0
MSGc: 53 12
B: 18 5
M: 8 16
T: 5 8
R: 16 16
S: 1.7387 7.4667
L: 773 120
D: 1 4

Counters
Bearer 0
SF: 1110549 67635
SFErr: 3 0
RS: 40951523 993670
RSCorr: 15389 206
RSUnCorr: 3 0

ReXmt: 8466 0
ReXmtCorr: 8466 0
ReXmtUnCorr: 3 0

Bearer 0
HEC: 1 0
OCD: 0 0
LCD: 0 0
Total Cells: 116209504 17014407
Data Cells: 4044175 784796
Drop Cells: 0
Bit Errors: 280 0

ES: 36013 73
SES: 3706 0
UAS: 12117 9607
AS: 17914

Bearer 0
INP: 29.00 2.00
INPRein: 0.00 0.00
delay: 8 7
PER: 16.02 16.80
OR: 29.44 8.57
AgR: 2768.30 409.82

Bitswap: 5054/5054 232/232

Total time = 6 days 12 hours 55 min 58 sec
FEC: 2024327 5371
CRC: 324338 93
ES: 36013 73
SES: 3706 0
UAS: 12117 9607
LOS: 268 0
LOF: 629 0
LOM: 127 0
Latest 15 minutes time = 10 min 58 sec
FEC: 375 12
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 15 minutes time = 15 min 0 sec
FEC: 509 6
CRC: 0 0
ES: 0 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Latest 1 day time = 12 hours 55 min 58 sec
FEC: 15435 730
CRC: 3 1
ES: 3 1
SES: 0 0
UAS: 54 54
LOS: 0 0
LOF: 0 0
LOM: 0 0
Previous 1 day time = 24 hours 0 sec
FEC: 1162 1762
CRC: 2 33
ES: 2 15
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
Since Link time = 4 hours 58 min 33 sec
FEC: 15389 206
CRC: 3 0
ES: 3 0
SES: 0 0
UAS: 0 0
LOS: 0 0
LOF: 0 0
LOM: 0 0
NTR: mipsCntAtNtr=0 ncoCntAtNtr=0