Kitz Forum

Broadband Related => ADSL Issues => Topic started by: Adamantium on June 27, 2011, 04:11:42 PM

Title: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on June 27, 2011, 04:11:42 PM
Hi,

First off apologies for the long post, but there's a lot of info to squeeze in...

The Background

Moved into a house in Sept last year, located in a village in Bedfordshire. ADSL provided by Zen via the exchange 4.73km straight-line distance away. Actual line length probably closer to 6km.
Line has always been a solid 1Mb with the BRAS profile sometimes creeping up to ADSL 1250 on occasions.
Router used = Edimax AR-7284WnA running the latest firmware 1.0.6. Connected to BT Test Socket. PC connected to router via ethernet.
BT line goes from master socket to telegraph pole outside on street. This pole is connected to various underground junction boxes which feed my line approx 0.5km to the green cabinet in the centre of the village.
Connection was stable for all online activities (streaming, downloading, gaming etc) up until March this year when the problem began...


The Problem

March 4th 2011 - I notice when playing an online FPS game that i'm getting what seems to be periods of near-constant packet loss (lots of "running on the spot", lots of "teleporting" - just basically all the signs of a laggy connection) However my ping remained normal throughout.
I then checked by streaming something on youtube, this had similar results (I have a bandwidth monitor that tells me how much traffic my PC is generating at any given time) and it showed that the stream was "cutting out" at intermittent intervals where the connection would still be active, but the stream would essentially stop completely for a short period of time and then resume it's download.
I experience similar symptoms when testing with anything that uses the internet connection (a chat on Skype would be rendered inaudible for short periods of time etc etc)

Having checked my router I found that it was reporting thousands of errored seconds (although no HEC, OCD or LCD errors reported)
I also noticed that the SNR Margin was intermittently fluctuating from it's normal 7-8dB right down to 0.0dB for short periods of time, before retuning back to the normal 7-8dB range.
I noted that the errored seconds being reported by the router were increasing in-line with the SNR margin drops I was seeing.
Further testing also showed that the SNR Margin drops were occurring at the same time I was experiencing the problems with my connection (as described above)

Router rebooted made no difference, so i log the fault with Zen Tech Support.


The Investigation

As with all ISP's, I was asked to perform all the normal checks to rule out the problem being at my end; replaced filters, cables etc. I even went out and purchased a new router to test, which turned out to be worse than my existing router in being able to handle the SNR Margin drops to 0.0 - with it constantly dropping sync.
Interestingly, Zen couldn't see any of the errored seconds on the line, even though the router was reporting literally tens of thousands for each 24hr period. They could however see that the router was losing sync with the exchange regularly (now that the alternative router was plugged in and dropping sync all the time) and so they booked in a BT engineer to investigate an "intermittent connection".

March 16th 2011 - Engineer comes out to house (visit #1) and checks connection. He see's there is an issue (BRAS profile by this time has dropped to ADSL150 due to all the disconnects) and goes off to investigate.
He calls an hour or so later from the exchange to say "he thinks he's fixed it". By this time i'm back at work, so can't test the connection until i get back home.
I get home to find everything exactly the same. SNR Margin still constantly dropping to 0.0. All symptoms the same.

I inform Zen that the problem has not been fixed.

March 17th 2011 - Zen confirm to BT that the issue is not resolved.
March 18th 2011 - BT engineer (visit #2) booked for Sunday 20th March - AM appointment.
March 20th 2011 - No sign of any engineer - waited all morning. Queried situation with Zen.
March 21st 2011 - Zen update to confirm engineer did attend, but didn't require access to my premises. Engineer performed D-side swap. Zen want to monitor connection for 24hr period to see if issue resolved.
March 22nd 2011 - No difference to the connection, still all the same issues. Spoke to neighbour, he confirms lots of problems with his internet since early March too. Confirm all this back to Zen.
March 23rd 2011 - Zen arrange for BT Engineer (visit #3) to attend March 24th - PM.
March 24th 2011 - BT engineer attends premises (speaks to the missus this time, as i'm at work) again tests the line and see's there are problems. Goes off to "investigate further" in the village / exchange.
March 25th 2011 - Update from Zen re; previous day's engineer visit - BT Engineer notes read; "EU in sync on arrival, but EU said he is facing very intermittent issues. Eclipse test ok. ADSL test show CRC errors. There is a REIN issue in area. REIN issue Ref 2755 that was built on job SM7XAF72. Have reported to SFI about REIN issue."

So looks like they'd discovered a REIN issue... As it was Friday, Zen suggested there wouldn't be any further update until Monday 28th....

March 28th 2011 - Update from BT via Zen to say that the REIN fault is raised, and that they are still working on it.
March 29th 2011 - Update from BT via Zen to say REIN engineer assigned, but no ETA.
March 30th 2011 - No further info on REIN engineer visit.
March 31st 2011 - No further info on REIN engineer visit.
April 1st 2011 - No further info on REIN engineer visit. Although connection seems to be back to normal (aside from BRAS profile which is still stuck at ADSL500). Did they fix something??
April 2nd 2011 - No further info on REIN engineer visit. Spoke to Zen, confirmed line seems to be back to normal aside from the BRAS profile. Requested manual raising of profile.
April 4th 2011 - No further info on REIN engineer visit. Zen get BT to manually adjust BRAS profile back to ADSL1000.
April 5th 2011 - No further info on REIN engineer visit. Monitoring of line stability ongoing. So far so good.
April 6th 2011 - Update from BT via Zen to say; "REIN team have confirmed that job is closed as engineer has visited site and found no evidence of REIN. Current test results shows line is stable for 3 days now with a speed of 1 Meg". Line still stable, Zen close their ticket as issue resolved.

So... It seems that the problem just went away by itself after 28 days of constant issues. Strange, but I didn't really care as I had my internet connection back.

April 26th 2011 - ARGH! The problem is back (after 25 days of working fine). Exactly the same symptoms - just started again out of the blue. I re-raise with Zen.
April 28th 2011 - Zen re-raise fault with BT.
April 29th 2011 - BT want to arrange engineer visit, I explain to Zen that it would need to be a weekend visit as I cannot take any more time off work. Zen will go back to BT.
April 30th 2011 - No further update.
May 1st 2011 - No further update. Line seems to be back to normal again..
May 2nd 2011 - No further update. Line is indeed working as normal. Inform Zen.
May 6th 2011 - Zen close off fault.

So that was a little 5-day "blip" of the problem, after which everything went back to normal again...

May 23rd 2011 - Nooooooooooooooooooooooooo!!! The b**ch is back. Exactly the same symptoms etc etc.
May 24th 2011 - I report to Zen that the issue has returned and request they log with BT.
May 25th 2011 - Update from Zen to confirm BT has stated there is an issue outstanding at my exchange which has to be resolved before they can investigate further.
May 26th 2011 - BT confirm issue at exchange resolved. I confirm problem still outstanding on my line. Ask Zen to chase with BT.
May 27th 2011 - No further update.
May 28th 2011 - No further update.
May 29th 2011 - No further update.
May 30th 2011 - No further update.
May 31st 2011 - Update from Zen to ask when BT engineer can attend (visit #4) to investigate. They inform me that earliest appointment would be June 7th. I ask for issue to be escalated as i don't want the issue disappearing again before BT have a chance to fix it.
June 1st 2011 - Zen escalate issue with BT management.
June 2nd 2011 - BT Manager confirms to Zen that he will "see this fault through personally". Engineer arranged to attend exchange to fix E-side (as that is where they think the problem lies) on June 7th (visit #4).

June 8th 2011 - Update from Zen; Thank you for the update. We have the engineers notes from the visit yesterday;
"The engineer has indicated an issue with the D-side of the network (the part from your premises to the Green Street Cab).
The engineer was fully aware of the issue that you have been having and saw it happen as well. However this engineer was unable to resolve the issue.
Engineer spoke to his Team Coach, who advised that they would need to send a specialist engineer to investigate the issue.
Engineer also advised that there are other people in the area having the same issue that have not been resolved.
The fault is not in BT Openreaches Escalation queue, waiting the assignment of the PTO Engineer to investigate that fault.
We have been advised by BT Wholesale that the next update will be tomorrow 09/06/11."


Maybe some progress at last? Hopefully PTO engineer will be able to finally shed some light on the issue....?

June 9th 2011 - PTO Engineer (visit #5) confirmed for June 13th.
June 13th 2011 - PTO Engineer calls me to confirm what speed my router would usually sync at. During the brief discussion, he confirms he'll be investigating from the pole to the exchange.
June 14th 2011 - Update from BT states; "Checked that the fault was dealt by the quality assurance team who have checked with the suppliers.
The PTO is still dealing on the fault and investigation a possible REIN issue.
We have been suggested to review this on 15-06."
Back to it being a REIN issue?
June 15th 2011 - Update from BT states; "ON this fault suppliers have update latest notes states that PTO found that duct and cable is required please provide estimate so work can be done. There is no estimate date provided on this. Suppliers advised to check tomorrow for more update.

Thanks & Regards,

Broadband Customer Service Team Leader"


I guess no REIN issue afterall?

June 16th 2011 - No further update.
June 17th 2011 - No further update.
June 18th 2011 - No further update.
June 19th 2011 - No further update.
June 20th 2011 - Zen contacted BT who advised; "Duct work is being undertaken by BT Openreach. They have advised that there will be an update available on 21st PM."
June 21st 2011 - Zen update; "I have been through to BT who advise that this duct work is now complete. We will check to see how this affects the stability of the line"

Success at last???

I check the line when I get in from work. Still exactly the same. All the symptoms remain. I inform Zen and ask them to go back to BT.

June 22nd 2011 - Update from BT; "PTO task is completed. Duct work was carried out. He confirmed customer was connected." Obviously the engineer did not check to see if the specific symptoms had been resolved, just that the line was connected (which it always has been throughout the problems!) ARGH. Useless ******* BT!!!!!

This is where it starts getting very confusing, as BT's own updates start contradicting themselves...

June 23rd 2011 - Update from BT; "We have contacted the suppliers who have informed that the PTO who was assigned on this has checked and confirmed that all is ok.
Hence there was no duct work carried out.
As you have informed that the issue is still there, would need an SFI engineer to progress further the escalation."
No duct work carried out?! WTF? I ask Zen to clarify with BT..

BT come back with two separate emails which state;

"when the engineer informed initially regarding the duct work, this was checked by the PTO engineer and found not to be needed, after which the task was closed."

"I have checked with the suppliers and they have confirmed that the duct work is completed, customer now has a working session."

So one update stating the duct work had been completed, the other stating it wasn't even required. But regardless of BT's conflicting views on what has happened - the actual problem remains unresolved!!! Requested that Zen escalate the issue once again.

June 24th 2011 - Update from Zen; "We have requested a complete breakdown of the the fault from start to finish from BT, including everything that has happened including the parts where BT wholesale have gaps in their knowledge and need to speak with Openreach. We are also pushing for a fresh PTO visit to do a full end to end check"
June 25th 2011 - Update from BT;

"07/06/2011 – Engineer requested the fault to be passed to the PTO.
09/06/2011- Fault passed to suppliers escalation queue to carry out investigation
10/06/2011 – Review provided for 12-06
13/06/2011 – Informed regarding the PTO visit for 13-06, to be reviewed on 14-06
14/06/2011 - PTO has found that duct and cable is required. Form sent to planning team.
16/06/2011 – review provided for 18-06
19/06/2011 – Task passed to repair team
21/06/2011 – Task closed after confirmed working with customer adam. Line tested ok, asked to close Fault not found"


Zen have replied to them stating they want the full notes, not just bullet-pointed summaries, and also the full details going back to when the fault was first diagnosed including full engineers notes and the notes form Openreach and the planning department.
I also pointed out to Zen that at no point have BT contacted me to confirm the state of the line, and if they had, i obviously wouldn't have said it was working! So that last update "confirmed working with customer adam etc etc" was a complete fabrication.

June 27th 2011 - Today. Issue is still with BT having been escalated once again... Awaiting their next update.

So that is where we currently stand. Still seemingly no closer to a resolution than we were 4 months ago when the problem first arose.. With very little faith in BT based on what i've seen so far..
I'll post some router stats when I get home, but i'd be interested to see what people think the source of the problem is going to be?

Again, apologies for the wall of text - hope some of you made it through to the end!

Thanks,
Adam.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: roseway on June 27, 2011, 04:36:46 PM
Hi and welcome,

I've made a little edit to your note to remove the bit of bad language.

If it is REIN from an external source, then it's quite possible that it would be intermittent in the way you describe. There's quite a bit of information about REIN and how to track it down here. (http://www.kitz.co.uk/adsl/rein.htm) If you fancy wandering around the neighbourhood with an AM radio in your hand you might get lucky and find a source.

The other thing which might be helpful would be to install Routerstats or Routerstats-Lite (http://www.vwlowen.co.uk/internet/files.htm) if it can be got to work with your router. This would enable us to see the shape of the interference, which might help to pinpoint the cause.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on June 27, 2011, 05:46:36 PM
Hello Adam,

I have read your post from beginning to end and would like to congratulate you on its clarity & precision. I wonder if there are two issues that are combining to give your the observed problem:

(1) REIN
(2) Borderline infrastructure failings.

Without the full details, which Zen are still attempting to obtain, the best thing you can do is to continue to monitor the situation as closely as you can and when the problem arises again, having given due consideration to the link that Eric has mentioned, then perform your own attempts at REIN tracking.

There are three entities involved in attempting to solve your problem:

(1) Your ISP, Zen, who are doing the best they can.
(2) BT Wholesale and their management, who I view with a big dose of suspicion.
(3) OpenReach, the infrastructure owner and maintainer. In this last case, I'm wary of upper level management but will always support the foot-soldiers on the ground, who are given specific jobs to perform by a poor (at times) management process.

I'm sure that Mr Pag will be along later and then make a few comments. However remote diagnosis can only be speculation without access to the precise information, first hand. ;)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: waltergmw on June 27, 2011, 06:26:49 PM
Hello Adam,

I note you say you have been talking to your neighbour who seems to have a similar problem. (Likewise other problem areas in the village?) Perhaps you could liaise with him quite closely as a common fault should be easier to eliminate. Apart from anything else, swapping your E side and D side cables should not affect your neighbour.  Just in case the pair of you are implicated it would be worth turing off everything in your house and observing next door and vice-versa.

Quote
Interestingly, Zen couldn't see any of the errored seconds on the line, even though the router was reporting literally tens of thousands for each 24hr period.
I think you'll find that some line data is purposely masked out as otherwise the ISPs would be inundated with transients.

Don't be too hard on the actual engineer's reports. Much of it is composed by box-ticking on proforma forms rather that the engineer being allowed to write his own observations freely.

Kind regards,
Walter

Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: razpag on June 27, 2011, 06:32:16 PM
Hi and welcome.

Can I also thank you on a clear and concise posting, well put together.

Hmmmm, trying to pick the bones out of it leads me to believe you have a faulty section of cable somewhere in the network. I can't see any other reason for a Precision Test Officer to be in attendance.

TBH, we lowly paid multi-skilled engineers do the work of the highly paid PTO's but only on the D-side of the network (ie- from Cab to premises) as these particular underground cables are not pressurised, as their protection from water ingress comes in the form of grease. The E-side cables (from the exchange to the cab) though are ,or should, be constantly pumped with air from the ECP (Equipment Cable Pressure) rack situated in every exchange. Occasionally, a fault will occur whereby the pressure will drop and water can and does get into the large E-side cables.

Putting 2 and 2 together (hopefully making 4) your account of a PTO being in attendance, and other EU's being affected, I would hazard a guess on it being a faulty E-side cable that doesn't have any 'Spares', 'Stops' or 'TOS's' available. In other words, there's no spare wires available within the large cable, to swap your circuit into. The other added factors that add a bit more bias to my thinking is that you are quite rural ?? The 'pair capacity' for rural cabs is far, far less than for urban, so this limits enormously the 'spare capacity' in any rural setting. Also, there is mention of 'Duct and cable' work in your posting, if this is to be believed then it again points to a 'faulty cable length/joint'. It doesn't appear to be REIN related going from you posting.

Hope this helps, and I hope I'm right. ;D
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 07, 2011, 05:07:43 PM
Thanks for your replies guys.

We're not much further since my last post, however there have been a few updates;

July 3rd 2011 - Update from BT; "An A55 is submitted to planning Team for new duct and cable changeover to cure broadband problems. The job is with the planning team and it will take some time to set up the estimation for the cabling and jointing work."

July 7th 2011 - Update from BT; "We have checked with the suppliers and they have contacted the controls team who advised that task is Awaiting Financial Authorisation from Planning Manager. This process takes time so they have advised us to check in next week for more updates. We will check with the suppliers on Tuesday and will update you accordingly."

Zen have also managed to get some of the (what i assume is) Openreach engineer notes, however they contain a lot of BT jargon i'm not familiar with, but hopefully Razpag will be able to translate?
Here are two of those updates which seem to contain potentially interesting info;

20/03/2011 – Engineer completed the task - D side UG, Local network changes made by BT engineer for broadband use. Notes - D side UG

Notes - EU reports dropping connection. Sync on arrival at Primary Connection Point down-2528k. Using JDSU with no errors, valid PPP session. Tried 4 pairs from PCP to JF4, no8 Station road. All have errors, possible REIN issue sending back to CP, Pair Quality test passed ok @ JF4 os no8 Station road on all Pairs.


07/06/2011 – Emailed CP on the escalation regarding the engineering task completion.

Notes - Please pass to pto. pto require d to help solve problems off of this dp 476 and dp 475. bb lines are dropping connection and now have got more and more slower. tried different e-side but still the same tried d-sides from pcp2 to jf 4 b/o dp482, bb then shows and improvement but doesn't last long. I believe faults are on d-side but again still finding it a problem to solve. they have been a cable changed over in the last 6 month which was the problem before. spoke to coach and only solution is to have pto onsite. eclipse & pq test always pass. I have inform adam at no 18 to what is to happen next. there is issues at no12, 18, 28 [**] and no 25 [**] over last few weeks and have not be resolved.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: tuftedduck on July 07, 2011, 05:33:53 PM
I've made a slight edit to your post, Adamantium, in order to remove the actual street names in the final paragraph.

Just a wee safety check.  :)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: razpag on July 07, 2011, 06:50:37 PM
Hmmmm ? Not what I was expecting from my first thoughts on your predicament, but that's how the dice roll sometimes.

Back to the notes ..........

20/03/2011 – Engineer completed the task - D side UG, Local network changes made by BT engineer for broadband use. Notes - D side UG (Just informing that the engineer has made some changes to the D-side network. It could be anywhere from the Cabinet (PCP) to the DP (Telegraph Pole).

Notes - EU reports dropping connection. Sync on arrival at Primary Connection Point down-2528k. Using JDSU with no errors, valid PPP session. Tried 4 pairs from PCP to JF4, no8 Station road. All have errors, possible REIN issue sending back to CP, Pair Quality test passed ok @ JF4 os no8 Station road on all Pairs. (Engineer has tested the Broadband signal at the green cabinet and it was working at roughly 2.5 Meg on his JDSU tester, with no error counts and a valid PPP session basically shows that the line is connected to the right equipment in the exchange. He seems to have found the fault on the D-side somewhere between the cabinet and a Joint Box 4 (underground single lid footpath box). He has swapped your circuit four times into different spare pairs between the Cab and this box, but even though the wires test absolutely perfect (via a PQT test), he can still see a high error-count when he connects his JDSU up at the JF4. This, he thinks is reminiscent of a REIN issue).

07/06/2011 – Emailed CP on the escalation regarding the engineering task completion.

Notes - Please pass to pto. pto require d to help solve problems off of this dp 476 and dp 475. bb lines are dropping connection and now have got more and more slower. tried different e-side but still the same tried d-sides from pcp2 to jf 4 b/o dp482, bb then shows and improvement but doesn't last long. I believe faults are on d-side but again still finding it a problem to solve. they have been a cable changed over in the last 6 month which was the problem before. spoke to coach and only solution is to have pto onsite. eclipse & pq test always pass. I have inform adam at no 18 to what is to happen next. there is issues at no12, 18, 28 [**] and no 25 [**] over last few weeks and have not be resolved. (Basically there appears to be two DP's (Telegraph Poles) affected by the fault you are experiencing. He's saying that the wires STILL test ok via our PQT and Eclipse test sytems, but that the speeds are getting worse. His Coach (ridiculous Americanism for 'Foreman') has suggested that a 'Precision Test Engineer' attend site along with the Broadband Engineer to try and iron out the problem).

My thoughts are, if this was my patch this would probably have been resolved by now. I do not know why a PTO should be in attendance, as our own testing is carried out to a high standard (on the D-side) that is equal to a PTO's testing ?? He evn syas that the cable has been replaced in the last 6 months ?? If there were split pairs caused by poor jointing, then they would be picked up on the PQT test usually on the 'AC Balance' parameter.Without knowing the area and the cable run, it is hard to give exact information. For example, are the 2 DP's fed from the same cable from the Cab ?? Things like this are of vital importance in drawing conclusions.

To change my stance slightly, it is beginning to sound REIN'ish, especially as multiple EU's are being affected  and nothing 'new' has been done recently, as the cable was changed over 6 months ago. Hard to put my finger on it from the comfort of my computer den, hundreds of miles away. ;D
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 08, 2011, 01:33:15 AM
Thanks for the edit tuftedduck ;)

And thanks for the translation work razpag, much appreciated. :)

From what you've said, i'm not confident the cable and ducting work they plan to do is going to resolve the issue.

I've dug out some routerstat graphs I collected when the issue first arose back in March - showing my SNR margin over the period of 80 minutes (see below - each graph is 20mins worth)
I don't have anything more recent as the router i was using at the time (and the only one i have that is compatible with routerstats) was having trouble staying in sync for more than a few hours - which was causing my BRAS profile to drop right down to ADSL135.

Do these graphs look like REIN? (I ran the capture for a full 24hrs, and all of the graphs keep to the same general pattern that you can see in these 4 screenshots - with the longest period without the SNR Margin dropping to 0.0 being a single 6 minute period at around 12:33am)

(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi115.photobucket.com%2Falbums%2Fn284%2FAdamantium_2006%2FREIN%2FNoiseMargin-2011Mar24-1546.jpg&hash=13f5e78e4bc2528565a7faa56e9a058ae43beab0)
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi115.photobucket.com%2Falbums%2Fn284%2FAdamantium_2006%2FREIN%2FNoiseMargin-2011Mar24-1607.jpg&hash=a478eb95b429b1118c94ed5d94542fa3048ae7a6)
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi115.photobucket.com%2Falbums%2Fn284%2FAdamantium_2006%2FREIN%2FNoiseMargin-2011Mar24-1628.jpg&hash=7b6e9b3d96b8ca5fab8a822c1ce260dfe82fff2b)
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi115.photobucket.com%2Falbums%2Fn284%2FAdamantium_2006%2FREIN%2FNoiseMargin-2011Mar24-1648.jpg&hash=6f9dbba9f90ec7f8d1c93778354fbc4fd38129f8)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: jeffbb on July 08, 2011, 05:47:15 PM
Hi
These seem to be old graphs ?.March time ?

They are a bit unusual ,normally the SNR plot is spiky above and below a mean track. This looks more like something is switching on and off .
regards Jeff
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 08, 2011, 06:31:26 PM
These seem to be old graphs ?.March time ?

I don't have anything more recent as the router i was using at the time (and the only one i have that is compatible with routerstats) was having trouble staying in sync for more than a few hours - which was causing my BRAS profile to drop right down to ADSL135.

;)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 14, 2011, 03:47:49 PM
July 12th 2011 - Contractors for BT starting digging trench and laying ducting from telegraph pole outside my house along to a junction box about 20m up the road (seemingly cutting through my neighbours phone line in the process). BT Openreach to follow with the laying of new cables - eta July 18th 2011 for task completion.

I'm still dubious as to whether this will resolve my issue, but at least it's progress..
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on July 14, 2011, 05:47:13 PM
Quote
(seemingly cutting through my neighbours phone line in the process).

D'oh!  :doh:  I suppose that is some sign of progress.

Please keep us updated with developments. :)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 20, 2011, 08:17:52 PM
July 20th 2011 - Having been fobbed off for the past two days when enquiring why there were no Openreach vans outside my house fitting the new cables (completion date of the 18th remember!) This morning I wake to find an Openreach van and a big Openreach truck outside with engineers busily doing their good work. Excellent. ;D

I get an update from BT just after lunch;

"Hi,

All work has been completed now. All ducting and 40 metres of cable have been laid and the job has been closed off. Let’s hope this was worthwhile and makes some difference to your customer.

Please retest and advise.

Thanks,
Kind regards,

High Level Escalations Team Gatekeeper
Customer Relationship Management | BT Wholesale"


I return home to find that nothing has changed........  :o :( :no: >:( :(

Still all the same problems as before. It seems I was right to be dubious that this would fix it.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on July 20, 2011, 08:50:42 PM
Quote
High Level Escalations Team Gatekeeper
Customer Relationship Management | BT Wholesale
:whip:  >:(

Just keep up the pressure on that "Gatekeeper". ::)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 22, 2011, 07:25:18 PM
July 22nd 2011 - Update from BT; "I and my colleague have carried out thorough investigations on this line and the neighbouring lines.

I have checked other circuits on the DP and all have been running very similar to your customer’s with the SNR of 0dB which has only recently dropped. It is running similar at times to the other lines on the DP with regards to speed etc but then the SNR just dips to 0dB on this one circuit.

All the lines are long which I’m sure you already knew, and their speeds are not the best.

So from what I am seeing all lines have been running similar but your customer’s has the dips to 0dB. The times it does this are random with no pattern whatsoever The line is also dropping now and then but the majority of the issue is this dip to 0dB. The dips are more than likely caused at the end user end (BT network side, not end users equipment) as other circuits are not experiencing these dips. The broadband is fit for purpose but this customer’s is probably being pushed to the limit if it is being used for gaming etc

I think we have no option but to send another engineer to see if the line can be improved anymore.

If you can advise of customer availability I will arrange an appointment to get this investigated further.

I was considering a TPM but don’t want to jeopardise the working circuit as it could end up having a detrimental affect and the issues do seem to point to the network as opposed to the port."


The words "fit for purpose" in their update worry me tremendously... :no:
As that smells to me like they're going to push to have the issue closed off if they're unable to find the source of the fault after their next engineer visit  >:(
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on July 22, 2011, 08:03:52 PM
Quote
The line is also dropping now and then but the majority of the issue is this dip to 0dB. The dips are more than likely caused at the end user end (BT network side, not end users equipment) as other circuits are not experiencing these dips. The broadband is fit for purpose but this customer’s is probably being pushed to the limit if it is being used for gaming etc

Looking at the above paragraph, it seems that the OR engineer is suspicious of your line from the DP to the NTE5/A. To eliminate that completely, I would expect a new cable run and an NTE5/A with filtered faceplate to be fitted.

The definition of "broadband is fit for purpose" is the achievement of a (chronically) low download sync -- I can't recall the exact figure but I believe it is 512Kbps. :(  However, with regular drop-outs (loss of sync) I would dispute the "fit for purpose". Care must be taken here, for OR is entitled to say that it is a satisfactory POTS line but it is unable to support an ADSL broadband service -- so "tough". :'(

Perhaps RP will drop in later on this evening and give his updated (remote professional) opinion on the situation.

If I were in your position, I would search eBay for a 2Wire 2700HGV (a.k.a. a BT Business Hub) and spend up to Ł30 in purchasing one -- brand new, factory sealed in box. For long lines, they are really the modem/router of choice. And OR will approve of the device. ;)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 23, 2011, 12:51:21 PM
Care must be taken here, for OR is entitled to say that it is a satisfactory POTS line but it is unable to support an ADSL broadband service -- so "tough". :'(

My issue with that statement is that I had perfectly working broadband service on this line for 6 months before this problem arose, so it's not as if the line has always had a problem...
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: waltergmw on July 23, 2011, 06:58:56 PM
@ Adam M,

Which is why I keep records of line performance from the moment anybody asks for my help.

Do try to gather whatever evidence you have of past performance.

Kind Regards,
Walter
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on July 25, 2011, 01:18:27 PM
July 25th 2011 - Update from BT when trying to book the engineer to come out to investigate further;

"I have raised a trouble report but ironically the line test failed showing a dial tone fault. A DT fault is either customer end or PSTN related. This type of fault is a “next available” and not appointed as it will be a line engineer that needs to go out to look at this.

Once this has been actioned we will reassess to see if this has had any bearing or improvement to the broadband and decide if we need to book a broadband engineer after this time.

There should be no need for the customer to be available for this particular visit.
"

I think i'd be over-optimistic if i were to think that this has been the problem all along, and will fix the underlying issue..
The pessimist in me just thinks that this will be another delay in getting my actual issue resolved..  ???
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on August 03, 2011, 01:36:48 PM
July 26th 2011 - Update from Zen;

"Hi Adam,

The openreach engineer did not find any DT faults on the line.
BT are requesting an engineer to come to site to do a complete end to end test.
We have provisionally booked the appointment for tomorrow 27th pm slot.
Can you let us know if that is ok for you, or let us know when you are available.

Regards,
"


July 27th 2011 - I had two Openreach engineers turn up briefly.
One of them said he'd been round the village looking at various sources of REIN that he'd found (he confirmed he wasn't a REIN engineer, but nevertheless had been round with an AM radio to look for sources).
He said he'd found 3 sources of REIN at different locations throughout the village, although he didn't elaborate what or where they were. He said this issue was affecting a number of households in the village - all with outstanding broadband problems.
He did say that he'd tried "suppressing" some of the sources (again, how this was done i don't know) and he thought that it may have fixed the problem. He added that the equipment he was using to detect the REIN had now broken, so he couldn't do any further testing.
At the time they arrived (approx 3:20pm) the stability of my connection was better than i'd seen it for a long time - with minimal SNR margin dips and errored seconds on the router (although i'm not ever generally home during the day to test it - so at the time i couldn't say whether this was due to the work they'd done, or if it was just due to the time of day).
The second engineer then plugged her device into the phoneline and performed some sort of test which seemingly came back fine.
So at this point it was looking like they had indeed resolved the problem.

However within an hour of them leaving, the connection was back to normal (i.e. 0.0dB dips constantly).
Both Openreach engineers were still in the area as they had been looking at a neighbours' issue. I went out and spoke to them again to confirm the problem was still apparent.
The engineer that had been performing the REIN "tests" said he would go up the road and do a few more checks and then write up his report.

I informed Zen of all the above and asked them to pass back to BT..


July 28th 2011 - Update from BT;

"I have spoken to the REIN Team, the REIN issue has been identified and closed off this morning.

The engineer went out yesterday and this morning and has liaised with several people in the village. He had identified 3 sources of REIN and has addressed them by suppressing them with a RF3 filter.

The issue should be resolved, if you can check with your customer as to any improvements.
"

I re-affirmed my previous update to Zen that the issue is no different.
I also swapped my router back over to my NETGEAR model which is compatible with Routerstats so I could capture the SNR margin drops..


July 29th 2011 - Update from Zen;

"BT have advised that the next step is to carry out a Tie Pair Modification.

This will move you on to a completely new line card and DSLAM at the exchange and will result in some downtime as they have to physically move your connection.

This rules out an issue with the port on the line card and any issues that may be caused by the line card and DSLAM

If you are happy for this to be done, then let us know and we will get this actioned with BT as quickly as possible.

Regards
"

I confirm I am happy for this to happen ASAP.


August 1st 2011 - Update from BT;

"TPM is done and customer is in sync on the new CBUK.

Please retest.

Thanks
"

No change - everything still exactly the same. Asked Zen to pass back to BT.


August 3rd 2011 - Update from BT;

"I have tested the circuit this morning and can see errors on the upstream. It looks like we are going to have to arrange another engineer visit to look at both the network and the REIN issue. Whilst installing filters will go some way to screening interference it won’t eradicate it altogether.

The customer does need to be aware though that REIN isn’t BT’s responsibility and is only investigated as a gesture of goodwill because it is affecting the broadband. There are very few REIN trained engineers and those that are REIN trained can’t always get hold of the specialist equipment needed. They sometimes have to travel several miles to attend a job so to manage the customer’s expectations it could turn out to be a lengthy investigation.

You say that the customer’s neighbours have reported issues with their CP’s but there are no other faults reported whatsoever on the NCIF this customer is on. There are 306 circuits but only this one fault. It could be that the other circuits are on a different NCIF.

So, if once again I can ask you for the customer’s availability I will arrange the appointment.

Kind regards
"

REIN is mentioned as i have been running routerstats on my line 24/7 for the past week. This has shown that the drops to 0.0dB generally show a trend towards being present more early in the morning and then late afternoon onwards until about 2-3am. Between these times there are periods where i get little or no drops at all. However this isn't always the case.
I have advised BT of this, which is why REIN is again back on the radar.
I will post up some of the graphs when i get home to illustrate the loose patterns i'm talking about..

In the meantime, I have asked that a new cable run from the pole to my house is done when they come out - along with a NTE5/A with filtered faceplate as burakkucat suggested previously..

So I await their next visit.....
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: waltergmw on August 03, 2011, 02:57:49 PM
@ Ezzer & RP,

Have you ever tried earthing the catenary wires in a drop cable to see if that might just help REIN Issues ?

Kind regards,
Walter
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on August 03, 2011, 05:22:31 PM
@ Walter,

My inventive mind promptly wanders off to consider the use of a screened (as co-axial) twisted pair (think low-impedance balanced microphone input) but comes back to the fact that the aerial drop wire is probably not where the majority of RFI or REIN is picked up. From my limited understanding of the propagation of REIN, more often than not it is injected into the multi-pair cables bundles via the pair from the premise of the REIN generating source.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: waltergmw on August 03, 2011, 06:04:14 PM
@ BKC,

IMHO there are two sources of RFI but they can't be easily defined precisely. The horrible mess that we loosely call REIN and the licensed spectrum users. I know of one (now removed) source which was a faulty first generation analogue cordless phone which just produced serious cross talk up the bundle whenever and only when a phone call was made.
For the second type for E.g. how many bit loading diagrams have you seen with Radio 4 missing ?

Except for the precedent problem, I cannot understand why (seemingly) BT don't use a foil wrapped cable whenever new or replacement cables are being installed.

Kind regards,
Walter
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: burakkucat on August 03, 2011, 07:24:54 PM
Quote
Except for the precedent problem, I cannot understand why (seemingly) BT don't use a foil wrapped cable whenever new or replacement cables are being installed.

I suspect there would be a two fold response: (1) It would involve a deviation from the "norm", thus a rewriting of the standards. (2) Cost. ::)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: razpag on August 03, 2011, 08:10:47 PM
As regulars will be aware, I'm a relative novice when it comes down to REIN and it's many guises, but have been in attendance on quite a few jobs, and been 100% succesful in the 2 issued to me since completing my training (big whoopee eh ?? ;D).

So, on the back of that, Walter ..... no, I've never 'earthed' the "Catenary" wires as you mentioned. By catenary, do you mean the steel strengtheners that are in the dropwire 10/12/14 ???

Again, from my limited experience, the higher percentage of REIN faults I've been involved in has seen the source begin at the EU's premises and 'track' back up the dropwire to the DP, and then (depending on the strength of the interference), into some or all of the other dropwires fed from that DP. In a recent case, it actually then went down the UG cable feeding the DP, and into another UG cable running parrallel to it in an underground box. This caused another DP and it's EU's to be affected.

I identified the source of the REIN, informed and proved it to the occupant of the house it was emmiting from, and then informed the REIN team. 'We' (Openreach) have no powers at all thus far to make the occupant switch off, or remove the offending item. 'We' feed back the information to the SP of where the REIN is coming from and it is down to them to liase with the occupant and try and reach some kind of compromise.

B*Cat is also correct (and covered by myself in another thread months ago) in that any change to 'flying wires' whether they be multi-core or otherwise, would need stringent load bearing tests/aerodynamic effects. We have  'certain' maximum distances we can erect different types of OH cables, and any further weight gain given to those cables may drastically affect this distance. I'm not sure how many poles we have nationwide, but the costing would be astronomical if major parameter changes were implemented.

We know REIN is more prevalent of late than in years gone by, due to much higher frequencies susceptible to noise, being pumped down our cables. Also, the comment below about access to REIN testing equipment was correct, but in-roads are being made to purchase more 444b testers. I can verify this to be fact, as I have filled in the neccessaries to get one myself just a few weeks ago and our Level 2 manager agreed to the price of it.

I reiterate, REIN is a problem, but as yet not directly under the juradstiction of Openreach, as we perform the location of it as a 'Goodwill gesture' only . 'We' are however  training lots more engineers in REIN fault-finding. HTH  ;D

Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: Adamantium on August 04, 2011, 03:34:02 PM
August 4th 2011 - Update from BT;

"Hi,

I have just cooped with the engineer who has been on site all morning. There is no fault on the line, they have checked all joints and cable, most of it has been renewed anyway. The engineer saw the dips but they are not caused by any line faults. The REIN engineer has been out with the SFI this morning and any REIN issues they were able to detect were actioned last week. There is no more that Openreach can do. The wiring has all been replaced, e and d side and there is nothing faulty for him to replace in any case. The port has been changed also so we have eliminated that.

The speed the customer is achieving is more than acceptable for this length of line and db loss, as for the dips there is nothing on the Openrach network causing this.

We have reached a stalemate with this as there is nothing else we can do, everything possible has been done for this customer. The line just goes and then comes back with no other reason other than being a completely random SHINE (single high impulse noise event).

We will not be able to resolve this, we do not have engineers that can find this.

The only thing I can suggest is that the router PSU be changed and maybe investigate any other electrical equipment with a power brick which might be on its way out but other than that we will not be able to help any further.

It would appear that this is down to the environment but there is no actual fault causing this.

Kind regards"


 :no: :'( :no: :'( :no: :'( :no: :'( :no: :'( :no: :'( :no: :(

P.S. Here's a screenshot of my routerstats capture for a period of 3 hours this morning...
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fi115.photobucket.com%2Falbums%2Fn284%2FAdamantium_2006%2FREIN%2FNoiseMargin-2011Aug04-0738.jpg&hash=dc69ea0b203169b8549be54fbed229a33d837f27)
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: cookieuk on August 09, 2011, 02:53:55 PM
August 9th 2011 - Get a life!
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: roseway on August 09, 2011, 03:10:48 PM
I'm not sure what you mean by that, cookieuk, but if you have nothing to contribute to the discussion it's probably best to say nothing.
Title: Re: Noise spikes on line, SNR Margin drops, REIN issue?
Post by: waltergmw on August 09, 2011, 04:14:32 PM
@ adamA,

The statements made by the support staff are clearly not logical. Given that, in electronic terms, there are really quite long periods of reasonable stability around 07:00 onwards where the trace stays much more at a reasonable operating level. There is definitely an external influence destroying most of your throughput for much of the night. I wonder if you might investigate, at least externally, visible lighting sources probably from discharge lamps such as street lighting or possibly PIR controlled building lighting? Also look for overhead phone lines running close to electricity cables.

It would be worth trying to trace possible RF Noise causes before continuing the battle.

Kind regards,
Walter