Kitz Forum

Broadband Related => FTTC and FTTP Issues => Topic started by: Crickers on February 23, 2013, 09:46:23 AM

Title: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 23, 2013, 09:46:23 AM
Hi Everyone,

A bit of background information, recently moved into a new house and had FTTC installed.

This took 8 engineer visits over a month, the final one being a boost engineer. I have lost days on the phone to BT and waiting for enigineer visits.

Faults included 18v battery fault, humming noise on the line, total loss of connection for 3 days. Also had a lift and shift at the FTTC cab, which resolved noise so far.

No JDSU has ever managed to sync on my line and the only modem to sync is a Huawei. Have tried ECI and my draytek 2750n, which worked fine at old house on FTTC.

Have had the underground cable to my house replaced with an overhead cable, after testing with a direct cable through the window from the pole gained 1Mb

I realise I have a long line at around 2.6km, but would really appreciate if someone could check my stats and see if anything looks unusal.

Spend quite a bit of time reading this forum, which has been invaluable and very informative.

Have attached a few logs using Bald_Eagles excellent scripts. The very last re-sync was me resetting the modem this morning.

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 23, 2013, 11:02:02 AM
Hi Crickers & welcome to the forum.


I'm not at all sure what speeds you should expect from a 2.6km line, but my gut feeling is that it looks 'reasonable'.

However, a slight problem can be seen in your Hlog graph.
It should ideally have a smooth curve. The plot at around tone 200 (roughly 860KHz) suggests a physical problem. Maybe a poor joint or even a bridged tap (although more of a 'V' shape would be usually seen if it were a bridged tap).

This 'problem' can be seen to be affecting bitloading (a gap) & SNR (a flat line) at the same frequencies.

QLN looks good, although that data is only obtained during the training up period from a resync, as is Hlog data.

Bitloading & SNR will vary dynamically for the duration of the connection between resyncs.

Running the snapshot script (Teststats2.BAT) at different times of day/night may paint a better/worse picture.

Your DS SNRM & Attainable rate vary quite a lot from around 17:00 each day (looking at the limited data so far), suggesting additional 'noise' from other users' evening connections and/or electrical interference from appliances, central heating etc.

I see from your Discovery Phase (Initial) Band Plan that you are connected to a Huawei cabinet DSLAM.

I also see that DS Interleaving was applied by DLM at a low depth just after 06:00 yesterday.
The depth then increased at a resync before 04:00 this morning, assumed to have been a DLM induced resync.

It then increased further when you initiated a resync this morning.
It is still at a reasonably low level (375) for a FTTC connection though.


The fact that Interleaving was OFF at first suggests that your connection has only recently become live. Is that the case?

You will see that error types swapped around when Interleaving was applied.
That means that some errors are being forwardly corrected (FEC) rather than needing data to be retransmitted.

The HG612 modem does seem to be very good at working on 'poorer' connections, hanging on to a connection even when SNRM is very low, even at negative values for short periods.

I suggest that you try not to force any more resyncs, at least for a week or so.
That will enable you/us to see how the connection performs when left to DLM to manage it.

Your current sync speed of 5816Kbps looks like DLM has not applied a 'banded' speed cap (yet), but it may do if many resyncs/high error counts are seen.

With a great deal of programming help from Burakkucat & from RONSKI who has developed a Windows fron end to control everything, I hope to release a new set of programs (hopefully NEXT weekend) that provide much more detailed data/graphs & can be easily fully automated for completely unattended monitoring/logging/scheduled graphing over very long periods of time (probably indefinitely).

It should be noted that the FEC & CRC error graphs from the script version, using incorrect data (as also seen in the HG612's buggy GUI) are incorrect.
RSCorr should be considered for FEC errors & OHFErr should be considered for CRC errors.


Overall then, for the line length, I would say your connection looks O.K.(ish) at the moment.

How does it actually feel in use, compared to ADSL?

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on February 23, 2013, 11:05:20 AM
FTTC will surely highlight line faults, as surely as a bear would visit the woods to ............  ;).

It sounds like you've had some good engineers on the job, to be fair. I won't bore you with our performance measures that dictate whether we get sacked or not.

As an aside, there's a little known function on the JDSU that most engineers will be unaware of. Whilst it is in 'boot mode' trying to handshake with the VDSL signal, the JDSU is auto-set to 'Annexe M'. By simply using the 'Action' key and from the drop down menu, 'Annexe A' can be selected which generally allows the JDSU to synch on very long D-side cable runs.
The only caveat I'll apply, is that I may have got the 'Annexes' the wrong way round, as I tend to be on auto-pilot when working on faults and don't take much notice of the wording.

Just out of interest, may I ask where you are in the UK ?? It sounds similar to a local fault around our way ?? :-) No problem at all if you don't wish to divulge any details.  :)
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 23, 2013, 03:47:37 PM
Thanks for responding so quickly guys.

Bald_Eagle,

The install was on the 23rd Jan, however it had ongoing issues.

The line had it's profile reset on the 19th Feb when the boost engineer came, installed new overhead cabling to the house and completed a lift and shift which had just been left by other engineer. So I'm guessing this is effectively the new install date, as all previous config/DLM stats would have been reset?

I have not resync'd since apart from this morning when I wanted to see if the sync speed would increase. Does the DLM process cause resync's?
I understand the DLM process needs to complete, so I will now leave it as you suggested for a week or so.

Interesting point regarding the Hlog graph, maybe not related but a TDR test showed a spike at around 60m, although this was not checked again after the overhead cable install. The engineer thought some of the issue may have been with the old underground cabling from the pole to my house, as we could not determine the entry point or even it's routing with in the house. Although I don't believe there could have possibly been 60m of cabling from pole to master socket, this has now been totally bypassed with the new cable straight to the master socket to which the modem is connected.

I will run some more snapshots over next few days.

I do understand the line length is the biggest limiting factor the speed of my connection, so I need to be realistic in my speed expectations.
However I want to ensure I get the best service possible that I'm a paying for, especially after the extremely poor overall customer experience I feel I have had so far from BT with myself chasing everything. Saying that a couple of engineers have been really great, going the extra mile to help out.

Black Sheep,

Interesting information about the JDSU as none of the engineers knew about that function, they all said it just wont sync under 15Mb.

My connection I believe is from the Braintree exchange EABNT cab P7.

The FTTC cab is located here https://maps.google.co.uk/maps?q=51.875859,0.513518&t=h&ie=UTF8&hl=en&view=map&ftid=0x47d8f1b7186fcd59:0x1a16bd8049cc3a5c&ftt=9&geocode=FTqQFwMd3dUHAA&split=0&sll=51.876200,0.516042&sspn=0.000653,0.005206&iwloc=A&ved=0CA8QpQY&sa=X&ei=H98oUfkfkdPxA47agNAG (https://maps.google.co.uk/maps?q=51.875859,0.513518&t=h&ie=UTF8&hl=en&view=map&ftid=0x47d8f1b7186fcd59:0x1a16bd8049cc3a5c&ftt=9&geocode=FTqQFwMd3dUHAA&split=0&sll=51.876200,0.516042&sspn=0.000653,0.005206&iwloc=A&ved=0CA8QpQY&sa=X&ei=H98oUfkfkdPxA47agNAG)

Have also attached full stats to present from the 19th Feb and latest sanpshot

Thanks again guys.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 23, 2013, 07:09:06 PM
Hello Crickers,

There is not a lot for me to say, as you have had replies from both the best data analyst and the best serving Openreach technician within the ranks of the Kitizens.

As Bald_Eagle1 has said there is, unfortunately, still a defect within your pair. I am, of course, referring to that spike (perhaps 'ringing' / oscillation) showing at around tone 180 (about 770 - 780 kHz) in the Hlog graph.

Just to clarify, was the spike at about 60 metres observed in the TDR trace before the new aerial feed was installed (replacing the old UG feed) and no TDR was subsequently performed on the newly provisioned drop-cable?

If that is the case, then the defect still present is somewhere from the pole-top DP outwards into the DP - PCP segment of the D-side.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 23, 2013, 07:26:00 PM
Hi burakkucat,

Correct, totally forgot to ask the engineer to re-run the TDR after the UG cable was replaced with aerial.

One other question on stats collecting, are there any scripts to collect stats for linux?
I have a ubutnu server running 24/7 which uses far less power than leaving my windows pc on, although would still use pc for graphing.

Thanks
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 23, 2013, 10:58:11 PM
Quote
One other question on stats collecting, are there any scripts to collect stats for linux?

Indeed, there is. As Baldy_Bird has previously mentioned, he is working on a utility for users of BGW, which has origins in some code that emanated from The Cattery.

If you would be happy compiling the code and creating a cron job to perform the scheduled sampling, then I can let you have a copy of the HG612_stats sources. That will then log the data into your /var/log/ directory and BE's original scripts should be able to produce the graphs.  :)

:hmm:  Thinking about it, I know that Mr Eagle has 'munged' the format of how the data is stored for his BGW code adoption, so before we go any further, perhaps we should 'whistle up' No-Feathers and see what he thinks will be the best way forward.  :-\
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 24, 2013, 06:48:52 AM
:hmm:  Thinking about it, I know that Mr Eagle has 'munged' the format of how the data is stored for his BGW code adoption, so before we go any further, perhaps we should 'whistle up' No-Feathers and see what he thinks will be the best way forward.  :-\

I am just adding the final touches to the code, ready for hopefully releasing the Windows version next weekend.

Although starting off as cross-platform compatible code, to get to the stage it's now at (manual or fully unattended logging/scheduled graphing/front end controlling GUI etc.) it has no doubt veered way off course in terms of Linux compatibility.

However, as the output logs are still plain space delimited text files (at the moment) it may be a reasonably simple matter to add the VDSL2 output format code to a version that still remains reasonably compatible.

Until the graphing elements have been added (for ongoing stats plotting) for Linux users, it would still be a case of graphing the output via a Windows system.

It has always been the intention to convert the code back to being compatible for use on Linux systems, once it's been fully tested and reliable.
I'm not the person with the knowledge of how to do that though.

So, in the short term I would stick with the Windows version when it's released.
Longer term, subject to sufficient interest, maybe you would fancy having a go at 'downgrading' the code for Linux users  ::)

The Windows programs MAY work in a Windows VM and/or by using Wine on a Linux box. That hasn't been fully tested.
They have been tested on Windows machines starting with XP Home Edition through to Windows 7 & related Home Server Editions.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: roseway on February 24, 2013, 07:42:11 AM
[Humble mode]I hope you'll forgive me for mentioning an alternative means of collecting data on a Linux machine - rs-ux  (http://rsux.plainroad.me.uk ) [/Humble]
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 24, 2013, 08:28:42 AM
[Humble mode]I hope you'll forgive me for mentioning an alternative means of collecting data on a Linux machine - rs-ux  (http://rsux.plainroad.me.uk ) [/Humble]

There's no need to be humble, Eric.

Your rs-ux program (& its partner rs-w for Windows) are excellent tools & the last time I checked, can run happily alongside my programs, as long as they are staggered by a few seconds - to avoid multiple attempts to access the modem's stats at the same time.

Our programs operate somewhat differently & are designed for slightly different purposes.
Together, they now seem to provide all the monitoring connection data & graphs necessary for general information purposes & for longer term trouble-shooting/fault reporting evidence (albeit mine are intentionally geared only for the HG612 modem - at the moment).

 
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 24, 2013, 10:43:56 AM
Thanks everyone for the replies about linux stats collecting.

I'm no linux programmer, compiling code would be about my limit I think.
My ubuntu server is headless so the aim to be to collect stats via linux then graph the data with windows.
Would be happy to help wherever I can though, windows or linux.

burakkucat, a copy of the HG612_stats sources would be great thanks.

roseway, thanks for link I'll check it out.

Bald_Eagle, I'm thinking if I can collect and format the output via linux, it should work with your existing and new graphing elements. Will also set up a windows VM and try that.

Have attached todays snapshot and ongoing stats.

One last question, the spike which is seen on the Hlog graph, is this something I should raise as a fault or just live with?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 24, 2013, 12:40:51 PM

Bald_Eagle, I'm thinking if I can collect and format the output via linux, it should work with your existing and new graphing elements. Will also set up a windows VM and try that.


If the new programs work reliably in VM, that would be the initial way to go as the Linux version doesn't yet calculate & store the error count differences between each one minute sample (unless burakkucat has been busy updating it), or create the output fileds in the logs in the same format as the Windows version.

However, once the Linux version produces ongoing & current logs in an identical format to the Windows version, they will indeed be able to be plotted using the Windows version.


Quote

One last question, the spike which is seen on the Hlog graph, is this something I should raise as a fault or just live with?


That's hard to say.
It certainly won't be helping matters, but on such a long line, who knows?

What were your estimated speeds pre-installation & have they now changed since your connection beaome live?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: asbokid on February 24, 2013, 01:13:39 PM
That's an interesting and unusual spike in the HLog (attenuation) graph.  It corresponds roughly to DMT#180.   It's actually a trough followed by a peak (or vice versa) which probably says more about its cause.
The QLN graph shows a moderate amount of frequency-specific noise, mostly in the AM broadcast radio band, but also at higher frequencies. There is RFI around the 7MHz range, which could correspond to the 40m ham radio band.   Is your line pole-strung for long distances, where it would act as an unwanted antenna for these signals?
The SNR graph is unusual, too.  Although you have an average SNR of some 6dB,  in practice, the SNRs on many DMTs are much higher, as high as 20dB or even 30dB.   The aggegrated SNR of 6dB masks that fact.
And yet Bit depth in Downstream band D1 - the most reliable downstream band for long-loops - is zero across DMT#120 - DMT#230 (approx). That's largely why your throughput is "pants".

A very unusual (and unwanted) set of characteristics!   As a rookie linesman, I would say it points to a line fault.  A time-domain reflectometer would soon reveal whether there's a bad joint, partial short from water ingress, etc..

cheers, a
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 24, 2013, 02:03:33 PM
My pre-install speed estimate was 8.5Mb, checking http://www.dslchecker.bt.com today says upto 8.2Mb.

Have never seen over 6Mb, speed generally seems to be 5.0-5.7Mb over the last few days since the boost engineer visit.

I live in a rural location and can see line is pole-strung for at least 1.5km along the road, although I suspect it might be more.
Would RFI from ham radio be intermittent? i.e just during transmission?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: asbokid on February 24, 2013, 03:04:44 PM
How are the estimates calculated?  Possibly from Hlog. That is what is called "electrical loop length", not physical loop length.
The estimates can be quite wide of the mark.   BT estimates this ADSL2+ line can achieve just 5.5Mbps - 12Mbps. Yet there is a pleasant surprise:

(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fwww1.picturepush.com%2Fphoto%2Fa%2F12270959%2Fimg%2F2hbline%2Fbt-infinity-2hb-estimate.jpg&hash=78f861bc53f3a046c1650ee3de5eb18d78cc53ae) (http://picturepush.com/public/12270959)

True performance, albeit with a reduced SNRM of 3dB and with an LLU provider rather than with BT, is closer to 20Mbps:

(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fwww2.picturepush.com%2Fphoto%2Fa%2F12270990%2Fimg%2F2hbline%2FScreenshot-from-2013-02-24-15%253A02%253A57.png&hash=5191630041b3406b001df7e6334694020fa67850) (http://picturepush.com/public/12270990)

cheers, a
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: asbokid on February 24, 2013, 03:13:45 PM

RFI from ham radio should only be present when the radio enthusiast "keys up".  However this is shortwave, where propagation can be thousands of miles. And the lowlands of north Essex are well known for their good radio reception, wanted or otherwise!  When we lived there, it was commonplace to receive UHF TV transmissions from the Netherlands!   Just the luck of the draw, or not!

cheers, a
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 24, 2013, 07:30:40 PM
burakkucat, a copy of the HG612_stats sources would be great thanks.

Please let me know, by PM, of an e-mail address to which I may dispatch the above.

Quote
One last question, the spike which is seen on the Hlog graph, is this something I should raise as a fault or just live with?

It would irk me, if I saw it for my line and I would investigate all possible causes for its appearance. Once I could prove the cause was an infrastructure defect, then I would report it and keep discrete pressure applied until the fault was rectified.

I wonder if Black Sheep has seen such a graphical abnormality before, in his professional capacity?  :-\  Let's wait for his input . . .
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 24, 2013, 07:38:00 PM
That's an interesting and unusual spike in the HLog (attenuation) graph.  It corresponds roughly to DMT#180.   It's actually a trough followed by a peak (or vice versa) which probably says more about its cause.

You observation of a trough & peak fits in with my comment about 'ringing', albeit highly selective to a precise frequency.

I wonder if the frequency of that spike can be transformed into a distance? And then I wonder if it would agree with the ~60 metres noted on the TDR trace, pre-replacement of the service feed?  :-\
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: NewtronStar on February 24, 2013, 07:49:18 PM

RFI from ham radio should only be present when the radio enthusiast "keys up".

it's a funny subject this now Ham radio users are being subject to interference by HomePlugs
and I can confirm this after two days trying to find the source  ;)

http://www.pcpro.co.uk/realworld/350648/does-powerline-networking-nuke-radio-hams
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on February 24, 2013, 09:05:40 PM
As most of us know, a 'peak' on the display screen when using a Time-Domain Reflectometer (TDR), either as a function of a multi-meter (such as a JDSU), or as a stand-alone tester, is generally indicative of high resistance. The higher the peak, the greater the resistance with an 'Open circuit' practically showing the 'peak' touching the top of the screen.

However, it can also show a 'Poundage change' between jointed cables. IE: The original UG feed to the house may well have been 0.4mm diameter wire, jointed to a 0.9mm diameter OH cable at the pole ??

Also, just an observation, but if I had my mate (an excellent engineer) stood at your premises, and asked him to estimate the distance to the pole, I can tell you now he'd be 30-40mtrs wide of the mark. That's why we have pedometers. What I'm trying to say, is judging distances is pretty hard, and the new drop-wire could well be 60mtrs long ?? Like I say, only an observation.

What is interesting is the comment about a 'trough' followed by a 'peak', this when presented on a TDR screen is more often than not a wet-joint. I've tried to find a picture to show this, but to no avail. >:(. Did the engineers trace actually show this, or is this just derived from HLog data ??

 
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 24, 2013, 10:53:00 PM
I do use homeplugs, have turned a couple of them off and attached a quick TestStats2. The other is in the garage and its too late to mess around tonight, will run some more stats with them powered off tomorrow afternoon.

Will also measure the length of the new drop cable that has been installed, the pole is just at the edge of my drive. I'm guessing no more than 25m, but I'll confirm tomorrow with a tape measure. This replaced an unknown length of UG cable as we could not determine the routing from pole to master socket, so this indeed could account for the 60m peak.

Stupidly I didn't really pay much attention to the engineers trace at the time, seem to recall a slight dip before the peak but can't be certain and even more annoying that another TDR trace was not completed after the new drop wire was installed.

Hopefully I'm better educated now so will be more observant in future  ;D

Also now have the linux sources so will have a play tomorrow, thanks burakkucat.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 24, 2013, 11:33:13 PM
I've had a quick look through my JDSU & Exfo documentation and, just like Black Sheep, I couldn't find the example for which I was looking!  :-X

I have snipped two example graphs from an Exfo document (as this is for educational and research purposes, I do not expect the company to niggle over copyright) and they are attached, below.

Finally I have expanded the Hlog graph obtained from Crickers line and I think we will all agree that the trace drops sharply before rising sharply. So Asbokid was quite correct in describing it as a trough followed by a peak.  :)

Quote
Also now have the linux sources so will have a play tomorrow, thanks burakkucat.

You are welcome. One quick comment. I have provided you with the means to harvest ongoing statistics, once per minute. To produce pretty graphs from the logged data, you will need to use one of Bald_Eagle1's scripts. I last used that harvesting code back in December (2012) when I was using a Raspberry Pi to monitor my line. At that time I was plagued by RFI hitting my US frequencies and I put it down to being in the 'flashing lights' season. I distinctly remember sending a weeks worth of logged data to The Eagle's Nest but the promised analysis has never appeared!  :(
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 25, 2013, 12:25:32 AM
Here's a Hlog graph from before the UG cable was replaced.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 25, 2013, 01:16:20 AM
Here's a Hlog graph from before the UG cable was replaced.

And we see a peak followed by a trough, then some 'fade-out' 'ringing'. I'm not sure what to make of that trace.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 25, 2013, 07:32:16 AM
I distinctly remember sending a weeks worth of logged data to The Eagle's Nest but the promised analysis has never appeared!  :(


Apologies for that. It seems to have dropped completely off the radar.

I have just run your 7 day ADSL modem_stats.log through my graphing program, but as the fields are not in the same order as those used for VDSL2 connections, the output is completely incorrect (see attached).

Either 2 separate programs would be required, or the program would need to determine the DSL mode & format the output accordingly to allow the one program to work for different DSL modes.

Also, the Windows version of the VDSL2 ongoing stats harvesting program calculates 'delta' values at harvesting time, storing the data in the modem_stats.log

I believe your Linux version doesn't do that at this time.
That element of the code MAY be more or less compatible across both platforms, so maybe it could be reasonably easily added to the Linux version's code?

I'd be happy to share the 'C' code for the Windows versions if that would assist anyone for conversion to Linux.
It does use some system calls, other programs & specially written utilites though.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 25, 2013, 09:01:45 AM
Apologies for that. It seems to have dropped completely off the radar.

I appreciate that Mrs Eagle was giving you a hard time over the refurbishments of The Eagle's Nest (which was supposed to have been finished by Christmas 2011 and was still not going to be finished by Christmas 2012), so I just kept quiet.  :-X

Quote
I have just run your 7 day ADSL modem_stats.log through my graphing program, but as the fields are not in the same order as those used for VDSL2 connections, the output is completely incorrect (see attached).

 :hmm:  Hmm . . . If only you could re-base your fork of the utility on the latest version (0.20, from last December), then there should not be any discrepancy.

Quote
Either 2 separate programs would be required, or the program would need to determine the DSL mode & format the output accordingly to allow the one program to work for different DSL modes.

I will remind you that I had to spend many hours putting conditional code into the harvesting utility so that the correct data was recorded for G.Dmt, ADSL2, ADSL2+ and VDSL2 services.  >:(

Quote
Also, the Windows version of the VDSL2 ongoing stats harvesting program calculates 'delta' values at harvesting time, storing the data in the modem_stats.log

I believe your Linux version doesn't do that at this time.

Correct.  :)  I recall recommending the use of a separate structure, 'HG612_deltas', for their storage. That should appear in version 0.21, when I get 'a round tuit'.  :angel:

Quote
I'd be happy to share the 'C' code for the Windows versions if that would assist anyone for conversion to Linux.

I have been 'threatened' with that, before, by you and I am still awaiting for its materialisation.

Quote
It does use some system calls, other programs & specially written utilites though.

You should be familiar with the --

Quote
#ifdef          __MINGW32__
// Do BGW style thingies.
#else
// Do sophisticated, quality *x code.
#endif

-- as shown many, many months ago by Asbokid. Hence all your 'doze specific code can be eliminated by the C-preprocessor when the source is compiled for -- er -- proper systems!  :P
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: kitz on February 25, 2013, 10:05:45 AM
 :o
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 25, 2013, 11:13:29 AM
Now, now, children play nicely  ;)

Have measured drop cable length from pole to master socket. I make it 24m, measured across the ground and then followed along walls etc. Think I'm within 2m of the actual length, so not even half the 60m distance seen on the TDR.

Also powered down all homeplugs at 10:30am and re-run TestStats2 (attached).

Had resync at 4am this morning, part of the DLM I guess?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on February 25, 2013, 01:00:43 PM
Wrong side of the bed this morning, B*Cat ??  :)
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: asbokid on February 25, 2013, 03:07:59 PM
Not sure where the diagram below came from, but the troughs and peaks, and the fault type they signify, are all captured in the time domain

(Time runs along the x axis, while magnitude is up the y axis)

Whereas the spike in Crickers' Hlog graph is captured in the (discrete) frequency domain.

(Discrete tone frequencies along the x axis and magnitude up the y axis).

Also below are screenshots from TDR devices showing a fault on the line.  A branch rubbing on the drop-wire caused a short just before the pole.   Credit where due, Openreach engineers sorted it out very efficiently:

(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fwww2.picturepush.com%2Fphoto%2Fa%2F12279540%2Foimg%2F2hbline%2Fshort-circuit-canoga-megger.annot.640.jpg&hash=877fcdfa356cb7d5a1c817e8abdb0258f1689398) (http://picturepush.com/public/12279540)

cheers, a
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 25, 2013, 03:40:00 PM
Right, so who's got a TDR and likes tea & biscuits?  :thumbs:

Thanks for the info asbokid.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on February 25, 2013, 06:39:03 PM

Quote
I'd be happy to share the 'C' code for the Windows versions if that would assist anyone for conversion to Linux.

I have been 'threatened' with that, before, by you and I am still awaiting for its materialisation.


That is no longer an idle threat, as you will see when you read through your emails ;)

As always, constructive criticism/advice is/are most welcomed.

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 25, 2013, 08:21:12 PM
Wrong side of the bed this morning, B*Cat ??  :)

No.  :no:  I hadn't made use of it at the time I posted!   :o

The Eagle and the Cat have a good understanding and 'get on' well together. (You should see the debris on the floor when we have concluded a discussion -- clumps of plucked fur and pulled feathers everywhere!  :)  )  I feel somewhat guilty that we have hijacked Crickers thread with our discussion of coding internals . . .  :-[
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on February 25, 2013, 08:22:18 PM
 ;D
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 25, 2013, 08:29:56 PM
Right, so who's got a TDR and likes tea & biscuits?  :thumbs:

To the best of my knowledge, in alphabetical order --Bacon sandwiches are also appreciated!  :yum:
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 25, 2013, 08:30:19 PM
Carry on, I love a good ruck. Thread will still be here once dust has settled.  :shoot: :shoot:
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 25, 2013, 08:50:51 PM
Any one got any ideas about progressing this, but without a TDR test or is this logical next step?
I'll keep logging stats, and as I guess the DLM process is still ongoing.
 
One other question, but what impact does internal cabling have if connected at the master socket using the NTE5 faceplate? Guessing not much as it doesn't seem to make any difference if connected or not.

Second other question, are Hlog stats collected at re sync only or against some other function?

Thanks me done, wine to drink  :drink:
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 26, 2013, 12:31:19 AM
Quote
Second other question, are Hlog stats collected at re sync only or against some other function?

The data is collated only at active-CPE / DSLAM synchronisation time.

Quote
One other question, but what impact does internal cabling have if connected at the master socket using the NTE5 faceplate? Guessing not much as it doesn't seem to make any difference if connected or not.

When a SSFP is fitted to a NTE5/A it completely isolates the xDSL signals from the telephony wiring. In other words, once the interstitial SSFP has been plugged into the NTE5/A, the only signals present at its lower right front socket are telephony. Hence the fitting of the original front face-plate, with telephony extension wiring connected, will not affect the xDSL circuit's balance and performance.

Quote
I'll keep logging stats, and as I guess the DLM process is still ongoing.

The Openreach DLM is continuously monitoring the line whilst the service is deployed. Bald_Eagle1 will be able to quote you the official wording on the subject, including how it will intervene -- if it considers such intervention to be necessary.

Quote
Any one got any ideas about progressing this, but without a TDR test or is this logical next step?

That question is best left for Black Sheep's (or perhaps Ezzer's) response.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: waltergmw on February 26, 2013, 12:33:15 AM
@ Crickers,

Extension wiring, if properly connected, doesn't seem to have much effect as it is after the filter circuit provided an effective SSFP has been fitted.
(Beware NOT to connect extension wiring to the two terminals meant for a VDSL extension socket though. EDIT I.e. the connector pair shown in BKK's second picture above should ONLY be used when the VDSL modem is connected remotely)

However if the master socket is not wired properly, or as several subcontractors have done around us with NO adjustments at all, quite horrendous performance follows usually leaving the poor EU with a massive battle. That is until Uncle Walter's wheelbarrow is deployed, ably assisted by remote cats and birds !

Kind regards,
Walter

PS Re a TDR; it may not come as too much of a surprise to note that Essex and Suffolk are in reasonably close proximity.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on February 26, 2013, 07:31:00 AM
 You might be able to get your ISP to perform a CIDT (Copper Integrated Digital Test). This initially performs a low-frequency test of the copper pair from the Exchange to your premises, which is extremely unlikely to show a HR if it's in it's infancy. However, it then goes on to ping your router at varying frequencies, and it's this part of the test that may see a HR, if indeed one does exist. 
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on February 28, 2013, 09:53:10 PM
Been really busy over the last few days...

Here's stats for the last week. Doesn't seem have adjusted sync speed for the last 3 days and looks like interleaving has dropped back down.

Wondering if DLM will re-adjust again, or if a reboot might be an idea?

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: asbokid on February 28, 2013, 10:51:36 PM
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fwww2.picturepush.com%2Fphoto%2Fa%2F12305895%2F480%2Fcrickers%2Fperiodic-noise.jpg&hash=82558e81245f46218e6b341f6ff83d72b0d66d27) (http://picturepush.com/public/12305895)

It should be possible to tell if the RFI is caused by an outside lighting circuit switched by a photocell.

As the days get longer, and dawn comes earlier while dusk falls later, the 'phase' of the RFI noise should shift.

The 'falling edge' of the noisy period should shift to the right, and the 'rising edge' should shift to the left.
Meaning the overall duration of that daily noisy period should shorten.

That's the theory any way!

cheers, a

EDIT: dawn before dusk!
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on February 28, 2013, 11:09:46 PM
All I shall say is that there is something definitely 'wrong' with that line.  :-X

Now where is Feathers and his Eagle-eye of analysis?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on March 01, 2013, 08:45:52 AM
A reboot of the modem this morning has given me 6.04Mb on speedtest.net, never been over 5.5Mb before.  :fingers:

Still seeing the spike in Hlog.

Won't be online much for the next week, but will gather stats and leave DLM to work things out.

Then once this is done, will raise another fault if required with BT.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on August 26, 2014, 04:52:03 PM
Well an update is long over due.....

Speed has been slowly deteriorating for some time but settled around 4.3Mb until an outage around which then resulted a speed of 2.5Mb after several frustrating calls to India.

Anyway after trying to resolve the speed issue via the normal process for 2 weeks I lost the plot (threw every dolly out of the pram) and emailed someone in BT which was redirected to Newscastle Technical Complaints team and got a response within a day.

So after another two engineer visit's and a swap to a new pair from the FTTC cab, the line speed is now around 4.7Mb.

But I still see missing bit loading and a strange Hlog graph. I have forwarded this info to BT asking for some explanation, but they are now saying they are unable to use the information as it has been obtained by using non standard firmware. Pretty poor excuse if you ask me.

As it stands no engineer has ever been able to get a JDSU to sync, I even asked the last engineer to change the annexe as suggested previously in this thread, but he was unaware of this setting and unable to find it in the menu. Are the JDSU's locked down in some way or maybe just lack of JDSU knowledge?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on August 26, 2014, 05:50:49 PM

So after another two engineer visit's and a swap to a new pair from the FTTC cab, the line speed is now around 4.7Mb.

But I still see missing bit loading and a strange Hlog graph. I have forwarded this info to BT asking for some explanation, but they are now saying they are unable to use the information as it has been obtained by using non standard firmware. Pretty poor excuse if you ask me.


Personally, I believe the non standard firmware is a smoke screen/excuse for doing nothing.

It is basically BT's own firmware, unlocked via removing firewall restrictons to provide access to connection stats.


You certainly do have an unusual looking Hlog graph.

A downward 'V' shape would usually indicate a bridged tap, but I'm not sure what the spike at around tone 250 indicates.



Quote
As it stands no engineer has ever been able to get a JDSU to sync, I even asked the last engineer to change the annexe as suggested previously in this thread, but he was unaware of this setting and unable to find it in the menu. Are the JDSU's locked down in some way or maybe just lack of JDSU knowledge?


At approaching 3km, I wouldn't expect great VDSL2 sync speeds at the best of times.
It has been mentioned to me by visiting engineers that the JDSU can present rather a pessimistic view of a connection & it is not unknown for them to fail to sync on long lines, even though the HG612 can indeed sync.

However, that Hlog graph & the bitloading gaps etc. do indicate some sort of line problem to me.
The Hlog graph should really be a steady downward curve as frequency increases.



I'm not at all sure how you could pursue further investigations.
Maybe an email to BTOR's CEO office would escalate matters?

I don't have the email address, but someone else reading this thread may be able to provide it for you.

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on August 26, 2014, 06:22:59 PM
Joe Garner, Openreach CEO -- profile (http://www.openreach.co.uk/orpg/home/aboutus/ourorganisation/ourexecutiveteam/executiveteam.do) -- e-mail: Joe.Garner <AT> openreach.co.uk

Gavin Patterson, BT Group CEO -- profile (http://www.btplc.com/Thegroup/Ourcompany/Theboard/Ourboard/GavinPatterson/) -- e-mail: Gavin.E.Patterson <AT> bt.com

 . . . if my memory serves me correctly.  :angel:
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on August 26, 2014, 07:41:38 PM
Thanks it was the email to Gavin that prompted the quick response from Newcastle.

I think another engineer visit will be arranged as the issue has not been closed yet.

Sent from my Nexus 5 using Tapatalk

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on August 26, 2014, 07:54:57 PM
Could ADSL cause the loss of tones in the D1 range? Think it might be about 6km to the exchange.

Sent from my Nexus 5 using Tapatalk

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: renluop on August 26, 2014, 08:34:47 PM
Has anyone noticed that all asbo's links are coming up file not found?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: burakkucat on August 26, 2014, 08:51:39 PM
Not all links, just those of photographs via a hosting website.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Bald_Eagle1 on August 26, 2014, 09:08:22 PM
Could ADSL cause the loss of tones in the D1 range? Think it might be about 6km to the exchange.



It could be or it could be a power reduction on your shorter VDSL2 connection to avoid your connection completely swamping already quite weak ADSL connections.

I'm around 5.4km from the exchange & could only achieve 1 Mbps ADSL connection speed, on a good day.
More often than not, it was lower than 1 Mbps.


I note your TX power is only 8.7 dBm in the D1 band, whereas it is usually around 11 to 12 dBm or so for VDSL2 connections.
i.e it looks as though it has been intentionally cut back.

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: NewtronStar on August 26, 2014, 10:47:28 PM
(https://forum.kitz.co.uk/proxy.php?request=http%3A%2F%2Fwww2.picturepush.com%2Fphoto%2Fa%2F12305895%2F480%2Fcrickers%2Fperiodic-noise.jpg&hash=82558e81245f46218e6b341f6ff83d72b0d66d27) (http://picturepush.com/public/12305895)

It should be possible to tell if the RFI is caused by an outside lighting circuit switched by a photocell.

As the days get longer, and dawn comes earlier while dusk falls later, the 'phase' of the RFI noise should shift.

The 'falling edge' of the noisy period should shift to the right, and the 'rising edge' should shift to the left.
Meaning the overall duration of that daily noisy period should shorten.

That's the theory any way!

cheers, a

EDIT: dawn before dusk!

It's called Radio propagation you will see the effects 1 hour before sunset, and as the sun is setting earlier (going back to winter) the SNRM dive time will become earlier.

For example RFI hit's my modem at 21:45 pm on June the 21st
and the RFI hit's my modem at 15:45 pm on December the 20th

Yours sincerely

NS
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on August 30, 2014, 08:46:33 AM
Another engineer visit booked for Weds, so will hope I get one who can use a JDSU outside of the auto settings......

Have also just registered to started uploading to mydslwebstats. Great idea  :)
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 02, 2014, 10:24:12 AM
Now uploading stats, do I win a price for having the slowest connection by a factor of ten?  :( :( :(
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 02, 2014, 01:40:30 PM
Found some info about syncing the JDSU on long lines over 1.6km as mentioned previously by changing the Annex....

Press the VDSL key ........ press the option to start the 'Modem training' just as you would any other time, when the screen appears where it's attempting to 'handshake', press the 'Config' button. Now, where it says 'Annex' (usually at No.4 or 5 in the menu), click on this. Change the setting from 'Auto' to 'Annex A'. Shazam ...... your JDSU will synch as far as VDSL limitations will take you. This takes approx 1 minute to do once the JDSU is switched on and ready to go.

PS ........ you will need to change the annex back to 'Auto' once you have done this, as it will remain set at 'Annex A' otherwise.

Maybe someone with a JDSU can confirm this works?
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on September 02, 2014, 07:17:36 PM
Yes, as I pointed out right at the very beginning of this thread, Crickers.  ;) ;D
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 02, 2014, 07:44:59 PM
Yep, not trying to steal your credit for pointing it out, but the last BT engineer couldn't get out of Auto mode. So trying to add some info for future reference and my engineer visit tomorrow.

Sent from my Nexus 5 using Tapatalk

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on September 02, 2014, 08:09:55 PM
 :)
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 03, 2014, 06:42:31 PM
Well the biggest communications company in the UK has ZERO communication skills !!!!!!!
Engineer no show and after I called to follow up, apparently he couldn't gain access....which is strange considering I've been at home waiting all day....

Sent from my Nexus 5 using Tapatalk

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Black Sheep on September 03, 2014, 07:21:58 PM
Did you leave an alternative number for the engineer to ring, i.e.: your mobile phone ??

I'm not justifying what may have gone on with your visit, as it is bang out of order if you've been the victim of a 'Can't be arsed' engineer. However, I can tell you from personal experience that on quite a few occasions over the years, no amount of banging on the door or ringing the landline resulted in the EU giving access.

"I was in the garden", "I don't answer numbers I don't know", "I just nipped out" ……….. are excuses given if I manage to get them on their mobile number.

The only other issue could be the EU being a proper oining sod ?. I've done it myself with an EU on my own patch. He was konrado's cousin, I'm sure ??  ;) ;D ;D. Before anyone jumps down my throat, there's a heck of a lot of history to that particular EU's story.

To sum up though, it does appear on the face of it your line-length may have put a feint-hearted engineer off attending. Especially if he's on a 'Performance Plan' for low productivity. He could be a world-class engineer, but that means jack in todays BT. More prod, more prod ….. is the mantra.  ::) :-X :no:
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 03, 2014, 08:58:08 PM
Yep, all contact details given.  I was sat at my desk which overlooks the drive for the entire time.

I try to be nice to all engineers who come visit, at least offer a cup of tea and maybe a biscuit too. I try to treat everyone as I expect to be treated myself. All the engineers have been good so far, except one.

Its just got to the point where I think the issues which can be seen from my stats & graphs need someone with a greater skill set than has attended so far.

Sent from my Nexus 5 using Tapatalk

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 10, 2014, 03:10:13 PM
Well same engineer as previously, although very helpful was hoping for a more technical engineer if such a person exists within BT.

Found a battery fault on the line which wasn't there 4 weeks ago and resolved by swapping pairs on a section of overhead cable.

Speed has increased from 4.7 to 5.8 Mbps although I expect this to drop again once DLM has it's wicked way.

Finally managed to get the JDSU sync'd although only at 1.8 Mbps by changing the annexe used from Auto to A using the instructions provided.

Same issues still can be seen in the bitloading and hlog graphs and still have no answer what is causing this.

I will accept if this cannot be resolved, but what is unacceptable is not providing any reason for it and basically saying as the customer I should not be questioning anything.

Stats on mydslwebstats
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 11, 2014, 01:59:01 PM
Bald Eagle mentioned power cut back in a previous post, which I understand is to prevent adsl connections being impacted.

Am I correct in thinking it would be applied across the entire D1 range rather than the specific tones that are missing?

Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Chrysalis on September 11, 2014, 03:31:35 PM
the tones affected depends on your distance from exchange.

Apparently the worst range is sort of mid to long range so the distance is not long enough to stop all of the adsl1 range been used but still long enough that those tones are weak and require heavy vdsl power cutback.  My area is like that.  If the range is really long where the users only get to use a tiny amount of adsl tones then only those tones would need power cutback.  Whilst if its really short the adsl1 tones are so strong on adsl they dont need power cutback and usually it would just be the upper adsl2+ tones but even then its a weak cutback as the signal on adsl for low attenuation is much stronger than for high attenuation.

It would be very unusual for the entire D1 to have power cutback applied. As D1 is bigger than the entire adsl2+ range. With that said when the VDSL is really long like yours, then you dont even get the full D1 range usually it goes to about 860 or so..

Your bitloading indicates power cutback up to about tone 250 which is the full adsl1 range.  It looks heavy as well.  Probably losing you at least 8mbit/sync in my opinion but quite possibly more.  The problem is to remove it BT would probably need to move all adsl peeps to the cabinet as well, which isnt a good business case for them, it makes VDSL a harder sell and they have the expense of moving more hardware to the cabinet. It would also upset ofcom, as ofcom are in love with LLU.
Title: Re: FTTC long line 2.6km - Stats check after problems
Post by: Crickers on September 12, 2014, 10:28:50 AM
So hopefully final update.

BT have come back with explanation of the missing tones and as expected it's due to power cut back for ADSL tones.

Thanks everyone for the info regarding this, one question remains would this cause the spike seen in the hlog graph centred on the missing tones?

It's been frustrating that it has taken this long to get an answer, BT are not the easiest company to deal with and seem to generally treat customers as idiots who should unquestionably accept things. I will however counter this by saying my contact in Newcastle has been very good, kept me informed at all times and has resolved things to my satisfaction.

I don't think other than checking every single joint and cable anything more can be done to resolve the speed issue at present. Still only obtaining half the lowest impacted speed estimate given by the checker though.

The line was upto 6Mbps but I guess DLM has now kicked in and it's back down to 5Mbps, hopefully it will stabilise and not drop down further.

Would still like to have someone with a JDSU who actually knows how to use it look at my line. Open offer with tea, biscuits and maybe a bacon butty if anyone's ever passing.