Kitz Forum

Broadband Related => Broadband Technology => Topic started by: sevenlayermuddle on August 16, 2010, 11:23:29 AM

Title: DLM, beaten again?
Post by: sevenlayermuddle on August 16, 2010, 11:23:29 AM
As mentioned in another thread, I recently got stuffed by a 15dB target caused by (I suspect and hope) a duff router that I'm no longer using.  Past experience with DLM suggested I held little hope of the margin coming back down again anytime soon.  Last time it stayed at 15dB for a whole year, before I tried to 'trick' it into action.  I've just repeated the same little trick...

Exactly three weeks ago I tweaked the margin up to circa 18dB.   That resulted in an ES rate of about 5 to 10 a day, but somedays I was seeing maybe 20 ES, and my hunch was that wouldn't be good enough.  A week later (two weeks ago) I tweaked it up further to  circa 22dB, which brought the error rate down to about 5 ES per day on average.

This morning, guess what...?  Yep, target margin's been reduced to 12dB.   :thumbs:

I know, I can't exclude the possibility that it was pure co-incidence, and the target would have come down without my trickery.  Personally my hunch is that it wouldn't but, understandably, I suspect not everybody will be convinced.   Anyway, I'll tweak it back up again, and see what happens next.....

- 7LM
Title: Re: DLM, beaten again?
Post by: waltergmw on August 16, 2010, 12:07:13 PM
@ 7LM,

VERY WELL DONE INDEED !

With your continual perseverance you might just have found a solution to the problem - that is until the DLM logic is changed again!

Kind regards,
Walter
Title: Re: DLM, beaten again?
Post by: roseway on August 16, 2010, 12:52:46 PM
Yes, I'm really quite impressed with your dedication, 7LM. There aren't many people who would voluntarily tweak their noise margin up, and it does very much look as though it pays off.
Title: Re: DLM, beaten again?
Post by: BritBrat on August 16, 2010, 01:02:01 PM
Could you explain how you did it.

Thanks.
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 16, 2010, 01:22:25 PM
Could you explain how you did it.

Thanks.

I currently have a DG834GT, running standard Netgear firmware.  The process for tweaking target margin is described here http://www.kitz.co.uk/routers/dg834GT_targetsnr.htm

There's probably other ways, such as using the GUIs in Routerstats etc, but I'm more at home with a command-line (something to do with my age!).   In my case, adslctl configure --snr190 resulted in an actual margin of about 21dB immediately after reconnecting (at night), drifting up to 22dB in the daytime.

Obviously I can't guarantee it'll work for anybody else, try it at your own risk!

And you do have to put up with life in the s-l-o-w lane during the experiment, my connection speed was 1792 at a margin of 22dB, vs upwards of 2700 with the 15dB margin.
Title: Re: DLM, beaten again?
Post by: jeffbb on August 16, 2010, 09:29:26 PM
Hi
Lucky you I  had very low errors for months ,< 2 CRCs and ~1 ES /24hr(tweaked to 9db from 15 db) . Target SNR did not drop from 15db.

Current situation Had some bad weather a few days ago had about 250 CRCs over a period of about  6Hrs ,since then back to normal low errors .

Regards Jeff
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 16, 2010, 10:45:11 PM
Jeff,

Sorry you've not been so lucky

A possible explanation for the difference is that DLM maybe keeps an eye on the actual SNR margin and, to qualify for a target reduction, expects it to be 'stable', in addition to low error rates.  By 'stable', I mean that  the actual SNRM reported by the router should rarely be seen to dip below the target set by DLM.  If DLM sees an SNMR that's been tweaked downwards, so it's always below the target DLM has set, it may decide the stability criteria have not been met, regardless of error rates.

In contrast, a target that's tweaked upwards may lead DLM to believe that the SNRM is highly 'stable', in as much as always well in excess of the target.  That, combined with good error rates, may possibly help?

But I've lost count of how the number of 'maybes' in above.  It's just a theory, and maybe I'm just lucky. Maybe (second 'maybe' this paragraph) l should buy a lottery ticket this week, just in case that's the case ?  :)

- 7LM
Title: Re: DLM, beaten again?
Post by: jeffbb on August 17, 2010, 07:46:16 AM
Hi
Good luck with lottery  :lol:

Regards Jeff
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 18, 2010, 12:21:23 AM
@Jeff,

You're obviously unconvinced.

But why not put it to the test and try it for yourself?   From the sound of things, all you have to lose is a few weeks (give it three weeks)  at a low rate, plus the time for the profile to recover if it doesn't work?

- 7LM
Title: Re: DLM, beaten again?
Post by: jeffbb on August 18, 2010, 06:38:54 PM
Hi

Might well do . I am still trying to read up more on how the DLM works . As you say it may look at the current snr margin and compare it to Target SNR margin . However if this were so there would have to be some Very clever programming that allows for "normal"  drop in SNR margin during the connection time  as the  ADSL connection get noisier during certain times of the day. The SNR margin is negotiated when first connecting to the exchange then by definition most times the actual SNR margin will probably be lower than the target  especially during "busy" periods  . Most people will also optimise  their synch rate (regardless of the target )by synching when the line noise is at its lowest.This of course means that the current SNR margin is almost bound to be lower than the Target SNR margin  . That been the case the DLM would never see the line at the target SNR margin setting.

Regards Jeff

Title: Re: DLM, beaten again?
Post by: geep on August 18, 2010, 07:08:38 PM
Hi,
As well as the "intelligence" in the DLM program, I wonder if there may also be some "intelligence" (or stupidity) in the router which is aiding (or fighting) with the DLM.

I have observed that my 2Wire BT27000HGV sometimes retrains (= resync?) in the evening as the SN Margin drops. It starts out happily at 9dB and about 6.8 Mbps sync in the afternoon. Evening comes and SN Margin drops. CRC errors increase so it retrains. Again at 9dB, but now syncing at about 7.5 Mbps. CRC rate is horrendous, but it doesn't seem to care. Then in the afternoon it decides to retrain again as it thinks the noise margin is too low - seemingly at about 6dB - so we're back again at 6.8 Mbps & 9dB and the cycle restarts.

But this cycle isn't consistent. Sometimes it does it for a few days, and other times it never retrains at all over several days.

My ST546 doesn't do this - it just maintains synch, and I watch the noise margin go from 9 to 6 in the evening, and back to 9 in the day. Maybe it's less "intelligent" than the 2Wire, and a tad slower too.

Cheers,
Peter
Title: Re: DLM, beaten again?
Post by: jeffbb on August 18, 2010, 10:33:59 PM
Hi
@Geep
quote : I have observed that my 2Wire BT27000HGV sometimes retrains (= resync?) in the evening as the SN Margin drops. It starts out happily at 9dB and about 6.8 Mbps sync in the afternoon. Evening comes and SN Margin drops. CRC errors increase so it retrains. Again at 9dB, but now syncing at about 7.5 Mbps.

I have not used any routers that have this facility(thank goodness).
The idea seems good so that you maintain a stable connection. It does seem that its not achieving that. You would expect that if it retrains because the SNR margin is too low then resynchronise at target  then the synch rate should drop thereby giving a more stable connection. How it can both increase the SNR margin and synch rate is difficult to say. Must be changing bit loading ,reducing the number Spare bits for bit swapping  .This would probably cause more errors I guess.

Is it not possible to stop this function ? as it seems to be counter productive .

Regards Jeff
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 18, 2010, 11:19:46 PM
@Jeff,

I seem to remember seeing, in one of the BT papers or patents, a statement, vague and all-encompassing, but suggesting that all sorts of errors and stats, as well as just reconnects, are included in DLM's stability assessment.

I've been wondering whether maybe (there, I said it again!) DLM may also include corrected FECs in it's equations?   Corrected FECs don't affect the data, don't need retransmissions, and so have no impact on user experience or perception of stability.  But a high number of FECs may still indicate that corruption is occurring, regardless of whether it's being corrected by FEC.  The authors of the DLM algorithm could have decided high FEC counts should not occur on a 'very stable line', and so take them into account before reducing target.  I've always tended to ignore FECs in my own line stats, but I bet there's a lot fewer FECs with a massive margin than with a normal margin, and that could be another reason DLM apparently responds favourably?

Just a(nother) thought.

- 7LM
Title: Re: DLM, beaten again?
Post by: geep on August 18, 2010, 11:33:42 PM
@Jeff

There's no control over anything with the 2700HGV.
One just looks on in awe via its comprehensive monitoring facilities and wonders "why the hell did it do that?"

Cheers,
Peter
Title: Re: DLM, beaten again?
Post by: jeffbb on August 19, 2010, 10:57:52 PM
Hi
@7LM quote I seem to remember seeing, in one of the BT papers or patents, a statement, vague and all-encompassing, but suggesting that all sorts of errors and stats, as well as just reconnects, are included in DLM's stability assessment.

Yep CRCs ES etc are all part of it . One other important item is the doubler counter . This is the counter that increments by 1 up to a max of 5 when a line has a problem after having a less restrictive Target SNR margin,causing it to be increased again . The max for this counter is 5 which equates to about 14 months before a reduction in the Target will be available

quote from  BT pattern European pattern application
Furthermore, a delay doubler is used to increase the delay
(i.e. by increasing the good threshold) required before a line which has moved down from a more aggressive profile to
a less aggressive profile level is allowed to re-
transition back up to the more aggressive level. The delay doubler is
therefore incremented (in the present embodiment up to a maximum of 5) whenever the line is re-
profiled to a less
aggressive level and then the delays are reset (as in the case where the line is re-
profiled to a more aggressive level)

### each time the lineís profile is transitioned to a less aggressive profile the
DELAY DOUBLER is incremented until after 5 such transitions, each time the DELAY is reset it is reset to a value of
448 (i.e. equivalent to approx 14 months). In the present embodiment, if a userís stability policy or level is changed the
delay doubler is reset back to zero
; furthermore, the delay doubler and even the delay counter may be manually reset
by an operator to cater for exceptional circumstances.

Basically it can take up to 14 months to improve Target SNR margin if there has been some Up and Down resetting of Of the target SNR margin during some previous period .

Regards Jeff
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 21, 2010, 07:03:33 PM
@Jeff,

I'm sceptical about that delay doubler.  According to the patent paper, it only ever gets incremented (up to a maximum of 5), it never ever gets decremented other than by manual intervention.  Whilst it would be possible to implement such a system, it would not be conducive to optimising the network.   It would mean that the first time your line suffers a high margin it might take 14 days to recover, then many years later the same thing happened, it could take 28 days, and so on.  Doesn't add up, to me.

The purpose of the delay doubler seems to be to prevent frequent oscillations between states, by adding an exponentially increased delay.  That's reasonable, if a line keeps yo-yo-ing between, say, 12 and 15dB then it's reasonable to slow down the transition.  But where a line is not oscillating, merely progressing through the states as a result of some line-fault being cleared, I find it hard to see that the doubler would serve any purpose.  For that reason, I'm inclined to think that the description in the patent paper may be incomplete.

Also, as I said when I posted the link to that patent, we don't know for sure whether it describes BT's currently deployed DLMs.

- 7LM
Title: Re: DLM, beaten again?
Post by: Mick on August 22, 2010, 11:35:55 PM
When errors decrease below a certain threshold your router will resync and you should get a better connection rate.  However, said threshold is not the same for all punters.  It depends on your stability setting (there's super stable, stable and standard).  If you ring your ISP and ask them to raise a ticket with BT so that they change your line stability setting from super stable to stable, you will discover that your snrm will drop (say from 12 to 9dB) and although the errors will increase your line will seem to not care about it.

If by reducing from super stable to stable you can hold a connection without continuous resync's then you're sorted.  No need to phaff with router settings to force it to work against the DLM.  The DLM will work for you as opposed to against you.  If you're brave and your connection can take it then wait for a month or so and ask them to drop it again, this time to standard setting.  However, if you're on a long/noisy line you will find that it will become very unstable.  So I suggest that only go one step at a time.

Good luck and let us know what happened with the lottery!  :)
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 22, 2010, 11:55:52 PM
@Mick,

All of that may be true, but my ISP is Demon.   Demon's call centre, since moving to India several years ago, have a vague idea how to talk you through the windows setup menus, or even configure individual routers, if you ask them nicely.  But when it comes to interpreting ADSL line stats, target margins and DLM queries, you may as well ask the cat.

But TBH, for me, this is largely just an exercise in working out how DLM operates, and maybe how to beat it.  

Stability options are described in that BT patent paper and scare me a little, as I really don't imagine many of the Indian call-centres (not just Demon) are going to have the slightest idea what to do with them.

- 7LM
Title: Re: DLM, beaten again?
Post by: waltergmw on August 23, 2010, 07:30:19 AM
@ 7LM,

We know of a much better cat here to consult !

I agree entirely re Asian call centres. I'm dealing with a disconnection for a monthwith 6 engineer site visits and 5 modems to fix an authentication problem which vanished by magic over the week end when the real people here were persuaded to pay attention. ( I did happen to say in high places "Crass stupidity" though !)

Kind regards,
Walter
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 23, 2010, 09:29:43 AM
Yes, I suspect it's a common problem with call foreign centres.  The worst I ever had was with a car insurance firm, but I'll resist telling you all about it.   Suffice it to say there could be a secure future for any polite and well-behaved cats, especially those with any knowledge at all of ADSL or car insurance.

Anyway, I've been running for a few days now with target increased from 12 to 18dB.  I'm not hugely optimistic as, even at 18dB, I'm getting typically 150 ES per day and twice that many CRCs.   That's only 6 ES an hour, which isn't much, but my hunch is that won't be good enough for another target reduction.  I'll let you know if it happens, but don't hold your breath.  

At least the Netgear seems to hang onto the connection.  It hasn't dropped once in the last 4 weeks, other than by my intervention or by DLM when the target dropped to 12.    In contrast the Thomson, which was in use when the target got raised, was crashing(i.e. rebooting) & reconnecting up to several times a day.  Admittedly, the Thomson didn't have my current massive SNR margin, so it would have been more vulnerable to noise storms that may have been making it crash, but I'm still working on the theory that the underlying problem, that caused DLM to raise my target, was probably a duff router more than anything else.

- 7LM
Title: Re: DLM, beaten again?
Post by: Mick on August 23, 2010, 02:06:15 PM
There is a solution to such problems:  move ISP!

There's no reason to hesitate for a minute voting with your feet, if the service you are getting is not to your satisfaction.  Businesses with call centres in India are targeting mass market customers who rarely if ever ring up support, only to be told to go away and reboot their MSWindows OS.

That said I managed to get very good level of support from a BT broadband support centre, but only after I insisted that they do not know how to resolve my problem and they should pass me over to the second line support.  I spoke to some guy in Ireland and got impeccable service and excellent technical knowledge out of him straight away!

These days switching over to an ISP with UK support, who do not throttle traffic, is quite easy.  If you continue paying money to companies who chose to outsource their relationship management with you to script reading Indians, then you will continue getting instructions to reboot your PC.  The worst thing is that such companies will continue doing just that!  Making money by reducing the value they offer to their customers.

Anyway, have you tried to elevate the request to an "engineer" and see if you can get any joy out of that?  Really, all you're asking is for demon to raise a call with BT on your line, to change your stability setting from super stable to stable.  You will still get resync's after the change, but it would take 2 to 3 times higher FECs to trigger a resync.  Meanwhile, your snrm will reduce and stay there without you needing to fight against the DLM logic.
Title: Re: DLM, beaten again?
Post by: HPsauce on August 23, 2010, 02:29:47 PM
That said I managed to get very good level of support from a BT broadband support centre, but only after I insisted that they do not know how to resolve my problem and they should pass me over to the second line support.  I spoke to some guy in Ireland and got impeccable service and excellent technical knowledge out of him straight away!
This is how my friend (whose travails I asked about here recently in another thread) finally got BT moving. Multiple complaints and eventually escalating to someone sensible, who also sounded Irish. Took him nearly a month though.... :'(
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 23, 2010, 04:06:57 PM
@Mick,

Thanks for the advice, but as I tried to say, this is really just an experiment.  I have no great need for a lower margin, or for a higher speed, I'm just trying to get some satisfaction from tricking DLM into doing so.  I wouldn't get the same satisfaction from asking an ISP to press a button.   >:D

For the record, moving ISP is not always an easy option.   We run a very popular website which in its infancy had a ......demon.co.uk domain name.  Since about 2002 it's had a .com domain, but the old demon web page still exists and just redirects to the .com.  Last time I looked, there were still hundreds if not thousands of links to the old demon address, and if it were taken off the air (by closing my demon account) these links would break.

The same applies to email addresses.  Despite setting up mail redirects using the .com addresses, and notifying everybody years ago and repeatedly, we still get a surprising amount of mail addressed to .demon.co.uk.  Some of it is mail that I'd not want to lose, even though it may be people I've not heard from in years.
 
One final point..., the only way to get a query escalated to a knowledgeable engineer at demon seems to be to submit a formal complaint, in writing, to the parent company.  That did the trick last time around, when they were owned by 'Thus', but the whole process took about three weeks from initial email to satisfactory outcome, and life's just too short to face up to that hassle every time you need something done. 

Thanks anyway for the advice.   :)

- 7LM
Title: Re: DLM, beaten again?
Post by: jeffbb on August 24, 2010, 10:30:55 PM
Hi
@7lm
quote Also, as I said when I posted the link to that patent, we don't know for sure whether it describes BT's currently deployed DLMs.

quote [0071] In the present embodiment, each line is categorised by the first sub-
function of the DLM function

0075] In the present embodiment, each line is processed once every 24 hours to determine how the line should be
categorised,

Throughout  the document the above phrase is used !.

From what I understand the pattern is actually to include the  ability to disregard Wide area events from being included . Much of the document refers to present embodiment.
There are a few other changes ,but the Delay counters etc seem to be as is now .

Regards Jeff
Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on August 25, 2010, 10:59:02 AM
Much of the document refers to present embodiment.

The word 'embodiment' is just a legal term commonly used in the legal jargon of patent claims.   I'm no lawyer, but as fas as I know 'embodiment' just translates to a particular detail that is an exemple/subset of the invention, it needn't mean it's actually been physically deployed on a large scale, or even on a small, scale.   See http://en.wikipedia.org/wiki/Claim_(patent)

Also, FYI, see also another DLM patent from BT, that mentions nothing about delay Doublers... http://www.freepatentsonline.com/EP1953959.html

but the Delay counters etc seem to be as is now .

My own experience directly contradicts the doublers.  My target went all the way up to 15dB last year, then eventually I got it back down, 3dB at a time, to 6dB again.  Then recently it crept up again through 9, 12 and 15dB.   If the doubler were implemented, that would have left me with a doubler value of at least 3 (depending how you interpret it), so it would take at 16 weeks to for my target to reduce again from its recent hike.  In fact, it reduced from 15 to 12 in just over 3 weeks.

I'd dearly like to say 'There, I told you so' next week, when my target gets reduced again to 9dB.  Sadly I don't think that'll happen as, even with a hugely generous SNR margin, I'm just not sure my error rates are  good enough at the moment.     :(

- 7LM








Title: Re: DLM, beaten again?
Post by: sevenlayermuddle on September 02, 2010, 01:32:10 AM
I'd dearly like to say 'There, I told you so' next week

Well, guess what, 9dB (reduced from 12) target now, exactly 14 days after cranking up my target margin once again.  Unfortunately,  I can't actually say ' I told you so', as I qualified the above with excuses lest it didn't happen, but all the same -  I'm feeling enthused.

:thumbs:

Error rates this past few days have been about 25 ES, 40 CRC per day, which is pretty good.  Over the past two weeks however, since I cranked up my margin again, it's been 1935 ES in total, so an overall average about 140 per day or 6 per hour.  So the signs are that miniscule error counts are not necessarily a prerequisite, whilst there's still more evidence to the theory that masses of spare margin does seem to help.





 
Title: Re: DLM, beaten again?
Post by: DrTeeth on September 05, 2010, 01:26:06 PM
This just illustrates why LLU is so great.

DrT