Chat > Chit Chat
Maths - hollow curve phenomenon detector - algorithm design
Weaver:
There’s a bug in that expression. Because of quantisation noise, the value of y can go up/down by 1 in a semi-random fashion. For this reason I’ve adapted the expression to give:
hollow = y[x==60 or 62] < y[40] - 1 && y[x==60 or 62] < y[85] - 1;
This is I hope correct? The y values can wobble up and down by 1 so y1 < y2 should really read y1 < y2 -1 - that’s my reasoning. Please someone sanity check me on this.
I’ve written the Shortcuts code for this. It has absolutely zero loops in the Shortcuts code[!], letting the engine runtime do the work at low level that I would otherwise have to write in lines of Shortcuts source code. This means that it’s not slow. It gets the bitloading data file from each modem as http://modem:8000/data/Bits/ and uses regexes like ^\X*\n[ \t]*40[ \t]+(\d+)\X*$ to get y[40] for example, searching the whole of the text for a line like
40 11
which means tone=40 bitloading=11 bits. Same for tone 60 or thereabouts and for tone 85.
I need to look back at some of the old bitloading hollow curve illness reports (not the SNR reports).
I need to test what happens if there are some small dips due to interference, downwards-going just a few bits deep and say 2-3 tones wide. My worry is that such could be mistaken for hollow curve. A more sophisticated test, which is capable of moving the y[x60] point sideways more, that would presumably do the trick but the details escape me just now.
A question: when writing posts in the forums, it does of course go crackers when I try to write certain things inside [ ]. How do I work around that?
A question for all ADSL experts : is the bitloading[pilot_tone] == 2 thing something that is always exactly true? I didn’t know whether the value can be zero or > 2 or what. Because I didn’t know, for now I made the pilot tone test "(bitloading y < 4)".
Weaver:
Here’s an example. Very very serious disease. What I also want is very mild early onset so the algorithm can detect the onset really early and warn in good time. The very serious disease thing would be detected by the min-sync rate test in any case.
And here are the latest perfectly healthy pictures:
Weaver:
A further thought. Using any kind of averaging, smoothing type filter would surely make the depth of the pilot tone dip much higher, would it not? Thus increasing the danger that a pilot tone in the wrong place could be mistaken for a hollow curve y-min if the smoothing filter was applied first, before all other inspection, and therefore affecting all data pilot tone dip included.
Despite this earlier post of mine which I had completely forgotten, I still don’t understand the data in pilot tone thing.
ALGORITHM II
Another idea about the central comparison in the algorithm. Take the two points ( 40, y40-1 ), ( 85, y85-1 ). Then consider the data point ( 60 or 62, y6x) where y6x is the corresponding data y value. Now draw a straight line between the first two points. Then where it reaches x=60 or 62, solve for ymid = y; that is the y value of the straight line at the middle point x== x60 = (60 or 62). Now we have
hollow_curve = ( y6x < ymid ); ymid = y40 - 1 + (y85 - y40)*(x60 - 40.0)/(85.0 - 40.0) — is that correct ?
I’m not sure if this is an improvement or not. It certainly handles one potential problem case, which is where the y85 value is really low, because the whole thing is very peaky/pointy, not like a nice rounded / domed loaf kind of shape. If the y85 value is really low then the second of the tests in the && in the first algorithm will be in danger and the expression will fail to spot very early mild hollow curve onset.
jelv:
--- Quote from: Weaver on January 21, 2022, 12:24:29 AM ---A question: when writing posts in the forums, it does of course go crackers when I try to write certain things inside [ ]. How do I work around that?
--- End quote ---
Enclose the text in [nobbc]...[/nobbc] tags (which I had to do in this post so they would show - start a quote of this post to see how).
burakkucat:
--- Quote from: Weaver on January 21, 2022, 01:18:26 AM ---Here’s an example. Very very serious disease. What I also want is very mild early onset so the algorithm can detect the onset really early and warn in good time. The very serious disease thing would be detected by the min-sync rate test in any case.
<snip>
And here are the latest perfectly healthy pictures:
<snip>
--- End quote ---
If it would not be too inconvenient, would you please send me (via e-mail) the "xdslctl info --Bits" data for those four plots? I would like to generate those plots with my usual utility, so I can see how different the plots look when compared with the "Easy Stats (tm)" output that you have shown, above.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version