I was thinking about a crude spike detector that looks at x==60 for a dip down to y=2 (or maybe 3,4 for a bit of insurance) and then if it sees one at x==60 it walks sideways in increasing x+=2 to get away from the spike. Then y[x==60 or 62] now is compared against y[40] and y[85] as mentioned in my first post. Something like
hollow = y[x==60 or 62] < y[40] && y[x==60 or 62] < y[85];
A question: when writing posts in the forums, it does of course go crackers when I try to write certain things inside [ ]. How do I work around that?
Here’s an example. Very very serious disease. What I also want is very mild early onset so the algorithm can detect the onset really early and warn in good time. The very serious disease thing would be detected by the min-sync rate test in any case.
<snip>
And here are the latest perfectly healthy pictures:
<snip>
I wanted to draw a straight line between the two humps and show on it the variables discussed, to make a pretty picture. Especially if I can find a picture where such a straight line would be downwards sloping to the right. (Increasing x, decreasing y). However I don’t have a tool for that. And would get lost immediately amongst the thousands of iOS apps. I would need something that can draw on an existing bitmap, draw straight lines in various line styles, drop various symbols such as dots, blobs, crosses and draw text too. And with multiple undo too.
Pilot tone shows bitloading of 2 always.
In my code as it is now, implementing algorithm one (of the two described above), I test y at 3 x values: x==40, x==60 or 62 and x==85. The x==60 thing is the approximate mid point, but there is a danger of hitting the pilot tone there, so if x < 4 (I see you have 3 there, so I should perhaps amend my code) then I assume I have hit a pilot tone and so I switch to sample y at x==62 instead. I hopped out of the way sideways by x+=2 because of nervousness about the width of the pilot tone but I was overly paranoid in this respect as you have told me, so x+=1 would have suffixed, but there’s no harm in x+=2 so I’ll leave it as is in case I have hit something other than a pilot tone, possible if the bitloadings are extremely low in a very very diseased system. (See one of my earlier pictures (https://forum.kitz.co.uk/index.php/topic,26716.msg447930.html#msg447930).)
So you haven’t seen a bitloading=2 data point deep dip in your ADSL days?
Until such time as you have the raw data, which produces such a plot (from a blighted line), it is difficult to suggest suitable code. Perhaps put together a few examples and wait for the "hollow curve phenomenon" to occur; then test the code.
I have once again altered the core algorithm, now using algorithm II, combined with general sharp dip detection (not just specific to the pilot tone) at the ‘mid’ test point. The x==40 test point has a specific test for being a pilot tone.
y40 = ( y[40] <= 2 ) ? y[41] : y[40];
y60_is_bogus = (y[60] < y[60 + 2] - 2) && ( y[60] < y[60 - 2] - 2);
y6x = y60_is_bogus ? y[62] : y[60];
ymid = y40 - 1 + (y85 - y40)*(60 - 40.0)/(85.0 - 40.0);
hollow_curve = ( y6x < ymid );
asked myself "How do I know it shows the ailment?" The answer is that we visually take in the shape of the bulk image and mentally compare it to what we have seen in the past. No precise mathematical transformations are required…
I am using that philosophy and I reckon that the answer is in the core of the new Algorithm II - the droop below a straight line as defined by the straight line equation, cutting a chord across the top of a good distribution, but in the case of a ‘bad’ distribution, the ‘midpoint’ (approx) is hanging down below the straight line, a ‘sag’. And the vertical distance of that droop is measured in the code above.
When I was a small boy, I visited a farm across the valley, ‘Thorn Top’, and sat in the kitchen. While the adults were talking, I was miles away, as ever, staring at a lead pipe on the kitchen wall. The pipe had support clips every so far along it, but in one place the lead had flowed and sagged into a deep dip, miles below the horizontal straight line. I thought this thing was strange and wondrous.
God only knows what all that lead did to them. Doesn’t lead form an insulating barrier compound ?an oxide? which prevents further dissolving into the water?
Referring to the line
y6x = y60_is_bogus ? y[62] : y[60];
taking the "? :" expression apart and turning it into an if-statement, we are not fully examining the conditions surrounding our new sideways-x-shifted choice of x=62; is there any problem with that? laxity?
xdslctl: ADSL driver and PHY status
Status: Showtime
Last Retrain Reason: 8000
Last initialization procedure status: 0
Max: Upstream rate = 321 Kbps, Downstream rate = 3304 Kbps
Bearer: 0, Upstream rate = 396 Kbps, Downstream rate = 3154 Kbps
Tone number Bit Allocation
0 0
1 0
2 0
3 0
4 0
5 0
6 0
7 8
8 9
9 9
10 10
11 9
12 9
13 9
14 8
15 7
16 7
17 7
18 6
19 6
20 5
21 5
22 5
23 4
24 4
25 4
26 4
27 4
28 3
29 3
30 3
31 3
32 0
33 8
34 9
35 10
36 11
37 11
38 11
39 12
40 12
41 12
42 12
43 12
44 12
45 12
46 7
47 12
48 2
49 12
50 12
51 11
52 11
53 11
54 11
55 11
56 11
57 11
58 10
59 10
60 11
61 11
62 12
63 12
64 12
65 12
66 12
67 11
68 12
69 11
70 11
71 11
72 11
73 11
74 11
75 11
76 11
77 11
78 11
79 10
80 10
81 10
82 10
83 10
84 10
85 10
86 10
87 9
88 9
89 9
90 9
91 9
92 9
93 9
94 9
95 9
96 8
97 8
98 8
99 8
100 8
101 8
102 8
103 7
104 7
105 7
106 7
107 7
108 7
109 7
110 7
111 7
112 6
113 6
114 6
115 6
116 6
117 6
118 6
119 5
120 4
121 3
122 2
123 4
124 4
125 3
126 4
127 3
128 4
129 3
130 3
131 4
132 4
133 3
134 3
135 3
136 2
137 3
138 3
139 3
140 3
141 3
142 2
143 3
144 3
145 3
146 2
147 2
148 0
149 1
150 0
151 1
152 0
153 1
154 1
155 0
156 0
157 0
158 0
159 0
160 0
161 0
162 0
163 0
164 0
165 0
166 0
167 0
168 0
169 0
170 0
171 0
172 0
173 0
174 0
175 0
176 0
177 0
178 0
179 0
180 0
181 0
182 0
183 0
184 0
185 0
186 0
187 0
188 0
189 0
190 0
Upstream Pilot Tone for ADSL (Tone 16 - 69kHz)
Downstream Pilot Tone for ADSL (Tone 64 - 276kHz).
Adaptive Pilot Carriers. VDSL & some adsl2+ systems select best channel conditions to determine use for pilot tones (eg 105,110 etc). Pilot carriers may change if either the DSLAM or the modem detects noise within the frequency range. The amount of pilot carriers used depends upon the line characteristics.
Upstream Pilot Tone for ADSL (Tone 16 - 69kHz)applicable only to ADSL1 ie G.992.1 ? That is, not to all ADSL?
Downstream Pilot Tone for ADSL (Tone 64 - 276kHz).
Questions:
- I can’t see an upstream pilot tone; am I misunderstanding?
- My downstream pilot tone seems to vary in frequency. The article mentions VDSL and ADSL2+, but I’m ADSL2 G.992.3 not G.992.5, so should G.992.3 be including in the part about adaptive pilot carriers ?
- Is this part:
QuoteUpstream Pilot Tone for ADSL (Tone 16 - 69kHz)applicable only to ADSL1 ie G.992.1 ? That is, not to all ADSL?
Downstream Pilot Tone for ADSL (Tone 64 - 276kHz).
Sorry, I didn’t explain. For question 3, I was asking about the tone numbers.
I’ll check with Janet but I’m pretty sure that the modems were on at all times until the power cycling experiment.
By the way, notice the amazing downstream sync rate of line 4.
Modems’ sync rates:
#1: downstream 2.964 Mbps, upstream 666 kbps
#2: downstream 3.081 Mbps, upstream 531 kbps
#4: downstream 3.253 Mbps, upstream 402 kbps
That is indeed an all time record for Torr Gorm ("mound green" = this site).
It’s the name of the house. I’m absolutely sure that A&A have a server called gormless . . .
Is the real pilot tone exactly 1 wide only and no more ?
. . .
It does seem that the pilot tone can be anywhere and I made far too many assumptions before and got this wrong not once but twice! Twice now I’ve had this blind spot about the possibility of embarrassing placement of the pilot tone. I just looked at the picture I had and assumed that everything was always exactly like that. Can the pilot tone be anywhere?
I could do with some advice about what to do with rounding and float-to-int conversion in the above expression. Should I do a certain type of explicit rounding and fp to int conversion?
So iOS Shortcuts gets us out of the problem, but is that what I should be doing?
Makes me think, if I process the SNR data, do I even need the bitloading "bits" data at all? What do we think?
The pilot tone isn’t a consideration in the SNR graph, so it actually simplifies the code if I just change over to use the smooth curve SNR dataset! If I go that way, it would massively simplify the algorithm at https://forum.kitz.co.uk/index.php/topic,26716.msg449098.html#msg449098 down to just one line of code approx as all the if-else tests before the final boolean assignment would be gone completely.
Makes me think I made a huge error in using the wrong dataset to begin with, no?
What’s your feeling about the SNR-only option?
I did a trial implementation of this which I will call algorithm III (forthcoming v shortly), and my first question is about the minimum noise stalactite height which is a "- 2" in "<y> - 2", would need to decide what the SNR curve equivalent height is. The result with a bad guess at the answer to this question worked really well on the current ‘fully-developed HCD’ dataset.
Latest picture of line 4. Has this changed? This has to be an early HCD, but the test point at x==60 is to far to the right to pick it up. Can change that, of course. Or have two such test points. I’m wondering if this is just a slightly different type of early HCD shape to the ones that I’ve seen before?
* The new algorithm works fine ;D, now detects HCD for modem 4 (only), which of course is the correct answer.
39 39.2500
40 40.0625
41 40.6250
42 40.8125
43 41.0625
44 41.2500
45 41.5000
46 35.3125
47 41.1875
48 41.5625
49 41.5000
50 38.6250
51 41.6250
52 41.1250
53 41.4375
54 41.3125
55 41.0000
56 40.8750
57 40.8125
58 39.0625
59 39.0000
60 39.9375
61 40.0000
62 40.2500
63 40.0625
64 39.4375
65 39.5000
66 39.0625
Final summary and up-to-date algorithm.
As mentioned earlier, I added an additional parameter ‘chord2_notch_min_height’ set to 1.5 dB.
<snip>
This has worked perfectly, blocking the detection of small chord2 notches if they are regarded as not genuine ‘disease’, although this classification is questionable. I would say that ‘notches’ are only bogus if they do not go on to develop into something larger.
Despite the ‘hollow’ being only slight, it was detected to the rhs of the centre of the region.
Could someone post an SNR vs tones graph for a VDSL2 circuit?
This is the problem picture.
/* Algorithm v 5.2 */
uint x4 = find the x value for which y[ x ] = max( [ y[40], y[43], y45] ); /* test three points just above tone 40, find the highest y */
double y4 = y[ x4 ];
This fix will take us to algorithm - what ? - v5.3 or v6 ? Which do you think it ought to be ?
So very big improvement.