@Wildy,
Did you ever get Sky or BT to take any more interest?
I see your attenuation now shows 47dB, whereas it used to be 55dB, is that correct and consistent? I guess it's theoretically possible that your new line could take a much more direct route to the exchange, but it doesn't seem likely. Normally, line attenuation shouldn't change much, except that different routers can report different figures (a couple of dB or so either way). Large changes in attenuation can suggest an intermittent line fault, so it's possible your old line had a fault, and your new one doesn't, which would explain you speed gain.
Regardless of why the attenuation has improved, I still think you're right, 2016 is disappointing even for a 55dB (as it used to be) line. For a 47dB line it's downright pathetic - the kitz calculator predicts over 5500. The 8.6dB noise margin is high, but nowhere near enough to explain that discrepancy.