So, the interleaving got removed on the upstream about an hour ago and the additional delay is just 8ms now.
hopefully it won't be completely crap again, but I won't hold my breath. My ISP said it would remain active, but it still seems to be controlled dynamically by Openreach.
EDIT - Yup, just started a Stadia stream and it's losing thousands of packets again in just 4-5 minutes, and the stream feels glitchy again. So, it seems to need the upstream interleaving to be enabled... Very specific problem.