Ok I have a little update.
I made some adjustments on the L2TP client side, MTU and MRU set to 1460, AAISP left at auto as oddly they dont have a 1460 option for MTU.
I decreased the ping interval on dpinger, and there is now no measurement of increased latency on tbb, pings from any network device, and pfsense itself are also normal, its now only dpinger been odd, and is threads on reddit about dpinger weirdness with bridge mode modems (higher latency vs normal pings), so this seems fine now and just a dpinger oddity.
The SLAAC issue I think is due to a pfsense bug, I have the same issue on some servers in datacentres, the problem seems to be triggered when the gateway is not pingable, when it isnt pingable (which seems common on ipv6) it will stay in pending state causing the routing to never get activated, choosing another IP to ping doesnt resolve it as I think pfsense expects the gateway to always be pingable, the workaround is to disable the monitoring at the loss of having no monitoring data or auto behaviour that relies on the monitoring. I am going to post on redmine bug tracker about it.
There is an existing bug report (fixed in 2.7.0) which was really interesting I read last week which had another similar issue where the debugger posted the process of scripts that are run to explain how he fixed it, which was really nice info but I have ended up losing the link. So will need to check my browser history on that one.