Completely missed this thread. Too old to be hitting the tiles so catching up.
Modulating and demodulating the DSL isn't immediate.
The fibre route will be circuitous - networks are not optimised for latency unless manually tuned or built for it.
The intermediate devices will add latency of varying degrees depending on the processing and configuration they are doing.
Etc.
The lowest latency links are not over fibre. The microwave dishes on top of financial houses are there for good reason