I'd predict that the degree of instability would vary, depending on the services you use.
As the margin reduces, the error rate will increase. So far as I recall, 0dB margin is simply the margin at which the BER exceeds a standardised threshold, 1 in 10^7 I think(?). As the BER increases, some non-recoverable errors will take place, leading to data retransmission.
For many services, these retransmissions go unnoticed. For time-critical stuff like video streaming, it might lead to freezes. For true IPTV, which does not benefit from TCP retransmissions at all, the lost data would simply lead to picture corruption.
Above is written on the assumption that TCP handles the re-tx. If using Physical retx, things are not as bad.