Well, yes, this is to keep using bridge mode with what I assume is ZyXEL(gateway)>Firebrick
To answer the original question:
Does QoS work even if your modem-router is set to just be a modem? Or does it only apply to routers being routers?
In bridge mode with what appeared to be simple criterias: no more than two devices bridged as Modem ---> Router - the answer was no for both devices.
Without bridge mode, the answer is yes. If this thread was only about non-bridge, then I've posted all that off-topic
Edit: My mistake, a router can do QoS in bridge mode [just the gateway aka modem can't only in bridge], and if the Firebrick doesn't have the QoS you need but you want to keep it, then you can add a switch. On the ZyXEL I put it by packet length on upstream which overrides all rules so I didn't set any.