I don't think it is to do with the "quick reply" (whose appearance is indeed controlled through a profile setting).
When you hit "reply, and get the text pane to fill in, it exists in one of two modes: one WYSIWYG, one displaying the formatting markup. The mode is controlled by a button with a red capital-A, and hovering over it displays "Toggle View".
In one mode, pressing the "Insert Hyperlink" causes the [url] markup to be added. In the other mode, a popup appears.
Edit: The reason that "quick reply" works is because the "preview" option of "quick reply" seems to drop you into WYSIWYG mode, even if you don't normally start in that mode on a standard reply.