It's clearly a bug in the site - the site is "HTML-escaping" the text twice. This is better than not escaping at all, which intoduces what are known as "cross-site scripting" (XSS) bugs, but is a bug nonetheless.
Put briefly, in a HTML page, certain characters (<, >, &, ', " and others) have a special meaning. If you want your HTML page to actually display one of those characters, you need to "escape" it, so the browser knows that it is a character to be displayed, and not part of the markup.
This escaping takes the form of "&something;". An & itself is escaped as "&", and as you've seen, a single quote (or apostrophe) is escaped as "'" which the browser will understand and display as an apostrophe.
However, if "'" is then escaped again, the HTML will contain "&#039;", which the browser will display as "'", and not as a single quote/apostrophe.