One of the main attractions of the islands is taking advantage of the very frequent ferries to all the other islands and exploring (starting with Tresco), so if mobility is an issue, I'm hesitant to recommend. I'm not sure how much there is on St Mary's alone - we did a day trip (silly o'clock flight out, boat back) a few years back when staying in St Ives specifically to get to Tresco and also explored Bryher - but we do a lot of walking anyway.