I disagree with some above. There should be safe swimming within docks and further out. From your description of where you were and what he did I'd consider reporting him to the Coastguard. No excuse for a change of course, on the plane, flying right at you, unless he was heading off another boat that hadn't seen you.
Boats should be way out - hundred plus yards from Dock ends and then come inshore off plane - no excuse or exception other than emergencies. It should be enforced, other than areas where they have to come closer to shore due to shallows/reef etc but thats almost exclusively further North with maybe a couple of exceptions. There's room for boats and swimmers ín almost all areas with a couple of exceptions perhaps.
We are a tourist destination and people should be able to safely swim in waist depth areas in spots like described above. Otherwise its lazy and cutting corners or trying to save a shilling here or there.