From San Pedro Town there are no reefs or patch reefs close enough to swim to. If you're up at Robles Point (which requires a boat to get to, then you can easily swim out to the reef, with is just a few hundred feet offshore of the beach). In town, swimming out to the reef is very dangerous because it's kinda far, and you're in the boat lanes all the way.