If kayaking is your thing, my recommendation is to consider www.slickrock.com
Swimming out to the reef (or kayaking) from anywhere in or even near San Pedro would be dangerous. The dive boats go very very fast
and kayaks are low in the water.
I don't know if there is a resort open right near Mexico Rocks, if so that would be a good place to be close to a good snorkeling reef. The last time I was out there, there was nothing happehing, but maybe something new has been built?