Katie - where's Shark's Cave, and what's it like? People might know of the cave that exits at Mexico Rocks - I've been into that 400-500 feet and there were clear signs that others had been there before me, but not in a long while. It was very silty, even that far in, and though it had been shallow (less than 50') it was about to get much deeper and I didn't have enough air. I know someone who dived the cave just behind the cut on Caye Caulker 25 years ago with Sheck Exley, and he said it was an excellent dive. Of course people have subsequently been lost inside there.
The safest rule for anyone not cave trained contemplating going into one is "don't". But there has to be a first time. Just make sure you get properly cavern trained before you go anywhere near a cave. In a cavern you can always see some daylight so in theory it's safer, but even there you can kick up sediment and obscure the way out. The same's true inside most wrecks, and they need to be treated with at least as much respect as caves.
A fundamental part of cavern/cave/wreck penetration training is developing superb buoyancy control and a range of suitable finning styles - people can work on this in open water on ordinary dives. Try swimming very close to a sandy bottom yet not kicking up any sand or silt. It's possible, and no instructor will take you into a cave until you have mastered it.