Some resorts and properties do have better beaches than others, but sea grass is necessary. Ramon's Village may have the kind of beach you are after. The bigger problem right now is the sargasso. We've been coming down staying in a condo south of town for 10 years now. I usually go swimming off the dock since the property has a seawall, however this year was the first year even at the end of the dock there was a lot of the seaweed. We went snorkeling outside the reef and were even running into sargasso out there.
I would not recommend taking 5 and 7 years old out to Secret Beach. It's kinda become a party atmosphere out there, not really geared for kids in my opinion. Find a nice quiet place either north or south of town.