We have been there a couple times during Carnival. The parade is fun. Getting painted and egged is also not my idea of fun, so we stayed away from the kids and the beach right in town. A couple of times some kids acted like they might throw paint my way, I just nodded my head no and smiled and they went on their way. With a little thought you should be safe 2mile. You could also stay 2 miles north for a couple days.
Life is not about waiting for the storm to pass, it's about learning to dance in the rain.