I always wear shorts and a bikini top around town/beach. It's sort of my staple apparel! Middle Street can get pretty toasty if you're not used to the heat (yes, this coming from a Texan!) but I always throw a t or tank on when going into a business establishment.
You'll love San Pedro!