jamie, are you planning on renting a golf cart? It's necessary if you plan on going to town a lot. As you mentioned, the upside to staying there (and I myself have not stayed there) is that they have a really nice pool and access to a fairly new rest./bar called the Sandbar or Fido's 2. It's pretty much just right outside your door! Also, there WAS a water taxi running from the Sandbar to town, the dock at the original Fido's. Not sure how often it runs. Are you planning on renting a goft cart during your stay?
Also, as you mentioned, you'd be fairly close to Victoria House! That's a nice place!