In the low season Cap't Morgans is not expensive at all ($159/nt). Does go up to $249 in high season but that's no more than a lot of the high end places.
Any of the above mentioned places would be good for a honeymoon. Personally I recommend Victoria House as well if it's in your price range. Corona del Mar would be the lower priced end of the places you mentioned but is nice as well, and has the "open bar" advantage. There are many, many options - depends on what type atmosphere and amenities you are looking for.