The tap water is safe to drink. However, I only use it to brush my teeth because it tastes funny. You can buy bottles of water in all sizes at the grocery stores, etc. Your hotel will probably also supply you with some, at least to get you started.
And yes, rum is always good too, though I can't remember ever brushing my teeth with it.