My friend and I sent the same request to this board about two months ago -- thought the water taxi would be an adventurous way to get to the island, and of course it seems much cheaper. In the end, we flew over, and honestly, I'm glad we did!! For 4 people, it would probably be quite a bit cheaper to take the water taxi, since you can all split the US$20 fare from the airport to the boat dock, but the hassle of transporting your luggage, and the limited hours the boats depart almost make it more of a headache to coordinate than simply jumping onto another plane (which leaves every hour) and settling into your hotel before you would even be getting on the boat in Belize City -- the boat ride over takes 70-80 minutes, whereas the flight is 15-20 min!
Also, a very good point made through a response when I asked this question was that if you are planning to do any adventures from AC that bring you back over to the maniland (ruins, zoo, cave tubing, etc.) or diving some of the farther spots, you will be making a long boat ride then, which is an enjoyable experience to have once! Any more than that and you may start cursing the wind and waves (they aren't exactly luxury liners that you're riding on
I know you mentioned that comfort isn't a high priority, but if convenience is, I would suggest the flight