I first went to Costa Rica in 1982 when a beer at a top hotel was 15 cents, and a meal in a top restaurant was $2 or $3, but in recent years, Costa Rica has become fairly expensive. Certainly at the top end hotels in Costa Rica are as expensive or more so than similar accommodations in Belize.
Costa Rica does have more budget-level accommodations than Belize, but in most areas of Belize you can find good, safe accommodations for under US$25 a night. (I just did the update for Belize for UpClose Central America guide, formerly Berkeley Guide, so I know there are plenty of decent budget places.)
Your best bets for inexpensive lodging are on Caye Caulker (where lots of places are under US$25), Placencia Village on the mainland in Stann Creek district, Tobacco Caye, and also San Ignacio in Cayo District, but there are affordable places in every town and area. Ambergris Caye has the fewest budget places, and some of these have recently closed or have been upgraded, but there are still four or five.
In August, which is low season, you should have no problems with accommodations without reservations.
Belize First Magazine http://www.turq.com/belizefirst/