Public baths are an ancient european invention, traceable at least as far as the Romans. I can't think of any sources indicating that the Greeks had public baths, but then I'm not so very well versed on Greek material culture—please, correct me if I'm wrong on that point. Either way, the tradition of the public bath was something deeply entrenched; hot water was, of course, part of the ritual. In ancient days it was warmed from beneath (as it is in the cities of Arunian elves and the great urban centers of the Third Empire were the infrastructure has survived) but by the Middle Ages we're looking at a much more primitive style of bathing.
From heated marble pools, we transition into a world of baths (both private and public) which are filled with water heated in jugs or kettles and then poured into wooden tubs.
But what about the public baths? From what I understand they were shacks or shanties erected near a water source where similar half-barrel tub constructions were rented out for a few sheckles. This wasn't a regular daily activity, but the health benefits of bathing were fairly well known. Not as potent, perhaps, as visiting a relic or shrine, but if you lived in London or Paris infinitely less costly than going on a pilgrimage to Santiago de Compostela.
What's my point? In any sufficiently developed medieval urban center, public baths are likely to spring up. The same goes for the fantasy coaching house that we've developed into the all-inclusive Inn in most fantasy. It is likely that such a place (assuming the trade on the road was great enough to even support it, but that's another entry I think) would keep at least a few tubs for bathing. Hell, even if they didn't they would have to keep one for washing that you could use unless they were close enough to a river.
Indeed, let us not discount bathing in rivers, which has Biblical connotations.