We have well water, and even so we have a Brita water filter that we use for our drinking water. We also have a fridge with a water and ice dispenser, but to be honest it takes SO long for the water to dribble out of the fridge that it's just easier to pour it out of the Brita.
Also for some strange reason I don't like cold water, I think it hurts my teeth. So I like room temperature water ...
