One of my friends suggested this might be mildew which apparently is common on indoor rosemary. I cut off the affected pieces. Then I made up a mixture of baking soda, mild dish detergent and water. I sprayed it on all the leaves for several days. The leaves now look a little powdery from this spraying, so I'll have to wait and see if it did help with the mildew!
