Dear Julieanne -
Welcome to our forums!
Definitely it is challenging sometimes to find that right mix so you are a healthy weight, not hungry, and not sluggy. It's great that you're working on it!
We all have different metabolisms and activity levels so a key seems to be to weight yourself at least once a week. That way you're aware of trends when they are beginning, and can adjust for them before they go too far.
It sounds like you want to lose a little weight, but not go overboard. Which is great! I think your changes like avoiding the potatoes will help. Do that for a week and see if the needle starts to slide down a bit. Your aim is for slow, gradual loss, so you can taper off as you approach your target weight.
Let us know how you do!