I totally understand about surroundings, people and the different energies. Some folks, I guess, bring you down but not on purpose. They're going through their own issues but you notice that you cringe when they call or come around. You love them to pieces but it's always negative.
I've only experienced negative or frightening energy in this house once. It was about 8 months ago. I was actually afraid to close my eyes or sleep with my back towards the door. I kept seeing a dark shadowy image. It hasn't happened since. Weird.