I can certainly empathize with you Laura. This may make you feel better. My fiance I trust implicitly but I have to say, it's a different ballgame out there when it comes to other women and even men for that matter.
When my fiance managed a restaraunt he was in actuality returning to a previous position so it would be normal for everyone to be friendly and joke around, etc. There was one person in particular I had very, VERY strong feelings about. She joked around like everyone else, but still that thing, you know? I mentioned it to him. He was receptive to my feelings but told me he could not say that was the case.
One particular night I had stopped by the restaraunt because he asked me to and within about an hour I was as tense as a bear. Couldn't explain it, but lost my appetite, couldn't focuss, etc. I ignored it, because, really, it could be anything, I didn't see anything and I felt very much like what you are describing, no trust issues and very confused as to why I was feeling what I was.
After they closed the restaurant he told me she went through a order-out window that night and handed him a hand written note asking him to phone her because she thought he wanted to but couldn't because I was in the restaurant. I said, "You have got to be joking. This reasoning comes from where??" She was calling him at work and he was not returning the calls.
Anyway, it's possible to pick up on other people's intentions and inadvertantly feel distant with your partner when they're not the ones instigating. Since you trust him and already have feelings about the woman I'll bet it has more to do with her than him. However I agree with phyllis, if you're married and feel you have to go to a party without your spouse, drink less and come home to them

and never be afraid to talk openly with the person you love most!
Karen Elleise
Clairvoyance Editor