To me, that's cheating. If I found my husband doing this, I'd feel betrayed. I'd feel disrespected, and I would definitely say something to him about it directly. I'd tell him how it made me feel, and I'd ask why he feels the need to do it, especially if it's only when I'm not at home. That sounds like he knows he's doing something wrong.
But you don't mention how YOU feel about it. If you want to say something to him, you should contemplate your own reaction and your own feelings first. Maybe if you communicate that to him, that'll get the conversation going.
Passive-aggressive moves have never gotten me anywhere but in fights . . .