Although I'm definitely not an expert, I can sympathize as well as maybe offer a bit of advice!
First, remember that your SD is 16 years old. Girls at this age are inherently distant, moreso when they're confused. So, I think you could chalk some of that up to age, horomones, inexperience, etc.
Second, being in a step relationship is going to be a bit weird for a long time. Even when "roles" are established, your SD is going to mature and go through different stages, and those "roles" will have to be re-defined.
Third, it seems like all you really want is a good relationship with her, and a strong relationship with your DH. It doesn't seem like you're trying to "be the mom," which I think is often disastrous. Does she know this? If you take a few private moments for just the two of you, you could let her know that you are feeling awkward, too, but that you want things to be open and honest between the two of you. Maybe acknowledge that you see how much she loves her dad, and let her know that you also truly love her dad. Allow her to respond to what you're saying and acknowledge any issues she might be having, if she's comfortable enough to bring them up. If not, don't try to force anything. Let her come to you.
Also, be sure her father spends time with her without anyone else around so they can continue to have a strong relationship.
Finally, just be patient. Step-relationships take time to build. They can't be forced. Eventually, she will see that you make her father happy and that you aren't trying to take her mom's place. The awkwardness, then, will dissipate.
My SD and I have a pretty good relationship. I guess I'm kind of like an aunt to her, in a very roundabout kind of way, if you have to define it. But there are times when she's upset that I do sit her down in her room and remind her that her dad and I love each other very much, that she can trust that I'm here to stay, that occasional anger toward the "step-parent" is totally normal, and that this anger eventually goes away--and when it does, our relationship will be back to normal again.