I've had two types of Clairaudience. One type is subtle. It comes with messages and/or information on a person and sounds similar to my own thoughts but, and this is important. It's information i could not have had prior knowledge to. It's different from telepathy where you are just translating a person's thoughts through your own. It feels like outside information.
The other type is very audial. It's loud, it can sound male, vibratory, echo-ey, or feminine, but nothing like my own sounds. Those are the types that shock me. I'll get them when I'm dozing, right before I wake up or around 3:00 am. The language used is almost shakespearian.
At one time I thought it might be a guide or spirit of some sort. It scared me or rather would jolt me out of sleep and finally I said, this is not o.k. I prefer not to work w/whoever you are and that particular voice, at least, never returned.
I'd say the mental anguish come in the anxiety of feeling it all. I don't know if that's an illness. it seems just to come w/the territory.
I think whether or not you have full access to your faculties or free-will would help to set the two apart. I've never had an experience that had a will greater than my own - startling isn't the same as losing control.