I know what you mean. And I've lived an extreme example. My son is gifted, and in his early years was very highly emotional. Frustration, anger, crying fits, you name it. And even if they didn't actually say it, everyone was thinking "Send him to school, that'll fix him."
"Yeah", I would think, "but whose going to help him to navigate school? He can hardly deal with is own immense feelings. How's he going to deal with 30+ other people's raw feelings all at one time? No thanks, I'll homeschool him."
I wasn't popular, but 12 years later, I think that I was right.