I would think that loud music or noises would irritate or stress hermit crabs. Considering that they live inside shells and are kept much of the time in plastic or glass aquariums where noise is echoed and amplified, being stressed would probably be common.
If your crab rarely comes out of his/her shell and is more defensive than usual, you may want to move him/her to a more quiet place.