Launa,
You're definitely correct! That is an issue that many jobseekers struggle with. Out of 5 applications, at least 3 of them will have references we are unable to reach - their phone is disconnected, they no longer work there...etc...
It's so important to remember to let your references know that you have applied for a job and may get a call about a reference. Sometimes we reach the person, but they give a bad reference!
Whenever we can't get a hold of a reference or hear bad things, it always reflects negatively on the candidate. If you're not sure, call your prior company and find someone, a supervisor you worked with, anyone who can attest to your time in the company, or perhaps steer you to someone who CAN give you a reference.
Never leave a job on bad terms, always make sure you have someone you can use as a reference.