If it's not coming up easy, then you may be in for a lot of hard scrapping. We removed some linoleum that was over concrete and it ended up leaving pieces of fiber that couldn't even be sanded off well. I was hoping we could stain and seal the floor. No such luck. We ended up putting laminate flooring over it.
Good luck.