Genetic erosion impedes adaptive responses to stressful environments