Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning