Does prednisone give you a burning esophagus, mouth, tongue and even lips? My mouth even tastes bad.

I have been taking Pantaloc and tums, but i have to suck on ice chips all day.
Is this normal?