1. Prograde metamorphism results in dehydration and loss of water⁽¹⁾. Most of the original water content of the rock was in various hydrous minerals, and as you progress in your metamorphic path you lose more and more water to dehydration reactions. While it is true that in blueschist waster can be retained in minerals such as lawsonite, phengite, epidote and glaucophane, most of it is still lost. If you want to reequilibrate the rock to its previous mineral assemblage, you need to reintroduce the water. This can occur occasionally, for instance in this classic example from Norway:

   ^{source: http://www.geol.ucsb.edu/faculty/hacker/geo102C/lectures/part11.html}
   ...where a granulite was rehydrated along a joint to form eclogite, but it's usually very localised and uncommon.

2. Most prograde metamorphic reactions are exothermic, i.e. they release heat. Retrograde reactions are usually endothermic. They require heat in order to proceed, but in the event of exhumation, temperatures are decreasing. There is simply not enough heat around to support the retrograde reactions. Furthermore, you have to remember that lower temperature also lowers the reaction kinetics.

This doesn't mean that fast exhumation cannot occur...

There are pieces of evidence that in some cases exhumation can be even faster than the erosion rate⁽²⁾. This suggest that orogenic events may also result from simple uplift and not only compression and folding. What causes the uplift? This may actually be the result of crustal thinning and extensional stress regimes, as in the examples of Liaodong⁽³⁾ and Brittany⁽⁴⁾. This phenomenon is also known as a core complex, where extensional tectonics result in low angle normal detachment faults that results in rapid unroofing and exposure deep seated rocks. You may be familiar with oceanic core complexes, so this processes is quite similar.

⁽¹⁾ Forgive me for not including the zeolite facies in my definition of metamorphism.

⁽²⁾ Rubatto, D., & Hermann, J. (2001). Exhumation as fast as subduction?. Geology, 29(1), 3-6. 10.1130/0091-7613(2001)029<0003:EAFAS>2.0.CO;2

⁽³⁾ Yang, J. H., Wu, F. Y., Chung, S. L., Lo, C. H., Wilde, S. A., & Davis, G. A. (2007). Rapid exhumation and cooling of the Liaonan metamorphic core complex: Inferences from ⁴⁰Ar/³⁹Ar thermochronology and implications for Late Mesozoic extension in the eastern North China Craton. Geological Society of America Bulletin, 119(11-12), 1405-1414. 10.1130/B26085.1

⁽⁴⁾ Brown, M., & Dallmeyer, R. D. (1996). Rapid Variscan exhumation and the role of magma in core complex formation: southern Brittany metamorphic belt, France. Journal of Metamorphic Geology, 14(3), 361-379. 10.1111/j.1525-1314.1996.00361.x