Where am I going with this?
If the ground gets the same energy. And if it's made of the same material and has the same color, it would therefore gain the same amount of temperature. And then the air's heating comes up from the ground below.
So then, once again, why are mountains cooler?!?
Because of their EXPOSURE.
Air predominantly flows horizontally rather than vertically (basically consider this an effect of gravity and the resultant air density distribution). And it's that horizontal wind that changes everything. Because most sites on mountains are generally quite proximate (at least regionally) to places where winds will blow in air that was higher above the lower elevation ground surface, and thus colder. The air on mountains, with limited protection (and because winds are higher aloft) is mixed strongly with the neighboring adiabatically cooler air that's sitting above the sea-level sites.
Just for more convincing this is vital, I invite you to look at these climatographs comparing two cities in the US.
What are these towns?
Red: Wendover, Utah (elevation 4,291', latitude 40.73° N)
Blue: Latrobe, Pennsylvania (elevation 997', 40.32° N) Over a km difference in height. So you'd expect Latrobe to be about 18°F warmer.
But it isn't. Instead it's cooler (especially during summer afternoons, which average up around 10 degrees apart).
Why? Because a lot of the intermountain west is actually quite a wide flat elevated basin\plateau. So in the central part of this area, there isn't as much exposure to the colder air coming in from places with lower topography.
Rest assured, it was very hard to find examples like this. You need a very wide flat elevated area, which is fairly uncommon. Plus just the right place to compare to for emphasis.
There are other common factors that do enhance the cold predilection on mountains, including the lower moisture content at elevation, the often lighter land color of raised terrain, and the increased cloud coverage due to air rising orographically.
But the fundamental reason it is colder on mountains is adiabatic cooling and the exposure of raised locations to that colder air.