At convergent boundaries where oceanic crust and continental crust collides, the mountain ranges can therefor form right at the coastline. Vulcanism is also important in the process as volcanoes forms where oceanic crust is subducted under the continental plates.

Some of the mountain ranges that today are located far from the coast, e.g. the Urals are formed from earlier convergent boundaries between continental plates that today are incorporated in larger plates.

Convergent boundaries explains most of the mountain ranges (e.g the Andes, Himalaya, the Alps, Nihon Arupusu) but there are however some curious exceptions, e.g. the Scandinavian Mountains that are not located near any convergent boundary and might be the result of regional uplift. However, there is an interesting ongoing discussion on how these processes work.

This is a simplified answer that I hope inspire to further studies in tectonic and geology.

While this is true in certain situations, it doesn't begin to explain how the vast majority of the earth's mountains are formed. In the global map below:

we see that the edges of many tectonic plates are edged by spreading ridges and subsequent mountain building. Obviously, most mountain building is actually the result divergent tectonics, and not convergent. Curiously, there is no interesting conversation about "how these processes work".

Nevertheless, it is interesting that Mid-Ocean Ridges as well as most continental margin mountains are aligned north-south. However, there is little reason to believe that mantle convection and thus plate motion MUST operate in this fashion and that it has always done so.

There is plenty of paleomagnetic evidence for landmasses moving in a wide variety of directions in the geologic past, that resulted in the creation of supercontinents. Today's N/S orientation may simply be an artifact of how the last supercontinent was broken up, and not evidence of the orientation of a permanent deep, driving force.