I understand that the integral of the spectrum over all frequencies, i.e. $\int S(\omega) \,d\omega $, is the variance which means that the integral should have $m^2$ units? Please correct me if I am wrong.
You are correct. If elevation $\eta(t)$ is the measured quantity (units of $m$), the Fourier transform of wave variance $\eta^2$ yields spectrum $S(f)$ with the units of $m^2/Hz$, or if you are working with angular frequency $\omega = 2\pi f$, it yields $S(\omega)$ with the units of $m^2/rad/Hz$.
More details can be found in the answer to this question.