The reason the mafic oceanic crust (which is indeed less dense than ultramafic mantle rock) sinks is because at enough depth, the temperature becomes high enough to cook out the portions of the sinking slab with a lower melting point. Basalt and related rock in oceanic crust is a mix of multiple chemicals, some with lower melting points, some with higher melting points. At enough depth, the materials with a lower melting point melts while the materials with a higher melting point remain solid.

The materials with a lower melting point also have a lower density than the materials with a higher melting point. The less dense liquid portion of the partial melt rises through the overlying material to form the volcanic arc oftentimes associated with a subduction zone. This is the key process by which much of the continental crust formed.

What's left behind in the solid portion of the partial melt is dense ultramafic rock, basically the same chemically as mantle material. As melting is endothermic, the partial melting process cools the remaining solid portion even further. (It was already a bit cooler than the surrounding upper mantle.) That makes it even more dense, denser than the surrounding mantle material, and hence it sinks.