The HFS are also enriched in the crust (eventually) but their name derives from their small radius compared to their high cationic charge: the z/r ratio. As a result, their bonding to nearby anions is very strong, that is - they have a high electrical field strength.
The subdivision between the two groups has been defined at z/r = 2.0, but as this is a continuous value, no strict theoretical definition of where the boundary lies can be given. Historically, the REE have been considered as LIL. In more modern times, the REE may be excluded when discussing HFS.
Why do we need the two groups?
Even though both the LILE and HFSE behave in an incompatible way during mantle melting, their response to post-magmatic processes differ. The HFSE are usually immobile: that is, they are mostly resistant to metamorphism and hydrothermal alteration. On the other hand, the LILE are fluid-mobile and hydrothermal alteration may change their contents in the studied rock.
Fresh rocks are a scarce luxury and many rocks that we study have experienced some kind of alteration. Because HFSE are resistant to these processes, their contents are likely to be representative of the original rock. This is extremely important: it is possible to look at a rock beyond the altered mineral composition and the modified major element contents. The LIL can teach us the opposite - we can learn about the alteration processes. If we do find fresh rocks and we find anomalies in the LIL systematics, we can learn about hydrothermal processes that occurred in the mantle that would otherwise not be able to see.
What about hexavalent cations?
One would think that hexavalent cations such as Mo6+, Cr6+, V6+ and U6+ should also appear in the figure as cations with an even more HFS character. However, when in the hexavalent state they form anionic complexes and do not behave like the cations in the figure.
Further reading
MIT OpenCourseWare - Trace-Element Geochemistry
The Use of Trace Elements in Igneous Petrology
About the figure
Inspired by figure 2.2 from Ore Deposit Geology / Scott. Data for figure from An Earth Scientist's Periodic Table of the Elements and Their Ions (also doi)