Paramagnetic point defects in solids may exhibit a rich set of interesting and not yet fully resolved physics. In particular, character of wave functions and electron-phonon coupling in these defects may highly influence their interaction with external magnetic fields. Complex interplay among the electronic orbitals, phonons, and electron spin determines the effective pseudospin of the system that we demonstrate on vanadium and molybdenum defects in hexagonal silicon carbide by means of ab initio calculations. In this Rapid Communication, we find a giant anisotropy in the g tensor of these defects with Kramers doublet spin ground state, resulting in reduced and vanishing interaction with the magnetic field in parallel and transverse directions, respectively. The consequences of our finding in the application of these defects for quantum information processing are briefly discussed.