The small grey mongoose scratches in the soil for insects, particularly grasshoppers and locusts, and also eats rats, mice, lizards, snakes, bird's eggs and chicks. They have a cat-like hunting style: crouching and stalking their prey, then dashing out and pouncing. The prey is usually killed by a head bite. The small grey mongoose is largely confined to the Cape Province: on farms it may make a home under the floorboards of an outbuilding or in some other man-made structure such as a barn or shed.
These mongoose sometimes climb trees either in search of food or to escape predators, but their claws are not particularly well equipped for either climbing or digging. They are normally solitary although occasionally they are seen in pairs. The small grey mongoose may shelter in another animal's burrow; alternatively they shelter among a pile of rocks or some other natural refuge. It is here that the young are weaned and they only emerge when they can fend for themselves: it is therefore very rare to see a female moving about with young.