Due to the enormous application, handwritten digit recognition (HDR) has become an extremely important domain in optical character recognition (OCR)-related research. The predominant challenges faced in this domain include different photometric inconsistencies together with computational complexity. In this paper, the authors proposed a language invariant shape-based feature descriptor using the refraction property of light rays. It is to be noted that the proposed approach is novel as an adaptation of refraction property is completely new in this domain. The proposed method is assessed using five datasets of five different languages. Among the five datasets, four are offline (written Devanagari, Bangla, Arabic, and Telugu) and one is online (written in Assamese) handwritten digit datasets. The approach provides admirable outcomes for online digits whereas; it yields satisfactory results for offline handwritten digits. The method gives good result for both online and offline handwritten digits, which proves its robustness. It is also computationally less expensive compared to other state-of-the-art methods including deep learning-based models.