Automatic Character Recognition for the handwritten Indic script has listed up as most the challenging area for research in the field of pattern recognition. Although a great amount of research work has been reported, but all the state-of-art methods are limited with optimal features. This article aims to suggest a well-defined recognition model which harnessed upon handwritten Odia characters and numerals by implementing a novel process of decomposition in terms of 3rd level Fast Discrete Curvelet Transform (FDCT) to get higher dimension feature vector. After that, Kernel-Principal Component Analysis (K-PCA) considered to obtained optimal features from FDCT feature. Finally, the classification is performed by using Probabilistic Neural Network (PNN) on handwritten Odia character and numeral dataset from both NIT Rourkela and IIT Bhubaneswar. The outcome of proposed scheme outperforms better as compared to existing model with optimized Gaussian kernel-based feature set.