In this paper, we propose a Hierarchical Frequency Sensitive Competitive Learning (HFSCL) method to achieve Color Quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a Frequency Sensitive Competitive Learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that the proposed HFSCL has desired ability for CQ.