Many studies, experimental, theoretical, and numerical, have been done on polymer nanocomposites, but nearly all of them have focused on a particular type of material system or some specific material properties. A comprehensive understanding of this complicated material system is still quite lacking. The objective of this study is to use mesoscale finite element simulation to gain insights on the reinforcing efficiencies of different types of carbon nanofillers as distinguished by their geometries and interfacial strengths. It is demonstrated that CNT (carbon nanotube) and CNF (carbon nanofiber) have larger load carrying capacity and potentially higher reinforcing efficiency than GNP (graphite nanoplatelet) due to their larger aspect ratio and physical length. However, the higher load carrying capacity is also associated with higher interfacial stress which can lead to earlier debonding, particularly for CNT. GNP, on the other hand, has lower load carrying capacity, and is thus less sensitive to the bonding condition and less susceptible to debonding. The overall reinforcing efficiency is a manifestation of the interplay between the load carrying capacity of the filler, which is limited by filler’s geometry, and the load transfer capability at the interface, which is limited by the filler/matrix interfacial strength. This interplay is also reflected in the effects of filler orientation on reinforcing efficiency. The insights gained from this study can be used to devise a strategy for developing advanced nanocomposites, such as hybrid composites.