Hybrid male progeny from interspecies cross are more prone to sterility or inviability than hybrid female progeny, and the male sterility and inviability often demonstrate a parent-of-origin asymmetry. However, the underlying mechanism of asymmetric sterility or inviability remains elusive. We previously established a genome-wide hybrid incompatibility (HI) landscape between Caenorhabditis briggsae and C. nigoni by phenotyping a large collection of C. nigoni strains each carrying a C. briggsae introgression. In this study, we investigate the genetic mechanism of asymmetric sterility and inviability in both hybrid male and female progeny between the two species. Specifically, we performed reciprocal crosses between C. briggsae and different C. nigoni strains that each carries a GFP-labeled C. briggsae genomic fragment referred to as introgression, and scored the HI phenotypes in the F1 progeny. The aggregated introgressions cover 94.6% of the C. briggsae genome, including 100% of the X chromosome. Surprisingly, we observed that two C. briggsae X fragments that produce C. nigoni male sterility as an introgression rescued hybrid F1 sterility in males fathered by C. briggsae, indicating that at least two separate X-autosome interactions are involved in the hybrid male sterility. In addition, we identified another two C. briggsae genomic intervals on the Chromosome II or IV, respectively, which can rescue the inviability, but not the sterility, of hybrid F1 males fathered by C. nigoni, suggesting the involvement of differential epistatic interactions in the asymmetric hybrid male fertility and inviability. Importantly, backcrossing of the rescued sterile males with C. nigoni led to isolation of a 1.1-Mb genomic interval that specifically interacts with an X-linked introgression, which is essential for hybrid male fertility. We further identified three C. briggsae genomic intervals on the Chromosome I, II and III, respectively that produce inviability in all F1 progeny dependent or independent of the parent-of-origin. Taken together, we identified multiple independent interacting loci that are responsible for asymmetric hybrid male and female sterility and inviability, which provides important insights into the asymmetric HI and lays a foundation for their molecular characterization.