Heritability, the fraction of phenotypic variance attributable to the action of genes, is usually derived from a linear statistical partition of variance. In this paper we study a dichotomous phenotype whose transmission from parents to offspring depends on the parents’ phenotypes and the offspring’s genotype. Each individual is then represented as a phenogenotype. We derive expressions for each component of phenotypic variance and for covariances between relatives of various degrees. The resulting heritability estimates vary with the rates of phenotypic transmission as well as with the genetic contribution to the phenotype. Assortative mating by phenotype in parents is also shown to contribute to the correlations between relatives. In addition, we show that the frequency of alleles at genes affecting the phenotypes strongly affects standard heritability measures. This is important because for most complex traits these allele frequencies cannot be ascertained.