The folding and subnuclear compartmentalization of chromosomes relative to nuclear bodies is an integral part of gene function. However, mapping the three-dimensional (3D) organization of all genes, in single cells, on a genome-wide scale remains a major challenge. Here, we demonstrate that data-driven population-based modeling, from ensemble Hi-C data alone, can provide a detailed description of the nuclear microenvironment of genes. We define the microenvironment of a gene by its subnuclear positions with respect to different nuclear bodies, local chromatin compaction, and preferences in chromatin compartmentalization. These structural descriptors are determined in single cell models on a genome-wide scale, thereby revealing the dynamic variability of the subnuclear microenvironment of a gene across a population of cells. We demonstrate that the microenvironment of a gene is directly linked to its functional potential in gene transcription, replication, and subnuclear compartmentalization. Some chromatin regions are distinguished by their strong preferences to a single microenvironment (either transcriptionally active or silenced), due to strong associations to specific nuclear bodies. Other chromatin shows highly variable microenvironments and lacks specific preferences. We demonstrate that our method produces highly predictive genome structures, which accurately reproduce data from TSA-seq, DamID, and DNA-MERFISH imaging. Thus, our method considerably expands the range of Hi-C data analysis.