Chromosomal instability (CIN) is a hallmark of human cancer that involves mis-segregation of chromosomes during mitosis, leading to aneuploidy and genomic copy number heterogeneity. CIN is a prognostic marker in a variety of cancers, yet, gold-standard experimental assessment of chromosome mis-segregation is difficult in the routine clinical setting. As a result, CIN status is not readily testable for cancer patients in such setting. On the other hand, the gold-standard for cancer diagnosis and grading, histopathological examinations, are ubiquitously available. In this study, we sought to explore whether CIN status can be predicted using hematoxylin and eosin (H&E) histology in breast cancer patients. Specifically, we examined whether CIN, defined using a genomic aneuploidy burden approach, can be predicted using a deep learning-based model. We applied transfer learning on convolutional neural network (CNN) models to extract histological features and trained a multilayer perceptron (MLP) after aggregating patch features obtained from whole slide images. When applied to a breast cancer cohort of 1,010 patients (Training set: n=858 patients, Test set: n=152 patients) from The Cancer Genome Atlas (TCGA) where 485 patients have high CIN status, our model accurately classified CIN status, achieving an area under the curve (AUC) of 0.822 with 81.2% sensitivity and 68.7% specificity in the test set. Patch-level predictions of CIN status suggested intra-tumor spatial heterogeneity within slides. Moreover, presence of patches with high predicted CIN score within an entire slide was more predictive of clinical outcome than the average CIN score of the slide, thus underscoring the clinical importance of spatial heterogeneity. Overall, we demonstrated the ability of deep learning methods to predict CIN status based on histopathology slide images. Our model is not breast cancer subtype specific and the method can be potentially extended to other cancer types.