A rising need for workers in science, technology, engineering and mathematics (STEM) fields has fueled interest in improving teaching within STEM disciplines. Numerous studies have demonstrated the benefits of active learning approaches on student learning outcomes. However, many of these studies have been conducted in experimental, rather than real-life class, settings. In addition, most of these studies have focused on in-class active learning exercises. This study tested the effects of a form of out-of-class active learning on exam performance for General Biology students at the University of Minnesota. An online database of 1,020 multiple-choice questions covering material from the first half of the course was generated, with feedback provided for 73% of the questions. Students in seven course sections (with an average of approximately 265 students per section) were given unlimited access to the online study questions. These students made extensive use of the online questions, with students answering an average of approximately 1,323 questions (94.5 questions per lecture) covering material from the half of the semester for which the questions were available. The extent to which access to the online study questions improved student learning outcomes was assessed by comparing the performance on exam questions of students in the seven course sections with access to the online study questions with the performance of students in course sections without access to the online study questions. Student performance was analyzed for a total of 89 different exams questions that were not included in the study questions, but that covered the same material covered by the study questions. Each of these 89 questions was used on one to five (average of 2.3) exams administered to students in course sections that had access to the online study questions and on three to 77 (average of 24.8) exams administered to students in sections that lacked access to the online study questions. Data from over 1,800 students in sections with access to the online study questions show that those students scored a statistically significant average of 6.6 percentage points higher on the exam questions analyzed than students in sections without access to the study questions. This difference was greater than the average amount necessary to raise students’ exam grades by one grade (e.g. from a “B-“ to a “B”). In addition, there was a higher correlation between number of questions answered and success on exam questions on material related to the study questions than between number of questions answered and success on exam questions on material unrelated to the study questions. The online study question system analyzed in this work required substantial effort to set up in the first year, but required minimal effort to maintain in subsequent years and was effective in significantly raising average exam scores for even very large course sections.