Heuristic and Formative Evaluation: A Case Study Illustration of a New Technique

The purpose of the present work was to carry out an evaluation of an interactive, instructional Website that teaches the basic tenets of human performance technology. The evaluation methodology was based upon a unique combination of heuristic and formative evaluation techniques. It involved measuring attitudinal reactions to the Website, learning gains from performance scores on practice exercises, and content, navigation, and design areas needing modification or revision. Evaluation data were gathered from five students pursuing graduate degrees in education at an urban university. Paper-based attitude surveys, think-aloud protocols, and heuristic response forms were utilized to collect data. Student evaluators found the content in the Website to be useful and interesting; however, in some instances the practice items were confusing. The site was found to be easy to navigate and, overall, evaluators enjoyed using it. The evaluation methodology was shown to be effective in assessing design, content, and attitudinal issues, although in the future think-aloud protocols may be optional because they do not provide sufficient data to warrant the time spent on their use. Data also revealed that measuring learning gains was critical to the accurate evaluation and educational effectiveness of instructional Websites.