Online Psychological Testing using Executables: A Case Study on Visual Memory Resolution

In psychological research, online experimentation is a valuable alternative to lab-based experimentation. However, converting existing psychological experiments to an online format can be difficult because this usually requires a translation from Python to JavaScript code. Here, we propose and validate a novel pipeline of online experimentation using executable files, which are downloadable programs that can be run by participants on their own computers. Critically, executables facilitate the transition from a lab-based behavioral experiment to online experimentation by allowing for flexible programming using well-established libraries (i.e., PsychoPy) without the need for translation across languages (i.e., from Python to JavaScript). As a case study, we paired the executable pipeline with a shape-color “conjunction” task to assess memory resolution (Li, Fukuda, and Barense, 2021), testing participants during a synchronous virtual conference room in order to maximize data quality. In a first experiment, we tested participants one-by-one (i.e., individual testing) and found that the reliability of timing and data quality was comparable to traditional lab-based experimentation, replicating a previously reported memory effect. In a second experiment, we assessed the test-retest reliability and the scalability of executables paired with virtual conferencing of multiple participants at once. We tested up to 18 participants in the same synchronous virtual conference room (i.e., concurrent testing), replicating the first experiment within 2% of original values. These results reveal that online experimentation using executables paired with concurrent testing via virtual conferencing is an efficient, reliable, and scalable alternative to browser-based online experimentation.