The Influence of Computer Experience on the Agreement between User Preference and Performance Rankings in Software Usability Testing

Author(s):  
George V. Kissel
2021 ◽  
Author(s):  
Xiao-Yue Gong ◽  
Vineet Goyal ◽  
Garud N. Iyengar ◽  
David Simchi-Levi ◽  
Rajan Udwani ◽  
...  

We consider an online assortment optimization problem where we have n substitutable products with fixed reusable capacities [Formula: see text]. In each period t, a user with some preferences (potentially adversarially chosen) who offers a subset of products, St, from the set of available products arrives at the seller’s platform. The user selects product [Formula: see text] with probability given by the preference model and uses it for a random number of periods, [Formula: see text], that is distributed i.i.d. according to some distribution that depends only on j generating a revenue [Formula: see text] for the seller. The goal of the seller is to find a policy that maximizes the expected cumulative revenue over a finite horizon T. Our main contribution is to show that a simple myopic policy (where we offer the myopically optimal assortment from the available products to each user) provides a good approximation for the problem. In particular, we show that the myopic policy is 1/2-competitive, that is, the expected cumulative revenue of the myopic policy is at least half the expected revenue of the optimal policy with full information about the sequence of user preference models and the distribution of random usage times of all the products. In contrast, the myopic policy does not require any information about future arrivals or the distribution of random usage times. The analysis is based on a coupling argument that allows us to bound the expected revenue of the optimal algorithm in terms of the expected revenue of the myopic policy. We also consider the setting where usage time distributions can depend on the type of each user and show that in this more general case there is no online algorithm with a nontrivial competitive ratio guarantee. Finally, we perform numerical experiments to compare the robustness and performance of myopic policy with other natural policies. This paper was accepted by Gabriel Weintraub, revenue management and analytics.


Author(s):  
William Porter ◽  
Carin Kosmoski ◽  
Rohan Fernando

Usability for any product, and especially for a lifesaving device, is critical in that the users will be interacting with the device in a highly stressful and complex environment. This study examined self- contained breathing apparatus (SCBAs) and conducted a usability assessment of these SCBAs with refill stations as it pertains to mine escape. Data was collected examining three usability topic areas (effectiveness, efficiency, and satisfaction) and eight constructs within these topic areas (completeness, accuracy, time requirements, overall satisfaction, discomfort, ease of use, system performance, and user preference). This paper documents the usability framework adopted and the methodology used to answer the research questions of the study and includes sample results and discussion. The methodology presented can be modified and used to test other lifesaving technologies to compare the usability of the devices and to estimate the ability of the devices to function as expected in a lifesaving situation.


2019 ◽  
Vol 36 (1) ◽  
pp. 1-8 ◽  
Author(s):  
Alyssa M. Valenti

Purpose This paper details a usability testing case study on a simplified homepage for [Library]. The usability testing was completed in Spring 2017 to meet the needs of diverse user groups and shifting trends in Web design and development. At the conclusion of the usability testing, recommendations for change informed the design decisions and a new homepage was implemented in October 2018. Design/methodology/approach The researcher performed eight usability tests with a combination of the different library user types; full-time faculty, students, an administrator and members of the public. The usability test consisted of 13 specific tasks. After testers completed the tasks, users filled out a 30-question Likert-scale questionnaire and answered a set of 8 open-ended questions. Findings This paper discusses the recommendations for change which the researcher discovered at the conclusion of the usability testing period. The research found the need to improve and include specific navigational, visual and easy-to-use elements to best meet the needs of the users in the usability tests. Changes were ranked and implemented on a scale of catastrophic to cosmetic. Research limitations/implications As websites, technology and user preferences continually evolve, the homepage will need to be tested for usability again in the next several years. Researchers are encouraged to adapt the methods to their own institutions. Practical implications This paper discusses findings specific to [Library], which in turn has proved to increase usage of certain features and functions by the user community. Originality/value This is the first time usability testing has been done for the [Library’s] website. It was the first time the design of the homepage was informed by real user preference. This paper is valuable to those looking to create a simple, easy-to-use homepage that best benefits their own unique community of users.


Author(s):  
Kaitlin M. Gallagher ◽  
Laura Cameron ◽  
Madison Boulé ◽  
Diana De Carvalho

Multiple monitor configurations are used in office settings to promote increased productivity by providing more screen space. Our systematic review compiled literature to determine if office workers who use multiple computer monitor configurations have altered health and performance outcomes compared to the use of a single monitor. A secondary purpose was to compare the studies’ monitor configurations to purchasing trends. Finally, we compiled user preference results and methodological information, such as the tasks used and participant placement. Our systematic review was registered on PROSPERO (Gallagher, Cameron, De Carvalho, & Boule, 2018) a-priori and conducted and reported according to the PRISMA statement guidelines (Moher, Liberati, Tetzlaff, Altman, & Group, 2010). Inclusion criteria were any study that assessed participants over the age of 18 years, looked at office work tasks, and assessed the use of either two or more monitors at a time in comparison to single monitor use. The primary outcomes were changes in health and performance-related variables. Secondary outcomes were user preference, the characteristics of the monitor configurations tested in the study, participant placement with respect to those monitors, and tasks used to assess configuration effectiveness. Two team members (KG & MB) independently screened the titles and abstracts to determine studies that potentially met the inclusion criteria. Justification for inclusion/exclusion was recorded on a standardized form. For all included studies, the independent reviewers separately extracted information and performed a risk of bias assessment. Discrepancies were resolved through discussion and if necessary, consultation with a third reviewer (DC). We included eighteen articles in the systematic review. Four studies were conducted in a field setting using workers’ real tasks and fourteen were conducted in a laboratory setting. Performance outcomes generally improved or remained the same with the use of multiple computer monitors versus a single monitor; however, results were shown to be influenced by the task involved. Health-related outcomes were less consistent and have not been investigated enough on multiple monitor configurations and larger displays. Head rotation from neutral is found with multiple monitor use. Muscle activity and discomfort measures need further assessment, especially for larger monitors. Future work should assess health and performance measures together to get a clear picture of the potential benefits and disadvantages of the monitor setup, be cognizant of the tasks and user placements chosen, consider recent purchasing trends when selecting monitors for research studies, and conduct field studies to assess the influence of monitor choice and placement on performance, and health and well-being.


1992 ◽  
Vol 36 (16) ◽  
pp. 1205-1209 ◽  
Author(s):  
George S. Hackman ◽  
David W. Biers

The purpose of the study was to compare a team usability testing paradigm with that of the typical single user paradigm in terms of the quantity and quality of the user's verbalization (i.e. thinking out-loud) and performance. The study employed a three group design in which the type of usability paradigm (Single, Observer, Team) was manipulated. Users first learned to use an off-the-shelf database management package by means of a short tutorial and then engaged in six structured tasks. While engaging in the tasks, the users either thought-out-loud alone (Single condition), in the presence of an observer (Observer condition), or as participants of a team working on the tasks together (Team condition). Results indicated that there were no significant differences among the three conditions in terms of performance nor any extensive differences in their subjective evaluation of the software. However, users in the Team condition spent more total time verbalizing than those in the Single or Observer condition. More importantly, results of a verbal protocol analysis revealed that the Team spent more time making statements which had high value for designers than did the other two conditions (which did not differ from one another). When broken out by individual users in the Team, there were no significant differences between individual team members and users in the other two conditions in making high value comments. The results suggest that the Team paradigm may be more efficient in extracting high value information without any noticeable differences in performance or subjective impression of the software.


1983 ◽  
Vol 27 (6) ◽  
pp. 441-444
Author(s):  
Craig J. Petrun ◽  
Suzanne Henry ◽  
Marian MacDonald ◽  
Robert Torrey ◽  
Eric Anderson

Three studies were performed to determine the effect of different formats on user preferences and performance. In Studies 1 and 2, operators were presented instructional formats which differed on the following variables: 1) the use of graphic illustrations, 2) the amount and use of color, 3) the type of blocking used to separate the information on each page. The results of both studies indicated that operator preferences were significantly affected by the use of graphics, random blocking, and amount of color. Study 3 examined the impact of preferred and non-preferred formats on operator performance and their preferences after using a given format. The results showed that there were no significant performance differences between the formats. The preference data, however, demonstrated that formats containing color, random blocking, and graphics were still the most preferred variables.


Sign in / Sign up

Export Citation Format

Share Document