scholarly journals Usability testing of a Digital Assessment Routing Tool: An Iterative Convergent Mixed Methods Study Protocol (Preprint)

2021 ◽  
Author(s):  
Cabella Lowe ◽  
Harry Hanuman-Sing ◽  
Mitchell Browne ◽  
Meshari F. Alwashmi ◽  
William Marsh ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Mixed Methods ◽  
Data Collection ◽  
Quantitative Data ◽  
Usability Testing ◽  
Manuscript Submission ◽  
Digital Assessment ◽  
Usability Problems ◽  
System Usability Scale ◽  
System Usability

BACKGROUND Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on healthcare utilization. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials, the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials. OBJECTIVE This study will use the iterative-convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. This will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. METHODS We will collect qualitative and quantitative data from between 20-30 participants aged 18 and over four months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 Standard and the System Usability Scale (SUS), providing a benchmark usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles. RESULTS This study received approval from Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) in June 2020. At manuscript submission, the first round of individual interviews has been completed, with data collection to be completed and results published in 2021. CONCLUSIONS This study will provide evidence concerning mHealth DART system usability and acceptance determining system improvements required to support user adoption and minimize sub-optimal system usability as a potential confounder within subsequent non-inferiority clinical trials. This deliberately rigorous approach to mHealth innovation could be used as a guide for other developers of similar applications. Success should produce a safe, effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services.

Validation of a Musculoskeletal Digital Assessment Routing Tool (DART): Protocol for a Pilot Randomised Crossover Non-Inferiority Trial (Preprint)

2021 ◽  
Author(s):  
Cabella Lowe ◽  
Harry Hanuman Sing ◽  
William Marsh ◽  
Dylan Morrissey
Keyword(s):  
Clinical Trials ◽  
Data Collection ◽  
Real World ◽  
User Satisfaction ◽  
Allocation Concealment ◽  
Trial Methodology ◽  
Primary Analysis ◽  
Digital Assessment ◽  
System Usability Scale ◽  
The Impact

BACKGROUND Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on healthcare utilization. Digital technologies to improve health care outcomes and efficiency are considered a priority; however, innovations are rarely tested with sufficient rigor in clinical trials, the gold standard for clinical proof of safety and efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires validation in a real-world setting prior to implementation. OBJECTIVE This pilot study will assess the feasibility of a future trial by exploring key aspects of trial methodology, assess the procedures and collect exploratory data to inform the design of a definitive, randomized, crossover, non-inferiority trial to assess DART safety and effectiveness. METHODS We will collect data from 76 adult participants presenting to an NHS England GP practice with a musculoskeletal condition. Participants will complete both a DART assessment and a physiotherapist-led triage with the order determined by randomization. The primary analysis will involve an absolute agreement ICC (A,1) estimate with 95% confidence intervals between DART and the clinician for assessment outcomes sign-posting to condition management pathways. Data will be collected to allow analysis of participant recruitment and retention, randomization, allocation concealment, blinding, data collection process and bias. In addition, the impact of trial burden and potential barriers to intervention delivery will be considered. DART user satisfaction will be measured using the System Usability Scale. RESULTS A UK NHS ethics submission will be submitted during June 2021 and pending approval, recruitment will commence during August 2021 with data collection anticipated to last for 3 months. Results will be reported in a follow-up paper later in 2021. CONCLUSIONS This study will inform the design of a randomized controlled crossover non-inferiority study that will provide evidence concerning mHealth DART system clinical sign posting in an NHS setting prior to real-world implementation. Success should produce evidence of a safe, effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This rigorous approach to mHealth system testing could be used as a guide for other developers of similar applications. CLINICALTRIAL This trial is registered with Clinical Trials number NCT04904029

Evaluasi Usability pada Aplikasi Mobile ACC.ONE menggunakan System Usability Scale (SUS) dan Usability Testing

2021 ◽  
Vol 7 (1) ◽  
pp. 1-10
Author(s):  
Veni Manik ◽  
Clara Hetty Primasari ◽  
Yohanes Priadi Wibisono ◽  
Aloysius Bagas Pradipta Irianto
Keyword(s):  
Usability Testing ◽  
Low Grade ◽  
Percentile Rank ◽  
System Usability Scale ◽  
System Usability ◽  
Grade Scale

Perusahaan pembiayaan kendaraan mobil saat ini semakin banyak dan berkembang begitu pesat. Setiap perusahaan berusaha untuk menarik perhatian pelanggan dengan berbagai cara, terutama peningkatan teknologi untuk mendukung dan memudahkan proses kegiatan pembiayaan mobil. Perusahaan Astra Credit Company memanfaatkan teknologi dengan menyediakan aplikasi mobile acc.one yang dapat digunakan oleh pelanggan untuk membantu segala kegiatan proses pembiayaan mobil di Perusahaan ACC. Namun aplikasi ini terdapat beberapa masalah yang dapat diketahui dari ulasan pengguna di play store seperti aplikasi tiba-tiba error, aplikasi tidak dapat dibuka, fitur tidak berfungsi dengan semestinya, dll. Sehingga dilakukan pengujian aplikasi untuk mengetahui tingkat usability aplikasi dengan menggunakan metode system usability scale dan usability testing untuk mengetahui pengalaman pengguna. Hasil yang diperoleh tingkat usability aplikasi dengan tingkat kemudahan sebesar 73,33% yang masuk dalam kategori buruk, tingkat kecepatan pengguna sebesar 0,0467 goals/sec yang termasuk kategori normal, tingkat kesalahan pengguna sebesar 0,1230 total defect yang masuk dalam kategori kecil, dan tingkat kepuasan pengguna menggunakan kuesioner SUS memperoleh rerata SUS sebesar 54,45945946 yang artinya tingkat kepuasan pengguna rendah terhadap aplikasi berdasarkan penentuan nilai SUS yaitu Acceptability atau tingkat penerimaan penggunaan masuk dalam kategori marginal low, Grade Scale atau tingkat grade skala masuk dalam kategori F dan Adjective Rating masuk dalam kategori OK. Selain itu, pada SUS skor percentile rank masuk kedalam grade D.

scholarly journals Usability Testing SIKTA di Program Studi Manajemen Informatika dari Pengguna Mahasiswa

2021 ◽  
Vol 18 (2) ◽  
pp. 206
Author(s):  
Ni Wayan Marti ◽  
Kadek Surya Mahedy
Keyword(s):  
Usability Testing ◽  
System Usability Scale ◽  
System Usability

Penelitian ini bertujuan untuk melakukan usability testing terhadap Sistem Informasi Kontrol Mutu dan Pengelolaan Tugas Akhir Mahasiswa di Program Studi Manajemen Informatika (SIKTA). SIKTA merupakan sistem informasi yang digunakan untuk melakukan kontrol terhadap proses penyelesaian Tugas Akhir Mahasiswa. SIKTA adalah aplikasi yang digunakan secara internal di Program Studi Manajemen Informatika. Metode yang digunakan untuk melakukan usability testing terhadap SIKTA adalah ISO 9241-11. ISO 9241-11 merupakan uji usability untuk mengukur tingkat efektivitas, efisiensi dan kepuasan pengguna. Tingkat efektivitas dan efisiensi diukur dengan pemberian tugas-tugas kepada responden, sedangkan untuk mengukur kepuasan pengguna digunakan kuisioner dari System Usability Scale (SUS).  Dari hasil uji diperoleh masing-masing yaitu uji keefektifan sebesar 92%, uji keefisienan memperoleh 88% dan kepuasan pengguna menggunakan SUS memperoleh angka 76,5. Dari uji tersebut, dapat dikatakan bahwa SIKTA merupakan sistem yang efektif, efisien dari sisi mahasiswa sebagai penggunanya. Dan angka 76,5 menyatakan bahwa pengguna puas dalam menggunakan SIKTA.

Assessing Multifunction Interfaces in Vehicles

2005 ◽  
Vol 1937 (1) ◽  
pp. 66-72
Author(s):  
Peter C. Burns ◽  
Patricia L. Trbovich ◽  
Joanne L. Harbluk ◽  
Tara McCurdie
Keyword(s):  
Usability Testing ◽  
Safety Performance ◽  
System Safety ◽  
Safety Testing ◽  
System A ◽  
Usability Problems ◽  
System Usability ◽  
Versus System

The goal of this research was to determine whether system usability is requisite for system safety. To this end, the usability and safety of two multifunction in-vehicle interfaces were assessed to verify the hypothesis that the system scoring highest on usability testing would also score highest on safety testing. Two multifunctional systems were subjected to ( a) heuristic evaluations to assess usability and ( b) occlusion tests to assess safety. There were more heuristic violations, indicative of more usability problems, in System B relative to System A. Similarly, with regard to safety, results of occlusion testing showed that greater demands on time and visual resources were required to perform tasks when System B was used versus System A. Thus, the usability problems identified through heuristic evaluations represent possible precursors to the safety problems identified by occlusion tests. Given the latter and the fact that heuristic evaluations can be applied quickly and easily whereas occlusion testing can be costly and time-consuming, the results of the current research suggest that developers would benefit from correcting the usability limitations of a system before its safety performance is evaluated.

scholarly journals ANALISIS USABILITAS PADA APLIKASI MANDIRI ONLINE

2020 ◽  
Vol 15 (1) ◽  
pp. 11
Author(s):  
Astrid Wahyu Adventri Wibowo
Keyword(s):  
Evaluation System ◽  
Internet Banking ◽  
Mobile Banking ◽  
Heuristic Evaluation ◽  
Online Application ◽  
Evaluation Approach ◽  
Usability Problems ◽  
System Usability Scale ◽  
System Usability ◽  
Search Feature

Internet banking atau mobile banking adalah salah satu bentuk aplikasi yang berbasis internet yang dapat digunakan sebagai pintu gerbang menuju dunia perbankan masa depan. Dalam memberikan layanan, pihak bank tidak ingin mengecewakan nasabahnya karena aplikasi internet banking yang sulit digunakan sehingga mengakibatkan nasabah gagal dalam mencari informasi atau gagal melakukan aktivitas perbankan. Melihat pentingnya usabilitas sebuah aplikasi, maka dilakukan penelitian terhadap aplikasi Mandiri Online untuk mengetahui kelayakan dari aplikasi tersebut, mengidentifikasi permasalahan usabilitas, serta menyusun rekomendasi perbaikan. Aplikasi tersebut diuji menggunakan System Usability Scale (SUS) dan pendekatan heuristic evaluation dengan responden sebanyak 40 orang untuk metode SUS dan 5 evaluator dengan kriteria single-expert untuk pendekatan heuristic evaluation. Hasil pengujian aplikasi Mandiri Online dengan instrumen SUS adalah sebesar 79,6 yang berarti aplikasi Mandiri Online dinyatakan dapat diterima (acceptable), termasuk dalam grade B dengan rating good. Dari penelitian ini juga diketahui bahwa terdapat beberapa permasalahan usabilitas mayor yang harus segera diperbaiki dan harus diberi prioritas tinggi. Dalam menyelesaikan permasalahan tersebut membutuhkan beberapa usaha, seperti  menambahkan fitur search untuk mencari nama bank, membuat pengelompokkan instansi pada menu multipayment, membuat atau menambahkan panduan penggunaan atau flowchart penggunaan Mandiri Online, dan menambahkan fitur help pada Mandiri Online. Abstract[Title: Usability Analysis of Mandiri Online Apllication] Internet banking or mobile banking is a form of internet-based application that can be used as a gateway to the future of banking. In providing services, banks won’t to let down their customers because of internet banking applications are difficult to use, resulting in failure to find information or fail to conduct banking activities. Seeing the importance of application usability, a study of Mandiri Online application was conducted to find out the appropriatness of the application, identify usability issues, and construct recommendations for application improvement. This application was tested using the System Usability Scale (SUS) and heuristic evaluation approach with 40 respondents for the SUS and 5 evaluators with single-expert criteria for the heuristic evaluation. The result of testing the Mandiri Online application with the SUS instrument amounted to 79,6 which means this application was declared acceptable, included in grade B, and with a good rating. From this study it is also known that there are some major usability problems that must be corrected immediately and must be given high priority. In solving these problems requires several efforts, such as adding search feature to search the bank’s name, creating agency groups on multipayment menu, creating or adding usage guidelines or flowcharts for using Mandiri Online, and adding help feature to Mandiri Online.Keywords: Heuristic Evaluation; System Usability Scale; Usability

scholarly journals USABILITY TESTING PADA WEBSITE E-COMMERCE MENGGUNAKAN METODE SYSTEM USABILITY SCALE (SUS) (STUDI KASUS : UMKMBULELENG.COM)

2019 ◽  
Vol 8 (2) ◽  
pp. 149
Author(s):  
Agus Aan Jiwa Permana
Keyword(s):  
Usability Testing ◽  
System Usability Scale ◽  
System Usability

Pengembangan perangkat lunak sangat bermanfaat untuk membantu di beberapa bidang. Salah satunya adalah bidang pemasaran. Dengan perkembangan internet, proses pemasaran sudah lumbrah dilakukan melalui online dengan beberapa metode jual beli yang dilakukan seperti cash on delivery (COD), bayar di muka,  atau bayar barang saja dengan ongkos kurir dibayar setelah barang diterima. Pemasaran online memerlukan E-commerce  untuk membantu menghubungkan pembeli dengan penjual. Seperti yang dilakukan oleh kelompok perajin di Singaraja, dengan alamat website yang dapat diakses pada http://umkmbuleleng.com. Website ini sudah online, namun perlu dilakukan uji terkait untuk mengetahui apakah user dapat dengan mudah menggunakan aplikasi secara efektif dan efisien. Hal ini dilakukan untuk mengetahui tingkat kepuasan pengguna. Sebelum produk di launching secara luas, perlu dilakukan proses pengujian ini. Metode yang digunakan untuk melakukan pengujian ini adalah System Usability Scale (SUS). Hasil pengujian diperoleh yaitu baik. Sehingga aplikasi yang dikembangkan untuk perajin secara umum sudah sesuai dengan harapan pengguna

scholarly journals USABILITY TESTING DENGAN ISO/IEC 9126-4 SISTEM INFORMASI AKADEMIK UNIVERSITAS PENDIDIKAN GANESHA DITINJAU DARI PENGGUNA DOSEN

2018 ◽  
Vol 15 (2) ◽  
Author(s):  
Ni wayan Erna Erawati ◽  
I Ketut Resika Arthana ◽  
I Made Ardwi Pradnyana
Keyword(s):  
Relative Efficiency ◽  
Usability Testing ◽  
Completion Rate ◽  
System Usability Scale ◽  
System Usability

Penelitian ini dilakukan untuk menganalisis tingkat Usability pada SIAK UNDIKSHA khususnya pada SIAK Dosen. Tujuan dari analisis SIAK Dosen yaitu untuk mengetahui pengalaman pengguna dalam mengerjakan tugas-tugas yang diberikan. Selain itu analisis Usability ini dilakukan untuk mengetahui apa saja kemudahan-kemudahan serta kesulitan yang dialami oleh pengguna dalam membantu menyelesaikan tugas.Metode yang digunakan dalam penelitian ini yaitu metode Usability Testing dengan menggunakan ISO/IEC 9126-4 yang memiliki tiga atribut yaitu Efektifitas, Efisien serta Kepuasan Pengguna. Untuk menentukan nilai atribut Efektifitas digunakan alat ukur Completion Rate dan Number Off Errors. Untuk menentukan nilai dari atribut Efisiensi digunakan alat ukur Overall Relative Efficiency serta menggunakan teknik Performace Measurement.  Atribut Kepuasan Pengguna menggunakan alat ukur Test Lever Satisfaction dengan menggunakan teknik Concurrent Thing Aloud (CTA) dan  serta menggunakan kuesioner System Usability Scale (SUS). Populasi dari penelitian ini yaitu seluruh dosen aktif UNDIKSHA. Responden dari penelitian ini berjumlah 20 responden yang digolongkan menjadi 2 kelompok yaitu kelompok responden pemula dan kelopopok responden mahir.            Hasil dari penelitian ini yaitu pada atribut efektifitas diperoleh hasil 86,92% pada responden pemula dan responden mahir 86,92% yang artinya SIAK Undiksha sudah Efektif, pada atribut efisiensi diperoleh hasil pada responden pemula sebesar 87.29776% dan kelompok responden mahir yaitu sebesar 84.6087% yang artinya SIAK Undiksha sudah efisien. Pada atribut kepuasan pengguna diperoleh hasil 77.1 % yang artiya responden Puas dalam mengerjakan tugas pada SIAK UNDIKSHA.

scholarly journals A Tool to Measure Young Adults’ Food Intake: Design and Development of an Australian Database of Foods for the Eat and Track Smartphone App (Preprint)

2018 ◽  
Author(s):  
Lyndal Wellard-Cole ◽  
Melisa Potter ◽  
Jisu (Joseph) Jung ◽  
Juliana Chen ◽  
Judy Kay ◽  
...  
Keyword(s):  
Young Adults ◽  
Data Collection ◽  
Fast Food ◽  
Dietary Assessment ◽  
Usability Testing ◽  
Mobile Phone Data ◽  
System Usability Scale ◽  
Food And Beverage ◽  
Indigenous Foods ◽  
Keyword Searching

BACKGROUND Dietary assessment is reliant on the collection of accurate food and beverage consumption data. Technology has been harnessed to standardize recording and provide automatic nutritional analysis to reduce cost and researcher burden. OBJECTIVE To better assess the diet of young adults, especially relating to the contribution of foods prepared outside the home, a database was needed to support a mobile phone data collection app. The app also required usability testing to assure ease of entry of foods and beverages. This paper describes the development of the Eat and Track app (EaT app) and the database underpinning it. METHODS The Australian Food and Nutrient Database 2011-13, consisting of 5740 food items was modified. Four steps were undertaken: (1) foods not consumed by young adults were removed, (2) nutritionally similar foods were merged, (3) foods available from the 30 largest ready-to-eat food chains in Australia were added, and (4) long generic food names were shortened and simplified. This database was used to underpin the EaT app. Qualitative, iterative usability testing of the EaT app was conducted in three phases using the “Think Aloud” method. Responses were sorted and coded using content analysis. The System Usability Scale (SUS) was administered to measure the EaT app’s perceived usability. RESULTS In total, 1694 (29.51%) foods were removed from the Australian Food and Nutrient Database, including 608 (35.89%) ingredients, 81 (4.78%) foods already captured in the fast food chain information, 52 (3.07%) indigenous foods, 25 (1.48%) nutrients/dietary supplements, and 16 (0.94%) child-specific foods. The remaining 912 (53.84%) foods removed were not consumed by young adults in previous surveys or were “not defined” in the Australian Food and Nutrient Database. Another 220 (3.83%) nutritionally similar foods were combined. The final database consisted of 6274 foods. Fifteen participants completed usability testing. Issues identified by participants fell under six themes: keywords for searching, history list of entered foods, amounts and units, the keypad, food names, and search function. Suggestions for improvement were collected, incorporated, and tested in each iteration of the app. The SUS of the final version of the EaT app was rated 69. CONCLUSIONS A food and beverage database has been developed to underpin the EaT app, enabling data collection on the eating-out habits of 18- to 30-year-old Australians. The development process has resulted in a database with commonly used food names, extensive coverage of foods from ready-to-eat chains, and commonly eaten portion sizes. Feedback from app usability testing led to enhanced keyword searching and the addition of functions to enhance usability such as adding brief instructional screens. There is potential for the features of the EaT app to facilitate the collection of more accurate dietary intake data. The database and the app will be valuable dietary assessment resources for researchers.

scholarly journals Analisa User Experience Pada Sistem Informasi Akademik Universitas Pendidikan Ganesha Ditinjau dari Pengguna Mahasiswa

2018 ◽  
Vol 15 (1) ◽  
Author(s):  
Dewa Ayu Putu Ari Widhiani ◽  
I Ketut Resika Arthana ◽  
I Made Ardwi Pradnyana
Keyword(s):  
Performance Measurement ◽  
User Experience ◽  
Usability Testing ◽  
Think Aloud ◽  
System Usability Scale ◽  
Testing Performance ◽  
System Usability

Penelitian ini bertujuan untuk mengetahui tingkat kepuasan pengguna dan mengevaluasi SIAK Undiksha ditinjau dari pengguna mahasiswa menggunakan metode Usability Testing dengan teknik Firstclick Usability Testing, Performance Measurement, Retrospective Think Aloud (RTA), kuisoner System Usability Scale (SUS) serta memberikan rekomendasi perbaikan terhadap sistem tersebut berdasarkan atas hasil penelitian. Ada beberapa aspek yang dikaji dalam penelitian ini yaitu efektivitas (effectiveness), efisiensi (efficiency), tingkat kesalahan (error), dan kepuasan pengguna (satisfaction) menggunakan SIAK Undiksha ditinjau dari pengguna mahasiswa.Populasi dari penelitian ini yaitu mahasiswa Undiksha dengan melibatkan 385 responden kuisioner SUS dan 20 orang responden usability testing. Hasil penelitian berdasarkan analisis Mann Whitney U-test dan SUS menunjukkan bahwa (1) SIAK Undiksha ditinjau dari pengguna mahasiswa sudah efektif akan tetapi masih terdapat beberapa error yang ditemukan dalam sistem, (2) SIAK Undiksha ditinjau dari pengguna mahasiswa sudah efisien dilihat dari tidak adanya perbedaan yang signifikan waktu pengerjaan tugas kelompok mahir dan kelompok pemula, dan (3) Pengguna merasa kurang puas menggunakan SIAK Undiksha ditinjau dari pengguna mahasiswa dilihat dari skor kuisioner SUS sebesar 61,6.Dengan demikian berdasarkan hasil analisis data tersebut disimpulkan bahwa SIAK Undiksha ditinjau dari pengguna mahasiswa sudah memenuhi kriteria usability baik akan tetapi pengguna belum puas  saat menggunakan SIAK Undiksha ditinjau dari pengguna mahasiswa. Hal ini dapat dilihat pada penilaian tingkat efektivitas, efisiensi, tingkat kesalahan dan kepuasan pengguna, SIAK Undiksha ditinjau dari pengguna tersebut hanya berhasil memenuhi 2 dari 4 aspek. Oleh sebab itu, rekomendasi perbaikan dalam penelitian ini difokuskan untuk mengubah tata letak (layout) halaman dan menu navigasi sesuai hasil data penyebab kesalahan (error) pengguna dan menyederhanakan penggunaa Bahasa. Perbaikan dilakukan dengan membuat wireframe halaman utama SIAK Undiksha itinjau dari pengguna mahasiswa, halaman menu prosedur, halaman submenu biodata, halaman ubah biodata, halaman statistic nilai mahasiswa dan halaman cetak kartu rencana studi.

Sign in / Sign up

Export Citation Format

Share Document