Cognitive Interviews: Testing the Digital Health Literacy Instrument (Preprint)
BACKGROUND Despite the increasing number of youth seeking health information on the internet, few studies have been conducted to measure digital health literacy in this population. Digital Health Literacy Instruments (DHLI) is defined as a scale that measures the ability to operate digital devices, and read and write in online-based modes; it assesses seven sub-constructs: operational skills, navigation skills, evaluating reliability, determining relevance, adding self-generated content to a web-based app, and protecting and respecting privacy. Currently, there has been no validation process for adolescents on this instrument yet. OBJECTIVE To explore the usability and content validity of DHLIs. METHODS Upon the approval of IRB protocol, cognitive interviews were conducted. Thirty-four adolescents aged 10-18 years old (50% female) participated in individual cognitive interviews. Two rounds of concurrent cognitive interviews were conducted to measure the content validity of DHLI utilizing the ‘thinking aloud’ method and probing questions. RESULTS Major issues of comprehension and communication including unclear wording, undefined technical terms, vague terms, and difficult vocabularies were identified. Problems related to potentially inappropriate assumptions were also identified. In addition, concerns related to recall bias with unclear reference periods, and bias by measuring socially uncommon phenomena were raised. No issue regarding response options or instrument instructions were noted. CONCLUSIONS The initial round of interviews provided potential resolution to the identified problems with comprehension and communication, while the second round prompted improvement in content validity. More work needs to be conducted to address issues related to inappropriate assumptions or bias when Internet and digital device use are considered uncommon phenomena among adolescents. Dual rounds of cognitive interviews provided substantial insight into survey interpretation when introduced to US adolescents. This validation study suggests revision points for assessing adolescent digital health literacy.