Using Computational Text Analysis to Explore Open-Ended Survey Question Responses

To capture a broader range of data than close-ended questions (often defined and delimited by the survey instrument designer), open-ended questions, such as text-based elicitations (and file-upload options for still imagery, audio, video, and other contents) are becoming more common because of the wide availability of computational text analysis, both within online survey tools and in external software applications. These computational text analysis tools—some online, some offline—make it easier to capture reproducible insights with qualitative data. This chapter explores some analytical capabilities, in matrix queries, theme extraction (topic modeling), sentiment analysis, cluster analysis (concept mapping), network text structures, qualitative cross-tabulation analysis, manual coding to automated coding, linguistic analysis, psychometrics, stylometry, network analysis, and others, as applied to open-ended questions from online surveys (and combined with human close reading).

Setting Up Education-Based “Crosswalk Analyses” on an Online Survey Platform

Practically, crosswalk analyses in education may be used to identify gaps for decision making and program planning, enable cross-system comparisons, promote cross-disciplinary work, and others. Often, crosswalk analyses require the expertise of a cross-disciplinary and/or distributed team. Setting up a crosswalk analysis on an online survey platform stands to benefit this collaborative work in ways that are more powerful than a co-edited shared online file. This chapter describes some ways to set up education-based crosswalk analyses on an online survey platform and highlights some online survey features that can enhance this work.

Conducting a Basic Self-Explicated Conjoint Analysis Online With Qualtrics®

A recent feature in the Qualtrics® Research Core Platform 2018 (or Qualtrics Research Suite) is a basic self-explicated conjoint analysis, which is a research method to understand respondent preferences in a real-world context with limited available features and selection tradeoffs at respective price points. This chapter will introduce the basic self-explicated conjoint analysis tool and how to design questions for this, how to deploy the conjoint analysis (as either part of a larger survey or as a stand-alone survey), and how to analyze and use the resulting data. This chapter will describe the assertability of the findings based on the back-end factorial statistical analysis and suggest ways to explore beyond the initial conjoint analysis.

Setting Up an Online Survey Instrument for Effective Quantitative Cross-Tabulation Analysis

The building of an online survey instrument involves sophisticated understandings of the research context, research design, research questions, and other elements. A lesser observed need is to consider what types of data analytics will be applied to the findings. With beginning-to-end online survey research suites, it becomes all the more necessary to think through the process from beginning to end in order to create an instrument that achieves all the necessary aims of the research. After all, improper online survey instrument designs will result in makework when it comes time to analyze data and will foreclose on particular data analytics opportunities. (Such instruments also will not have second or third uses after the first one-off.) This chapter explores how to build an effective online survey instrument to enable a quantitative cross tabulation analysis with the built-in analysis Qualtrics.

Setting Up and Running a Q-Methodology Study in an Online Survey Research Suite

The q-method, as a graphic (visual) elicitation, has existed since the mid-1930s. Setting up a q-method, with q-sort capabilities, in an online survey platform, extends the reach of this method, even as data has to be processed in a quantitative data analytics suite. This chapter describes the setting up of a visual q-sort and the related debriefing on the Qualtrics Research Suite. The available data may be extracted and analyzed in a basic statistical analysis tool for factors and preference clusters.

Designing, Deploying, and Evaluating Open-Access Online Delphi Studies (OAODS)

This chapter describes the work of creating multimodal open-access online Delphi studies (OAODS). These are electronic Delphi studies that do not begin with an invited group of identified experts to seat a Delphi panel but rather with self-identified domain-specific authorities active on the Social Web, with post-data-collection vetting of the participants (when knowable) and their responses. This chapter explores how to design such instruments with efficacy and nuance, and built-in tests of respondent expertise, and fraud detection, and further, how to test such instruments for efficacy, reliability, and validity, while using some of the latest features available in online survey research platforms. The platform used in this work is the Qualtrics Research Suite.

Basics to Branching Logic in Self-Administered Online Surveys

Essentially, the “branching logic” feature enables survey developers to create one survey and deploy it in different ways to different respondents; this enables survey developers to create one survey with variant question elicitations among different respondents, based on a wide variety of variables. This chapter provides a basic overview of some common types of branching logic in self-administered online survey design, development, and deployment, and highlights some considerations in effective branching logic and some precautions related to survey taker trajectories and experiences, proper survey setups for desired data collection and data analysis, survey pilot testing, and other aspects.

Applying Qualitative Matrix Coding Queries and Qualitative Crosstab Matrices for Explorations of Online Survey Data

Two computation-enabled matrix-based analytics techniques have become more available for the analysis of text data, including from online surveys. These two approaches are (1) the qualitative matrix coding query and (2) the qualitative crosstab matrix, both in NVivo 12 Plus. The first approach enables insights about the coding applied to qualitative data, and the second enables the identification of data patterns based on case (ego or entity) attributes of survey respondents. The data analytics software has integrations with multiple online survey platforms (Qualtrics and Survey Monkey currently), and the automated coding of the data from these respective platforms and other software features enable powerful data analytics. This chapter provides insights as to some of what may be discoverable using both matrix-based techniques as applied to online survey data.