Reviews and Self-Selection Bias with Operational Implications

Reviews for products and services written by previous consumers have become an influential input to the purchase decision of customers. Many service businesses monitor the reviews closely for feedback as well as detecting service flaws, and they have become part of the performance review for service managers with rewards tied to improvement in the aggregate rating. Many empirical papers have documented a bias in the aggregate ratings, arising because of customers’ inherent self-selection in their choices and bounded rationality in evaluating previous reviews. Although there is a vast empirical literature analyzing reviews, theoretical models that try to isolate and explain the bias in ratings are relatively few. Assuming consumers simply substitute the average rating that they see as a proxy for quality, we give a precise characterization of the self-selection bias on ratings of an assortment of products when consumers confound ex ante innate preferences for a product or service with ex post experience and service quality and do not separate the two. We develop a parsimonious choice model for consumer purchase decisions and show that the mechanism leads to an upward bias, which is more pronounced for niche products. Based on our theoretical characterization, we study the effect on pricing and assortment decisions of the firm when potential customers purchase based on the biased ratings. Our results give insights into how quality, prices, and customer feedback are intricately tied together for service firms. This paper was accepted by David Simchi-Levi, operations management.

Do Bond Issuers Shop for Favorable Credit Ratings?

This paper provides evidence of ratings shopping in the corporate bond market. By estimating systematic differences in agencies’ biases about any given firm’s bonds, I show that new bonds are more likely to be rated by agencies that are positively biased toward the firm—a pattern that is strongest among bonds that have only one rating. The paper also shows that issuers often delay less favorable ratings until after a bond is sold. Consistent with theoretical models of ratings shopping, these effects are strongest among more complex bonds that are more difficult to rate. Bonds with upward-biased ratings are more likely to be downgraded and default, but investors account for this bias and demand higher yields when buying these bonds. This paper was accepted by Gustavo Manso, finance.

Social Structures and Reputation in Expert Review Systems

We model an expert review system where two producers competing for market share each are evaluated by two raters. Employing economics experiments, the paper examines how the rater’s incentive to provide objective feedback can be distorted in the presence of social ties and different penalty structures for assigning unobjective ratings. The results reject the self-interested model. We find that raters assign more biased ratings to help the producer they know compete, and this distortion is exacerbated when the reputation cost for rating unobjectively is lowered. Counterintuitively, when both of the raters know the same producer, the likelihood of biased ratings drops significantly. To explain the empirical regularities, we develop a behavioral economics model and show that the rater’s utility function should account not only for social preferences toward the producer, but also the rater’s psychological aversion toward favoring a producer more than the other rater. Our findings demonstrate that it is critical for policymakers to be cognizant of the nonpecuniary factors that can influence behavior in expert review systems. This paper was accepted by John List, behavioral economics.

Online travel review rating scales and effects on hotel scoring and competitiveness

Purpose The purpose of this paper is to determine whether different scales and ways to collect reviews and ratings found on online travel agencies (OTAs) can affect hotels, and whether hotels obtain the same or different evaluations. Design/methodology/approach Hotel ratings from five OTAs in four European markets were collected and compared in pairs. An initial comparison was made with the hotel scores of each OTA to show what a typical user would see. Then, a rescaled score (0-10) was used to compare all the OTA scales appropriately and to distinguish between what customers observe and what the reality is. Findings The results reveal that Booking.com that uses a scale (2.5-10) and Agoda with a scale (2-10) seem to give higher rating scores than Atrapalo (1-10), Travel Republic (0-10) and hotel reservation service (1-10). However, when the scores are rescaled (0-10), the worst ratings are found on Booking.com followed by Agoda. Practical implications OTAs should include, next to the scores, the scale used to rate hotels so as to provide users with better and clearer information. Moreover, rating questionnaires should match the verbal denominations with their numerical values to avoid biased ratings. Social implications OTAs and hotel managers are losing information provided by customers because customers are not aware of the scale when rating hotels. Moreover, hotel ratings are used by potential customers to obtain a clearer image of an establishment. However, if some hotels are being overrated by some scales, customers might have higher expectations, which may not be met. Originality/value The unique rating scales of Booking.com and Agoda provide additional insights into their hotel evaluations, which seem to be apparently higher when in fact they are not.

Analyst Promotions within Credit Rating Agencies: Accuracy or Bias?

We estimate Moody’s preference for accurate versus biased ratings using hand-collected data on the internal labor market outcomes of its analysts. We find that accurate analysts are more likely to be promoted and less likely to depart. The opposite is true for analysts who downgrade more frequently, who assign ratings below those predicted by a ratings model, and whose downgrades are associated with large negative market reactions. Downgraded firms are also more likely to be assigned a new analyst. These patterns are consistent with Moody’s balancing its desire for accuracy against its corporate clients’ desire for higher ratings.

Examining Rater Effects of the TGMD-2 on Children With Intellectual Disability

The purpose of this study was to investigate rater effects on the TGMD-2 when it applied to children with intellectual disability. A total of 22 children with intellectual disabilities participated in this study. Children’s performances in each of 12 subtests of the TGMD-2 were recorded via video and scored by three adapted physical activity specialists who have expertise in the TGMD-2. Two advanced measurement theories, Generalizability-theory (G-theory) and many-facet Rasch model (MFRM), were applied in data analyses. There were relatively large variances attributed to rater effects on the scores of the TGMD-2 awarded to children with intellectual disabilities. The severity of each rater significantly differed across all subtests of the TGMD-2. There was a set of biased ratings interacted with measurement conditions of the TGMD-2.