Please use this identifier to cite or link to this item: http://hdl.handle.net/11718/25913
Full metadata record
DC FieldValueLanguage
dc.contributor.authorDeodhar, Swanand J.
dc.contributor.authorBabar, Yash
dc.contributor.authorBurtch, Gordon
dc.date.accessioned2022-12-07T06:15:36Z
dc.date.available2022-12-07T06:15:36Z
dc.date.issued2022-12
dc.identifier.citationDeodhar, S. J., Babar, Y., & Burtch, G. (2022). The Influence of Status on Evaluations: Evidence from Online Coding Contests. MIS Quarterly, 46(4), 2085–2110. https://doi.org/10.25300/MISQ/2022/16178en_US
dc.identifier.issn02767783
dc.identifier.urihttp://hdl.handle.net/11718/25913
dc.description.abstractIn many instances, online contest platforms rely on contestants to ensure submission quality. This scalable evaluation mechanism offers a collective benefit. However, contestants may also leverage it to achieve personal, competitive benefits. Our study examines this tension from a status-theoretic perspective, suggesting that the conflict between competitive and collective benefits, and the net implication for evaluation efficacy, is influenced by contestants’ status. On the one hand, contestants of lower status may be viewed as less skilled and hence more likely to make mistakes. Therefore, low-status contestants may attract more evaluations if said evaluations are driven predominantly by an interest in collective benefits. On the other hand, if evaluations are driven largely by an interest in personal, competitive benefits, a low-status contestant makes for a less attractive target and hence may attract fewer evaluations. We empirically test these competing possibilities using a dataset of coding contests from Codeforces. The platform allows contestants to assess others’ submissions and improve evaluations (a collective benefit) by devising test cases (hacks) in addition to those defined by the contest organizer. If a submission is successfully hacked, the hacker earns additional points, and the target submission is eliminated from the contest (a competitive benefit). We begin by providing qualitative evidence based on semi-structured interviews conducted with contestants spanning the status spectrum at Codeforces. Next, we present quantitative evidence exploiting a structural change at Codeforces wherein many contestants experienced an arbitrary status reduction unrelated to their performance because of sudden changes to the platform’s color-coding system around contestant ratings. We show that status-loser contestants received systematically more evaluations from other contestants, absent changes in their short-run submission quality. Finally, we show that the excess evaluations allocated toward affected contestants were less effective, indicating status-driven evaluations as potentially less efficacious. We discuss the implications of our findings for managing evaluation processes in online contests.en_US
dc.language.isoenen_US
dc.publisherMIS Quarterlyen_US
dc.relation.ispartofMIS Quarterlyen_US
dc.subjectEvaluationen_US
dc.subjectarbitrary status changeen_US
dc.subjectcompetitive codingen_US
dc.subjectonline contestsen_US
dc.titleThe Influence of Status on Evaluations: Evidence from Online Coding Contestsen_US
dc.typeArticleen_US
Appears in Collections:Journal Articles

Files in This Item:
File Description SizeFormat 
The_Influence_of_Status_on_Evaluations_Evidence_from_Online_Coding_Contests.pdf
  Restricted Access
821.21 kBAdobe PDFView/Open Request a copy


Items in IIMA Institutional Repository are protected by copyright, with all rights reserved, unless otherwise indicated.