Soylent
Michael S. Bernstein, Greg Little, Robert C. Miller et al. · ACM Symposium on User Interface Software and Technology (UIST) · 2010
This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.
Cited by 686 works. doi:10.1145/1866029.1866078
Authors
- Michael S. Bernstein
- Greg Little
- Robert C. Miller
- Björn Hartmann
- Mark Steven Ackerman
- David R. Karger
- Katrina Panovich
References
- Labeling images with a computer game
- Utility data annotation with Amazon Mechanical Turk
- Cheap and fast---but is it good?
- Awareness and coordination in shared workspaces
- Crowdsourcing user studies with Mechanical Turk
- Techniques for automatically correcting words in text
- Who are the crowdworkers?
- Financial incentives and the "performance of crowds"
- Financial incentives and the "performance of crowds"
- VizWiz