Productivity or Equity? Tradeoffs in Volunteer Microtasking in Humanitarian OpenStreetMap
Article No.: 113, Pages 1 - 34
Abstract
Microtasking, the decomposition of tasks into small units of work, is prolific in human computation and crowdsourcing. Some peer production systems are beginning to leverage this same technique in volunteer contribution-based settings. While early research suggests that focusing volunteer work in this way using microtasking may be fruitful, the effects of microtasking on contributor behavior in volunteer peer production settings, like OpenStreetMap, remain unclear. This paper takes advantage of a natural experiment facilitated by the Humanitarian OpenStreetMap's Tasking Manager, a microtasking system, and employs causal inference analysis to evaluate the effects of a microtasking intervention on contributor dynamics. Our study systematically leverages a global dataset to analyze peer production dynamics, building on prior research to address a gap in peer production literature and informing the design of the microtasking interfaces. We causally show that, indeed, microtasking can be an effective intervention in peer production settings, but it may exacerbate power-law patterns that are common in such settings. By further analyzing project design decisions and characteristics, we develop implications for platform practitioners, with a focus on addressing engagement and contribution inequity issues prevalent in settings like OpenStreetMap.
References
[1]
Irasema Alcántara-Ayala. 2002. Geomorphology, natural hazards, vulnerability and prevention of natural disasters in developing countries. Geomorphology 47 (2002), 107--124.
[2]
Nezih Altay and Melissa T. Labonte. 2014. Challenges in humanitarian information management and exchange: evidence from Haiti. Disasters 38 Suppl 1 (2014), S50--72.
[3]
Justin Cheng, Jaime Teevan, Shamsi T. Iqbal, and Michael S. Bernstein. 2015. Break It Down: A Comparison of Macroand Microtasks. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015).
[4]
Chris Michael. 2014. Missing Maps: nothing less than a human genome project for cities - The Guardian. https://www.theguardian.com/cities/2014/oct/06/missing-maps-human-genome-project-unmapped-cities, Last accessed on 2022-07--14.
[5]
Camille Cobb, Ted McCarthy, Annuska Perkins, Ankitha Bharadwaj, Jared Comis, Brian Do, and Kate Starbird. 2014. Designing for the deluge: understanding & supporting the distributed, collaborative work of crisis volunteers. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, Baltimore Maryland USA, 888--899. https://doi.org/10.1145/2531602.2531712
[6]
Dan Cosley, Dan Frankowski, Loren Terveen, and John Riedl. 2007. SuggestBot: using intelligent task routing to help people find work in wikipedia. In Proceedings of the 12th international conference on Intelligent user interfaces - IUI '07. ACM Press, Honolulu, Hawaii, USA, 32. https://doi.org/10.1145/1216295.1216309
[7]
Joe Cox, Eun Young Oh, Brooke D. Simmons, Gary Graham, Anita Greenhill, Chris J. Lintott, Karen L. Masters, and JamieWoodcock. 2018. Doing Good Online: The Changing Relationships Between Motivations, Activity, and Retention Among Online Volunteers. Nonprofit and Voluntary Sector Quarterly 47 (2018), 1031 -- 1056.
[8]
Martin Dittus, Giovanni Quattrone, and Licia Capra. 2016. Analysing Volunteer Engagement in Humanitarian Mapping: Building Contributor Communities at Large Scale. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (2016).
[9]
Martin Dittus, Giovanni Quattrone, and Licia Capra. 2017. Mass Participation During Emergency Response: Eventcentric Crowdsourcing in Humanitarian Mapping. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (2017).
[10]
Featured HOT projects. 2016. Humanitarian OpenStreetMap Team Projects. https://www.hotosm.org/projects/, Last accessed on 2022-07--14.
[11]
J. Feinmann. 2014. How MSF is mapping the world's medical emergency zones. BMJ 349, dec09 13 (Dec. 2014), g7540--g7540. https://doi.org/10.1136/bmj.g7540
[12]
Joseph Feller, Brian Fitzgerald, Scott A. Hissam, and Karim R. huff. 2007. Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects.
[13]
Frank A. Farris. 2010. The Gini Index and Measures of Inequality. The American Mathematical Monthly 117, 10 (2010), 851. https://doi.org/10.4169/000298910x523344
[14]
Gary Esworthy. 2016. COMPARING OPENSTREETMAP AND WIKIPEDIA EDIT PATTERNS DURING MAJOR EVENTS. http://mars.gmu.edu/bitstream/handle/1920/10251/Esworthy_thesis_2016.pdf;sequence=1, Last accessed on 2022-07--14.
[15]
Michael F. Goodchild. 2004. The Validity and Usefulness of Laws in Geographic Information Science and Geography. Annals of the Association of American Geographers 94 (2004), 300 -- 303.
[16]
Muki Haklay. 2013. Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation. In Crowdsourcing Geographic Knowledge, Daniel Sui, Sarah Elwood, and Michael Goodchild (Eds.). Springer Netherlands, Dordrecht, 105--122. https://doi.org/10.1007/978--94-007--4587--2_7
[17]
Muki Haklay. 2016. Why is participation inequality important.
[18]
Mordechai (Muki) Haklay, Vyron Antoniou, Sofia Basiouka, Robert Soden, Vivien Deparday, Ryan Sheely, and Peter Mooney. 2018. Identifying Success Factors in Crowdsourced Geographic Information Use in Government.
[19]
Brent Hecht and Darren Gergle. 2009. Measuring self-focus bias in community-maintained knowledge repositories. In Proceedings of the fourth international conference on Communities and technologies - C&T '09. ACM Press, University Park, PA, USA, 11. https://doi.org/10.1145/1556460.1556463
[20]
Andreas Henrich and Volker Lüdecke. 2009. Measuring Similarity of Geographic Regions for Geographic Information Retrieval. In Advances in Information Retrieval, Mohand Boughanem, Catherine Berrut, Josiane Mothe, and Chantal Soule-Dupuy (Eds.). Vol. 5478. Springer Berlin Heidelberg, Berlin, Heidelberg, 781--785. https://doi.org/10.1007/978--3- 642-00958--7_85 Series Title: Lecture Notes in Computer Science.
[21]
Benjamin Herfort, Sven Lautenbach, João Porto de Albuquerque, Jennings Anderson, and Alexander Zipf. 2021. The evolution of humanitarian mapping within the OpenStreetMap community. Scientific Reports 11, 1 (Dec. 2021), 3037. https://doi.org/10.1038/s41598-021--82404-z
[22]
hotosm. 2010. hotosm introduction. https://www.hotosm.org/, Last accessed on 2022-07--14.
[23]
C. Jackson, Carsten S. Østerlund, Gabriel Mugar, Katie DeVries Hassman, and Kevin Crowston. 2015. Motivations for Sustained Participation in Crowdsourcing: Case Studies of Citizen Science on the Role of Talk. 2015 48th Hawaii International Conference on System Sciences (2015), 1624--1634.
[24]
Brian Keegan, Darren Gergle, and Noshir Contractor. 2013. Hot Off the Wiki: Structures and Dynamics of Wikipedia's Coverage of Breaking News Events. American Behavioral Scientist 57, 5 (May 2013), 595--622. https://doi.org/10.1177/ 0002764212469367
[25]
Nisar Ahmad Khan. 2023. The Effect: An Introduction to Research Design and Causality. Technometrics 65 (2023), 297 -- 298.
[26]
Shashank Khanna, Aishwarya Ratan, James Davis, and William Thies. 2010. Evaluating and improving the usability of Mechanical Turk for low-income workers in India. In Proceedings of the First ACM Symposium on Computing for Development - ACM DEV '10. ACM Press, London, United Kingdom, 1. https://doi.org/10.1145/1926180.1926195
[27]
Aniket Kittur, Ed Chi, Bryan A. Pendleton, and Bongwon Suh. 2006. Power of the Few vs . Wisdom of the Crowd : Wikipedia and the Rise of the Bourgeoisie.
[28]
Hanlin Li, Brent Hecht, and Stevie Chancellor. 2022. Measuring the Monetary Value of Online Volunteer Work. Technical Report arXiv:2205.14528. arXiv. http://arxiv.org/abs/2205.14528 arXiv:2205.14528 [cs] type: article.
[29]
Liat Clark. 2014. The race to contain West Africa's Ebola outbreak. https://www.wired.co.uk/article/ebola-openstreet-map, Last accessed on 2022-07--14.
[30]
Zarif Mahmud, Aarjav Chauhan, Dipto Sarkar, and Robert Soden. 2022. Revisiting Engagement in Humanitarian Mapping: An Updated Analysis of Contributor Retention in OpenStreetMap. CHI Conference on Human Factors in Computing Systems Extended Abstracts (2022).
[31]
Missing Maps. 2015. putting the world's vulnerable people on the map. http://www.missingmaps.org, Last accessed on 2022-07--14.
[32]
missingmaps. 2010. How to Host a Mapathon Event Planning Edition. https://www.missingmaps.org/assets/downloads/ mapathon-materials/english/How_to_Host_a_Mapathon_EN.pdf, Last accessed on 2022-07--14.
[33]
Jonathan T. Morgan and Aaron L Halfaker. 2018. Evaluating the impact of the Wikipedia Teahouse on newcomer socialization and retention. Proceedings of the 14th International Symposium on Open Collaboration (2018).
[34]
Eni Mustafaraj, Samantha Finn, Carolyn Whitlock, and Panagiotis T. Metaxas. 2011. Vocal Minority Versus Silent Majority: Discovering the Opionions of the Long Tail. In 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing. IEEE, Boston, MA, USA, 103--110. https://doi.org/10. 1109/PASSAT/SocialCom.2011.188
[35]
Abhishek Nagaraj. 2017. Information Seeding and Knowledge Production in Online Communities: Evidence from OpenStreetMap. IRPN: Open Collaborative Innovation (Topic) (2017).
[36]
Oded Nov. 2007. What motivates Wikipedians? Commun. ACM 50 (2007), 60--64.
[37]
OpenStreetMap. 2004. OpenStreetMap wikipedia. https://en.wikipedia.org/wiki/OpenStreetMap, Last accessed on 2022-07--14.
[38]
OpenStreetMap. 2010. MapSwipe app lets public help deliver medical care to most vulnerable. https://www.hotosm. org/partners/, Last accessed on 2022-07--14.
[39]
osm. 2022. Armchair mapping - OpenStreetMap Wiki. https://wiki.openstreetmap.org/wiki/Armchair_mapping, Last accessed on 2022-07--14.
[40]
Leysia Palen, Robert Soden, T. Jennings Anderson, and Mario Barrenechea. 2015. Success & Scale in a Data-Producing Organization: The Socio-Technical Evolution of OpenStreetMap in Response to Humanitarian Events. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, Seoul Republic of Korea, 4113--4122. https://doi.org/10.1145/2702123.2702294
[41]
Paul Sawers. 2018. OpenStreetMap now lets you edit maps with Microsoft's Bing Streetside imagery. https: //venturebeat.com/mobile/openstreetmap-now-lets-you-edit-maps-with-microsofts-bing-streetside-imagery/, Last accessed on 2023-07--14.
[42]
Thiago Henrique Poiani, Roberto Dos Santos Rocha, Livia Castro Degrossi, and Joao Porto De Albuquerque. 2016. Potential of Collaborative Mapping for Disaster Relief: A Case Study of OpenStreetMap in the Nepal Earthquake 2015. In 2016 49th Hawaii International Conference on System Sciences (HICSS). IEEE, Koloa, HI, 188--197. https: //doi.org/10.1109/HICSS.2016.31
[43]
Reid Priedhorsky, Mikhil Masli, and Loren Terveen. 2010. Eliciting and focusing geographic volunteer work. In Proceedings of the 2010 ACM conference on Computer supported cooperative work - CSCW '10. ACM Press, Savannah, Georgia, USA, 61. https://doi.org/10.1145/1718918.1718931
[44]
Ashesh Rambachan and Jonathan Roth. 2019. An Honest Approach to Parallel Trends.
[45]
Robinson Meyer. 2013. How Online Mapmakers Are Helping the Red Cross Save Lives in the Philippines. https://www.theatlantic.com/technology/archive/2013/11/how-online-mapmakers-are-helping-the-red-crosssave-lives-in-the-philippines/281366/, Last accessed on 2022-07--14.
[46]
Robinson Meyer. 2016. The Internet Mapmakers Helping Nepal. https://storytracker.solutionsjournalism.org/stories/ how-to-donate-time-to-help-in-nepal, Last accessed on 2022-07--14.
[47]
Robert Soden and Leysia Palen. 2014. From Crowdsourced Mapping to Community Mapping: The Post-earthquake Work of OpenStreetMap Haiti. In COOP 2014 - Proceedings of the 11th International Conference on the Design of Cooperative Systems, 27--30 May 2014, Nice (France), Chiara Rossitto, Luigina Ciolfi, David Martin, and Bernard Conein (Eds.). Springer International Publishing, Cham, 311--326. https://doi.org/10.1007/978--3--319-06498--7_19
[48]
Kate Starbird and Leysia Palen. 2011. "Voluntweeters": self-organizing by digital volunteers in times of crisis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, Vancouver BC Canada, 1071--1080. https://doi.org/10.1145/1978942.1979102
[49]
Kate Starbird and Leysia Palen. 2013. Working and sustaining the virtual "Disaster Desk". In Proceedings of the 2013 conference on Computer supported cooperative work - CSCW '13. ACM Press, San Antonio, Texas, USA, 491. https://doi.org/10.1145/2441776.2441832
[50]
Bongwon Suh, Gregorio Convertino, Ed H. Chi, and Peter Pirolli. 2009. The singularity is not near: slowing growth of Wikipedia. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration - WikiSym '09. ACM Press, Orlando, Florida, 1. https://doi.org/10.1145/1641309.1641322
[51]
The Centre for Humanitarian Data. 2010. The state of open humanitarian data. https://centre.humdata.org/wpcontent/uploads/2020/01/StateofData2020.pdf, Last accessed on 2023-07--14.
[52]
Jacob Thebault-Spieker, Aaron Halfaker, Loren G. Terveen, and Brent Hecht. 2018. Distance and Attraction: Gravity Models for Geographic Content Production. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1--13. https://doi.org/10.1145/3173574.3173722
[53]
Jacob Thebault-Spieker, Brent Hecht, and Loren Terveen. 2018. Geographic Biases are 'Born, not Made': Exploring Contributors' Spatiotemporal Behavior in OpenStreetMap. In Proceedings of the 2018 ACM Conference on Supporting Groupwork. ACM, Sanibel Island Florida USA, 71--82. https://doi.org/10.1145/3148330.3148350
[54]
Waldo Tobler. 1970. A Computer Movie Simulating Urban Growth in the Detroit Region. Economic Geography 46 (1970), 234--240.
[55]
United Nations. 2016. The Sustainable Development Goals Report. https://unstats.un.org/sdgs/report/2020/, Last accessed on 2023-07--14.
[56]
Rajan Vaish, KeithWyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch crowdsourcing: crowd contributions in short bursts of time. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2014).
[57]
Youcheng Wang and Daniel R. Fesenmaier. 2004. Modeling Participation in an Online Travel Community. Journal of Travel Research 42 (2004), 261 -- 270.
[58]
Morten Warncke-Wang, Vladislav R. Ayukaev, Brent Hecht, and Loren G. Terveen. 2015. The Success and Failure of Quality Improvement Projects in Peer Production Communities. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, Vancouver BC Canada, 743--756. https://doi.org/10. 1145/2675133.2675241
[59]
wikipedia. 2023. Wikipedia:AutoEd. https://en.wikipedia.org/wiki/Wikipedia:AutoEd, Last accessed on 2023-07--14.
[60]
wikipedia. 2023. Wikipedia:Paid-contribution disclosure. https://en.wikipedia.org/wiki/Wikipedia:Paid-contribution_disclosure, Last accessed on 2022-07--14.
[61]
Haiyi Zhu, Robert Kraut, and Aniket Kittur. 2012. Effectiveness of shared leadership in online communities. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 407--416.
Index Terms
- Productivity or Equity? Tradeoffs in Volunteer Microtasking in Humanitarian OpenStreetMap
Recommendations
Large-scale Volunteer Engagement in Humanitarian Mapping
CSCW '16 Companion: Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing CompanionOrganisers of large crowdsourcing initiatives need to con-sider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial contributor ex-perience plays an important role in this, particularly when contributions ...
Analysing Volunteer Engagement in Humanitarian Mapping: Building Contributor Communities at Large Scale
CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social ComputingOrganisers of large-scale crowdsourcing initiatives need to consider how to produce outcomes with their projects, but also how to build volunteer capacity. The initial project experience of contributors plays an important role in this, particularly when ...
Value Tensions in OpenStreetMap: Openness, Membership, and Policy in Online Communities
CSCWThe social life and long-term trajectories of online peer production communities are shaped and animated in part by value tensions that arise when distributed, heterogeneous participants are brought together into collaboration. This study of ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 April 2024
Published in PACMHCI Volume 8, Issue CSCW1
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Wisconsin Alumni Research Foundation
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 339Total Downloads
- Downloads (Last 12 months)339
- Downloads (Last 6 weeks)39
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in