Yesterday, Mechanical Turk announced a fee increase taking effect on July 22, 2015. Currently, Amazon charges a 10% commission on top of what is paid to workers. In one month, this will increase to at least 20%. But for jobs with 10 or more assignments (e.g., 10 or more survey takers), the new commission will be 40%. With this quadrupled commission, survey costs will increase by 27.3%.
(Image source: The original Mechanical Turk via Wikimedia Commons)
@morewedge Quadrupling the commission, not cost. $.25 hit used to be $.275, is now $.35, a 27% increase, not 300% #mturkgate
— ᶨᶬᶵ ᵴᴏᴏᴚ (@BattleCobra90K) June 23, 2015
To put new #mturk fees in perspective. Here's a quick chart of cost/100 workers at varying reward amounts. #mturkgate pic.twitter.com/I8LXNEsu7p
— Thomas Leeper (@thosjleeper) June 23, 2015
Update 3:10pm CDT 6/23:
A better graph of #mturkgate. Keep in mind total pay in most psychology studies is much >$100 ($8/hr x 200+ turkers) pic.twitter.com/Gwwuzd5J2t
— Carey Morewedge (@morewedge) June 23, 2015
Update 1:50pm CDT 6/24: There are many ways to frame the Mechanical Turk pricing change, and they might inspire different reactions. Thomas Leeper’s got a pretty exhaustive list on his blog.
Framing Numbers: "How #mturkgate Is A Good Reminder To Practice Our Fractions" on my blog: http://t.co/ky52jAnfJ5 #mturk #math
— Thomas Leeper (@thosjleeper) June 24, 2015
Fee increases will affect workers
Mechanical Turk workers were concerned when they heard the news. More money spent on fees to Mechanical Turk might mean less money available to pay workers, leading to fewer HITs or lower wages.
What MTurk workers are saying about the @amazonmturk commission fee changes: http://t.co/rwHokraFib
— Rochelle (@Rochelle) June 23, 2015
@ashleydavis07 After int'l requesters were blocked, HITs dipped (mturk tracker), I am scared of what happens July 22nd. #mturkgate
— Kristy@TurkerNation (@TurkerNational) June 23, 2015
@joshdeleeuw @ecsalomon Definitely workers. Already heard worries about how to pay grocery bill, utilities, rent if MTurk income drops.
— Rochelle (@Rochelle) June 23, 2015
@thosjleeper @page_gould @ecsalomon correct, hashtag was coined by a worker #mturkgate
— Kristy@TurkerNation (@TurkerNational) June 23, 2015
Update 1:50pm CDT 6/24: Workers are organizing responses to Amazon, hoping that their concerns about lost income will be addressed.
@ecsalomon Might add the http://t.co/4ekNBw1IaX campaign page for this: http://t.co/VvpuloksC1
— Rochelle (@Rochelle) June 24, 2015
Keep The Base Commission Where it Is (worker petition) #mTurkGate https://t.co/GVmxwUrxKq
— Kristy@TurkerNation (@TurkerNational) June 24, 2015
Academic researchers affected as well
Academic requestors were largely disappointed and angry about the news.
So long @amazonmturk, it's been a great ride. MTurk increasing commission from 10% to 40% for academic researchers https://t.co/rql3ENSMlB
— Gilad Feldman (@mgto_org) June 23, 2015
.@amazonmturk please sum up all innovations since 2008 that fee increase will help continue. Im sure itll fit in a single tweet. #mturkgate
— Andrew Long (@andrewrlong) June 23, 2015
The language in Amazon’s email to requestors suggests that they are specifically targeting survey requesters and marketers with this increase:
"HITs requiring 10 or more assignments will incur an additional 20% commission above the base commission. These HITs comprise less than 0.3% of Mechanical Turk’s overall HITs, and are most commonly used by Requesters to survey Workers and conduct market research."
Still, some are hoping for an exemption for academic research. Update 3:10pm CDT 6/22: Amazon may be considering an academic pricing scheme.
Message I sent to @amazonmturk about their new fee policy. May be a long shot, but I hope they consider it. pic.twitter.com/PGGGop7VIh
— Aaron Zimbelman (@azimbelman) June 23, 2015
Say, @amazonmturk, what the heck? Counteroffer: certified academic accounts, as we always need HITs > 9 (via @joboegershausen). #mturkgate
— Rhiannon MacDonnell (@Rhiannon) June 23, 2015
Retweet this to let Amazon Mechanical Turk know you'd like academic pricing #mturk #mturkgate @amazonmturk
— Dan Goldstein (@dggoldst) June 23, 2015
Many are worried that the fee changes will exacerbated existing funding disparities between labs:
@amazon's #mturk change obv meant 2 push out academics. Burden 4 early career/non-Ivy scientists #backtousingundergrads #mturkgate @ksearles
— Matt Hitt (@matthewhitt) June 23, 2015
Academic requester's perspective: Amazon's policy shifts perpetuate the monied and American hegemonies of science #mturkgate
— Elizabeth Page-Gould (@page_gould) June 23, 2015
Others think that reaction to the fee increase is overblown or off-target:
I wrote a blog post on #mturkgate http://t.co/1z5rAUdhug Doubtless full of inaccuracies, but I think the discussion is worthwhile. Comment!
— Nick Brown (@sTeamTraen) June 23, 2015
I don't see why academics think they should be exempt from the rate hike. They do high volume HITs same as anyone else. #mturkgate
— Kurt Luther (@kurtluther) June 23, 2015
"Amazon is exploiting us." -Researchers paying participants sub-minimum wage rates to take surveys. #mturkgate
— Jimmy Ivory (@jivory) June 23, 2015
Mechanical Turk alternatives
#mturkgate has brought attention to the many other crowdworking platforms available to researchers and inspired some academics to collaborate on new alternatives.
open github issue for discussing building an alternative to AMT (not limited to psiturk) https://t.co/da49aX3rJT
— psiturk (@psiturk) June 23, 2015
A group discussing MTURK alternatives for social science researchers in light of #mturkgate https://t.co/881kbgbDcN
— Carey Morewedge (@morewedge) June 23, 2015
Timely given Mechanical Turk fee hike: @msbernst et al. effort @StanfordHCI to develop new crowdsourcing platform: https://t.co/BeKcZCbwtI
— Brendan Nyhan (@BrendanNyhan) June 23, 2015
Academics: any experience with @CrowdFlower? Are they a viable alternative to Amazon? #mturkgate
— Yoel Inbar (@yorl) June 23, 2015
Run your first online study on Prolific with a free 10GBP and 0% commission. https://t.co/5eRkzD2opW @ProfRichCrisp @GeorginaRdeM @AbiPlayer
— Prolific (@ProlificAc) June 23, 2015
Re: Mturk. Here's the failed 2006 app for an international participant pool: https://t.co/kbFC6cuHZK Want to do it? pic.twitter.com/CjPJhIq06d
— Brian Nosek (@BrianNosek) June 23, 2015
hey, who wants to help me develop a open alternative to MTurk? maybe use @square or PayPal to transfer funds. if u build it they will come
— todd gureckis (@todd_gureckis) June 23, 2015
Anyone know anything about these #mturkalternatives http://t.co/kFTxMHdTAn
— Mina Cikara (@profcikara) June 23, 2015
There’s also this SSRN paper comparing alternative platforms from April, though we should expect changes to the various user bases as a result of workers and requesters moving sites in response to the fee hike. Update 1:50pm CDT 6/24: Here are a few key take-aways from the paper:
Other sites generally have large workforces. Though I think Crowdflower has bad rep among workers? @TurkerNational pic.twitter.com/SQIjKZV9uI
— Erika Salomon (@ecsalomon) June 24, 2015
Many more workers on on other platforms (up to 93% on RapidWorkers) failed attention checks. Cronbach alphas were lower but generally > .7
— Erika Salomon (@ecsalomon) June 24, 2015
Why the change?
Update 7:05pm CDT 6/23: Although most of the reaction has focused on the effects the new policy will have have on workers and requesters, some have speculated about the reasons for the change.
Murmurings on Reddit that MTurk commission change aimed at behavioral research HITs, anyone know anything? #mturkgate
— Kathleen Searles (@kesearles) June 23, 2015
@Rochelle @ecsalomon @ProlificAc I suspect AMZN may lose money on MTurk. It's a small operation & they can't have reliable figures on costs.
— Nick Brown (@sTeamTraen) June 23, 2015
Meanwhile, the Wall Street Journal reports the official line from Amazon as
"The price increase is necessary to continue growing the Amazon Mechanical Turk marketplace and innovating on behalf of our customers,” according to an Amazon spokeswoman. “These changes will help us better serve our community of Requesters and Workers."
Media picks up #mturkgate story
Update 7:05pm CDT 6/23: Media outlets have begun covering the response to Mechanical Turk’s pricing change. The articles below include interviews with workers and researchers expressing their concerns abut the policy. The story has been covered in:
Update 1:50pm CDT 6/24: A few more media outlets have covered the price change, mostly building on the Wall Street Journal‘s coverage:
What is the future of online survey research?
Amazon’s announcement has raised several issues online researchers have been grappling with since the trend exploded several years ago:
- What obligations do researchers have to workers, and what are the best ways to meet them?
- How do the samples and data obtained from Mechanical Turk—and its alternatives—compare to those from traditional methods?
- How can we use online survey methods to improve representative sampling?
#mturkgate will hopefully be a catalyst for addressing these questions and more.
Please chime in by commenting below.